How to Encrypt and upload data using selling-partner-api in Amzon using .net

Question

I want to use selling-partner-api-docs for .Net . I found one reference from below url but that is use Java example coding : https://github.com/amzn/selling-partner-api-docs/blob/main/guides/use-case-guides/feeds-api-use-case-guide-2020-09-04.md#step-2-encrypt-and-upload-the-feed-data

But i want to use .Net coding can any one suggest of .Net coding of below java coding part

    import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.PipedInputStream;
import java.io.PipedOutputStream;
import java.nio.charset.StandardCharsets;

import com.amazon.spapi.documents.UploadHelper;
import com.amazon.spapi.documents.UploadSpecification;
import com.amazon.spapi.documents.exception.CryptoException;
import com.amazon.spapi.documents.exception.HttpResponseException;
import com.amazon.spapi.documents.impl.AESCryptoStreamFactory;

/* We want to maintain encryption at rest, so do not write unencrypted data to disk.  This is bad:
InputStream source = new FileInputStream(new File("/path/to/myFeed.xml"));

Instead, if your data can fit in memory, you can create an InputStream from a String (see encryptAndUpload_fromString()).
Otherwise, you can pipe data into an InputStream using Piped streams (see encryptAndUpload_fromPipedInputStream()).
 */
public class UploadExample {
  private final UploadHelper uploadHelper = new UploadHelper.Builder().build();

  // key, initializationVector, and url are returned by the createFeedDocument operation.
  public void encryptAndUpload_fromString(String key, String initializationVector, String url) {
    AESCryptoStreamFactory aesCryptoStreamFactory =
      new AESCryptoStreamFactory.Builder(key, initializationVector)
      .build();

    // This contentType must be the same value that was provided to createFeedDocument.
    String contentType = String.format("text/plain; charset=%s", StandardCharsets.UTF_8);

    // The character set must be the same one that is specified in contentType.
    try
      (InputStream source = new ByteArrayInputStream("my feed data".getBytes(StandardCharsets.UTF_8))) {
        UploadSpecification uploadSpec =
          new UploadSpecification.Builder(contentType, aesCryptoStreamFactory, source, url)
          .build();

        uploadHelper.upload(uploadSpec);
      }
    catch (CryptoException | HttpResponseException | IOException e) {
      // Handle exception.
    }
  }

  // key, initializationVector, and url are returned from createFeedDocument.
  public void encryptAndUpload_fromPipedInputStream(String key, String initializationVector, String url) {
    AESCryptoStreamFactory aesCryptoStreamFactory =
      new AESCryptoStreamFactory.Builder(key, initializationVector)
      .build();

    // This contentType must be the same value that was provided to createFeedDocument.
    String contentType = String.format("text/plain; charset=%s", StandardCharsets.UTF_8);

    try
      (PipedInputStream source = new PipedInputStream()) {
        new Thread(
          new Runnable() {
          public void run() {
            try
              (PipedOutputStream feedContents = new PipedOutputStream(source)) {
                // The character set must be the same one that is specified in contentType.
                feedContents.write("my feed data\n".getBytes(StandardCharsets.UTF_8));
                feedContents.write("more feed data".getBytes(StandardCharsets.UTF_8));
              }
            catch (IOException e) {
              // Handle exception.
            }
          }
        }).start();

        UploadSpecification uploadSpec =
          new UploadSpecification.Builder(contentType, aesCryptoStreamFactory, source, url)
          .build();

        uploadHelper.upload(uploadSpec);
      }
    catch (CryptoException | HttpResponseException | IOException e) {
    
    }
  }
}

Edit -----------------------------------------------------

This is what I have tried and this is what I have got.

STEP 1

REQUEST URL: https://sellingpartnerapi-na.amazon.com/feeds/2020-09-04/documents REQUEST BODY:{"contentType":"text/plain;charset=utf-8"} Request Headers = {Host: sellingpartnerapi-na.amazon.com x-amz-date: 20210203T120516Z Authorization: AWS4-HMAC-SHA256 Credential=XXXXXXXX/20210203/us-east-1/execute-api/aws4_request, SignedHeaders=host;x-amz-date, Signature=XXXX

The Credential and Signature are created on the partner portal.

RESPONSE STEP 1

{"payload": {"encryptionDetails":{"standard":"AES","initializationVector":"TTAVo5bUDNfuk7KPzgm+ow==", "key":"GrpKm3UIvxiM5xUTlzaCC9xJFORMX41chAKUk0G6Cbg="}, "feedDocumentId":"amzn1.tortuga.3.9968967c-048c-4e8b-a6c1-ffd764f005d4.T508PJ0OCPKJ3", "url":"https://tortuga-prod-na.s3-external-1.amazonaws.com/%2FNinetyDays/amzn1.tortuga.3.9968967c-048c-4e8b-a6c1-ffd764f005d4.T508PJ0OCPKJ3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20210203T114111Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=300&X-Amz-Credential=AKIA5U6MO6RANYPNEUPL%2F20210203%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=1fd8b69523c06d76664c22c4093be5e8adc187436f7119aa9d4b51302cc8ae84"}}

STEP 2: In step 2 I am using the URL coming from the first Step Response but it is not getting me result.

REQUEST URL:

https://tortuga-prod-na.s3-external-1.amazonaws.com/%2FNinetyDays/amzn1.tortuga.3.9968967c-048c-4e8b-a6c1-ffd764f005d4.T508PJ0OCPKJ3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20210203T114111Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=300&X-Amz-Credential=AKIA5U6MO6RANYPNEUPL%2F20210203%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=1fd8b69523c06d76664c22c4093be5e8adc187436f7119aa9d4b51302cc8ae84

See the Signature and the Credential here are coming different than one we have got from the response of Step 1

RESPONSE FROM STEP 2

<?xml version="1.0" encoding="UTF-8"?>

-<Error>

<Code>SignatureDoesNotMatch</Code>

<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>

<AWSAccessKeyId>AKIA5U6MO6RANYPNEUPL</AWSAccessKeyId>

<StringToSign>AWS4-HMAC-SHA256 20210203T114111Z 20210203/us-east-1/s3/aws4_request 057d93b83f8254c64b8ffccdfb885b79e5d96c0d2045c27732fc42ae722e335e</StringToSign>

<SignatureProvided>1fd8b69523c06d76664c22c4093be5e8adc187436f7119aa9d4b51302cc8ae84</SignatureProvided>

<StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 32 31 30 32 30 33 54 31 31 34 31 31 31 5a 0a 32 30 32 31 30 32 30 33 2f 75 73 2d 65 61 73 74 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 30 35 37 64 39 33 62 38 33 66 38 32 35 34 63 36 34 62 38 66 66 63 63 64 66 62 38 38 35 62 37 39 65 35 64 39 36 63 30 64 32 30 34 35 63 32 37 37 33 32 66 63 34 32 61 65 37 32 32 65 33 33 35 65</StringToSignBytes>

<CanonicalRequest>PUT //NinetyDays/amzn1.tortuga.3.9968967c-048c-4e8b-a6c1-ffd764f005d4.T508PJ0OCPKJ3 X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA5U6MO6RANYPNEUPL%2F20210203%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210203T114111Z&X-Amz-Expires=300&X-Amz-SignedHeaders=content-type%3Bhost content-type:text/plain; charset=utf-8 host:tortuga-prod-na.s3-external-1.amazonaws.com content-type;host UNSIGNED-PAYLOAD</CanonicalRequest>

<CanonicalRequestBytes>50 55 54 0a 2f 2f 4e 69 6e 65 74 79 44 61 79 73 2f 61 6d 7a 6e 31 2e 74 6f 72 74 75 67 61 2e 33 2e 39 39 36 38 39 36 37 63 2d 30 34 38 63 2d 34 65 38 62 2d 61 36 63 31 2d 66 66 64 37 36 34 66 30 30 35 64 34 2e 54 35 30 38 50 4a 30 4f 43 50 4b 4a 33 0a 58 2d 41 6d 7a 2d 41 6c 67 6f 72 69 74 68 6d 3d 41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 26 58 2d 41 6d 7a 2d 43 72 65 64 65 6e 74 69 61 6c 3d 41 4b 49 41 35 55 36 4d 4f 36 52 41 4e 59 50 4e 45 55 50 4c 25 32 46 32 30 32 31 30 32 30 33 25 32 46 75 73 2d 65 61 73 74 2d 31 25 32 46 73 33 25 32 46 61 77 73 34 5f 72 65 71 75 65 73 74 26 58 2d 41 6d 7a 2d 44 61 74 65 3d 32 30 32 31 30 32 30 33 54 31 31 34 31 31 31 5a 26 58 2d 41 6d 7a 2d 45 78 70 69 72 65 73 3d 33 30 30 26 58 2d 41 6d 7a 2d 53 69 67 6e 65 64 48 65 61 64 65 72 73 3d 63 6f 6e 74 65 6e 74 2d 74 79 70 65 25 33 42 68 6f 73 74 0a 63 6f 6e 74 65 6e 74 2d 74 79 70 65 3a 74 65 78 74 2f 70 6c 61 69 6e 3b 20 63 68 61 72 73 65 74 3d 75 74 66 2d 38 0a 68 6f 73 74 3a 74 6f 72 74 75 67 61 2d 70 72 6f 64 2d 6e 61 2e 73 33 2d 65 78 74 65 72 6e 61 6c 2d 31 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 0a 0a 63 6f 6e 74 65 6e 74 2d 74 79 70 65 3b 68 6f 73 74 0a 55 4e 53 49 47 4e 45 44 2d 50 41 59 4c 4f 41 44</CanonicalRequestBytes>

<RequestId>48A2CCE3EFA66E89</RequestId>

<HostId>hiZxZwoTgGG4PBvGLchnKV94AA57zzGqnHh5BbTCIAt1ubD47O+8uQMClkDDBoJBgiXgVb57TRE=</HostId>

</Error>

Coast

Stack Overflow does not exist to translate code for you. – Verenaverene

Yeah, but this being an API then there might be a way to make VS make the code your looking for via a wsdl – Thierry

Anyone can please tell me on which URL we have to push this encrypted data. Posting to this { access-key-when-create-IAM }/20201013/us-east-1/execute-api/aws4_request Drops me this error AuthorizationHeaderMalformedThe authorization header is malformed; incorrect service "execute-api". This endpoint belongs to "s3" – Fabaceous

Great to see you questioned this, helped me to save a lot of my time. Cheers. – Ari

I am almost in the same situation, but I have the following error when sending the first request of creating feed: #64560663 Did anyone get the response above? If so what action solve the issue? I have stucked in the authentication and can not get further. – Levitate

The link at the4 top of the question returns a 404 error now. I tried searching, but it looks like Amazon removed it. – Brigandage

Answer 1

Hopefully this helps some c sharpers out there for Step #2 https://github.com/amzn/selling-partner-api-docs/blob/main/guides/use-case-guides/feeds-api-use-case-guide-2020-09-04.md#step-2-encrypt-and-upload-the-feed-data

Assuming you have received an OK response for Step #1

The response for Step #1 will look something like this:

{
  "payload":
  {
    "feedDocumentId":"amzn1.tortuga.3.920614b0-fc4c-4393-b0d9-fff175300000.T29XK4YL08B2VM",
    "url":"https://tortuga-prod-na.s3.amazonaws.com/%2FNinetyDays/amzn1.tortuga.3.920614b0-fc4c-4393-b0d9-fff175300000.T29XK4YL08B2VM?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200919T035824Z&X-Amz-SignedHeaders=<headers>&X-Amz-Expires=300&X-Amz-Credential=<credential>&X-Amz-Signature=<signature>",
    "encryptionDetails":
    {
      "standard":"AES",
      "initializationVector":"kF3bZt0FSv6JQEimfEJD8g==",
      "key":"5EZo/P06OGF0UAy8QuOnMIaQbkAvYBru6EGsFvK8wJ2="
    }
}

Convert EncryptionDetails.Key and EncryptionDetails.InitializationVector to bytes and read your flat file (feed) into a string variable...

var key = Convert.FromBase64String(createFeedDocumentResponse.Payload.EncryptionDetails.Key);
var iv = Convert.FromBase64String(createFeedDocumentResponse.Payload.EncryptionDetails.InitializationVector);
string feedData = File.ReadAllText(@"C:\temp\AmazonFlatFileTest.txt");

Encrypt the feed using AES w/ the key and iv variables above...

Here's an encrypt and decrypt function I took from Microsoft and altered slightly (credit to https://learn.microsoft.com/en-us/dotnet/api/system.security.cryptography.aescryptoserviceprovider?view=net-5.0)...

private byte[] EncryptStringToBytes_Aes(string plainText, byte[] key, byte[] initializationVector)
{
  // Check arguments.
  if (plainText == null || plainText.Length <= 0)
    throw new ArgumentNullException("plainText");
  if (key == null || key.Length <= 0)
    throw new ArgumentNullException("Key");
  if (initializationVector == null || initializationVector.Length <= 0)
    throw new ArgumentNullException("initializationVector");
  byte[] encrypted;

  // Create an Aes object
  // with the specified key and IV.
  using (Aes aesAlg = Aes.Create())
  {
    aesAlg.Key = key;
    aesAlg.IV = initializationVector;

    // Create an encryptor to perform the stream transform.
    ICryptoTransform encryptor = aesAlg.CreateEncryptor(aesAlg.Key, aesAlg.IV);

    // Create the streams used for encryption.
    using (MemoryStream msEncrypt = new MemoryStream())
    {
      using (CryptoStream csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write))
      {
        using (StreamWriter swEncrypt = new StreamWriter(csEncrypt, Encoding.UTF8))
        {
          //Write all data to the stream.
          swEncrypt.Write(plainText);
        }
        encrypted = msEncrypt.ToArray();
      }
    }
  }

  // Return the encrypted bytes from the memory stream.
  return encrypted;
}

And the decrypt function... You will need this in Step #5 decrypt and process the results.

EDIT: It's noticed for larger inputs, Amazon will use GZIP compression with their response (Payload.CompressionAlgorithm). Included 'compressionAlgorithm' parameter to the DecryptStringFromBytes_Aes method.

private string DecryptStringFromBytes_Aes(byte[] cipherText, byte[] Key, byte[] initializationVector, string compressionAlgorithm)
{
  // Validate Compression Algorithm
  var isGzip = string.Equals(compressionAlgorithm, "GZIP", StringComparison.OrdinalIgnoreCase);
  var compressionAlgorithmValid = compressionAlgorithm == null || isGzip;

  if (!compressionAlgorithmValid)
  {
    throw new InvalidOperationException($"Unexpected CompressionAlgorithm encounted. compressionAlgorithm = {compressionAlgorithm}");
  }

  // Check arguments.
  if (cipherText == null || cipherText.Length <= 0)
    throw new ArgumentNullException("cipherText");
  if (Key == null || Key.Length <= 0)
    throw new ArgumentNullException("Key");
  if (initializationVector == null || initializationVector.Length <= 0)
    throw new ArgumentNullException("IV");

  // Declare the string used to hold
  // the decrypted text.
  string plaintext = null;

  // Create an Aes object
  // with the specified key and IV.
  using (Aes aesAlg = Aes.Create())
  {
    aesAlg.Key = Key;
    aesAlg.IV = initializationVector;

    // Create a decryptor to perform the stream transform.
    ICryptoTransform decryptor = aesAlg.CreateDecryptor(aesAlg.Key, aesAlg.IV);

    // Create the streams used for decryption.
    using (MemoryStream msDecrypt = new MemoryStream(cipherText))
    {
      using (CryptoStream csDecrypt = new CryptoStream(msDecrypt, decryptor, CryptoStreamMode.Read))
      {
        if (isGzip)
        {
          using (var decompressedFileStream = new MemoryStream())
          {

            using (GZipStream decompressionStream = new GZipStream(csDecrypt, CompressionMode.Decompress))
            {
              decompressionStream.CopyTo(decompressedFileStream);
              decompressedFileStream.Position = 0;

              using (var writer = new StreamReader(decompressedFileStream))
              {
                plaintext = writer.ReadToEnd();
              }
            }

          }
        }
        else
        {
          using (StreamReader srDecrypt = new StreamReader(csDecrypt, Encoding.UTF8))
          {
            // Read the decrypted bytes from the decrypting stream
            // and place them in a string.
            plaintext = srDecrypt.ReadToEnd();
          }
        }
      }
    }
  }

  return plaintext;
}

With the returned result from EncryptStringToBytes_Aes, you can now upload to S3 url (provided to you in Payload from Step #1). Here is an example using the library Rest Sharp (which some Amazon employee seems to like judging by the reference in Amazon's C# client example).

NB: The Content Type must match the content type in your CreateFeedDocument request (Step #1).

Here is a function you can use to upload to S3...

private async Task UploadFile(byte[] bytes, string url)
{
  var contentType = "text/plain; charset=utf-8"; // this should be the same as what was used in Step #1 (in the CreateFeedDocument API request)

  RestClient restClient = new RestClient(url);
  IRestRequest restRequest = new RestRequest(Method.PUT);
  restRequest.AddParameter(contentType, bytes, ParameterType.RequestBody);

  var response = await restClient.ExecuteAsync(restRequest);

  if (!response.IsSuccessful)
  {
    // your error logic
  }

  // success. Move to Step #3
}

Happy days...

Abeyant

Anyone can please tell me on which URL we have to push this encrypted data. – Fabaceous

Amazon returns a presigned URL. Check step #1 – Abeyant

Thanks for pointing me. I have moved forward with new error : AuthorizationHeaderMalformedThe authorization header is malformed; incorrect service "execute-api". This endpoint belongs to "s3".9DD75A286E7422B6rxCxNZ3veB/3ZJ1qrtvleA0JaHTPqprLYe3I5mM/LYLLEVPL6iKGv0irGmV1O9SS4AcmPsM/8/I= – Fabaceous

Header I have passed : $headers = [ 'Content-Type: text/plain; charset=utf-8', 'X-Amz-Content-Sha256: UNSIGNED-PAYLOAD', 'X-Amz-Date: 20210105T095435Z', 'Authorization: AWS4-HMAC-SHA256 Credential={ access-key-when-create-IAM }/20210105/us-east-1/execute-api/aws4_request, SignedHeaders=host;x-amz-access-token;x-amz-content-sha256;x-amz-date, Signature={generated through auth process}', ]; – Fabaceous

I am using the same function but unable to to upload feed details,I am using this response Url as 'url' parameter of the UploadFile(byte[] bytes, string url) function but getting this error Error Details: <Code>SignatureDoesNotMatch</Code> <Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message> – Coast

Can anyone please suggest me How to use Price.xsd in Pricing Feed. – Daniels

When I tried the URL, it says invalid signature. Please help us in this regard. – Ari

As per the solution, the step 2 request would need a content Type of "text/tab-separated-values; charset=UTF-8" Which I was missing. Now it is working. – Ari

My question is how this works with th4e Amazon envelope. To summarize your answer the steps are to (1) CreateFeedDocument - which returns a payload for (2) encrypt "body" (can the body be a json string of a serialized envelope class like orderAcknowledgement) (3) CreateFeed - which (sends) the body and finally (4) decrypt the response and figures what to do next? I'm switching from MWS where we use the envelope for pricing, inventory, fulfillment, and order acks, not flat files. – Brigandage