Azure rest api put blob

Question

I am trying to put a blob with Azure rest api. i made a "GET" request successfully but i had issues with "PUT" request. When i try to make "PUT" request i get a 404error (i have seen same post in stackoverflow but it didnt help me).i am not sure if the MessageSignature that i use is correct(i have tried MessageSignaturePut but didnt work). Any suggestions?

public void UploadBlobWithRestAPI(string uri,  DateTime now)
{
    string blobName = "test.txt";
    string method = "PUT";
    string sampleContent = "This is sample text.";
    int contentLength = Encoding.UTF8.GetByteCount(sampleContent);
    string queryString = (new Uri(uri)).Query;
    string blobContainerUri = uri.Substring(0, uri.Length - queryString.Length);
    string requestUri = string.Format(CultureInfo.InvariantCulture, "{0}/{1}{2}", blobContainerUri, blobName, queryString);
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(requestUri);
    string nnow = now.ToString("R", System.Globalization.CultureInfo.InvariantCulture);

    request.Method = method;
    request.Headers.Add("x-ms-version", "2015-02-21");
    request.Headers.Add("x-ms-date", nnow);
    request.ContentType = "text/plain; charset=UTF-8";
    request.Headers.Add("x-ms-blob-type", "BlockBlob");
    request.ContentLength = contentLength;

    using (Stream requestStream = request.GetRequestStream())
    {
        requestStream.Write(Encoding.UTF8.GetBytes(sampleContent), 0, contentLength);
    }

    request.Headers.Add("Authorization", AuthorizationHeader(method, now, request, "", ""));

    using (HttpWebResponse resp = (HttpWebResponse)request.GetResponse())
    {
        MessageBox.Show(resp.StatusCode.ToString());
    }
}


public string AuthorizationHeader(string method, DateTime now, HttpWebRequest request,
        string ifMatch = "", string md5 = "")
{
    string MessageSignature;
    string StorageKey = "xxx";
    string StorageAccount = "upgradedevstorage";

    MessageSignature = String.Format("{0}\n\n\n{1}\n{5}\n\n\n\n{2}\n\n\n\n{3}{4}",
        method,
        (method == "GET" || method == "HEAD") ? String.Empty : request.ContentLength.ToString(),
        ifMatch,
        GetCanonicalizedHeaders(request),
        GetCanonicalizedResource(request.RequestUri, StorageAccount),
        md5
        );

 ???   //string MessageSignaturePut= String.Format("{0}\n\n{1}\n\n{2}{3}",
    //    method,
    //    "text/plain; charset=UTF-8",
    //    GetCanonicalizedHeaders(request),
    //    GetCanonicalizedResource(request.RequestUri, StorageAccount)
    //    );

    byte[] SignatureBytes = System.Text.Encoding.UTF8.GetBytes(MessageSignature);

    System.Security.Cryptography.HMACSHA256 SHA256 =
        new System.Security.Cryptography.HMACSHA256(Convert.FromBase64String(StorageKey));

    string signature = Convert.ToBase64String(SHA256.ComputeHash(SignatureBytes));

    string AuthorizationHeader = "SharedKey " + StorageAccount
        + ":" + Convert.ToBase64String(SHA256.ComputeHash(SignatureBytes));

    return AuthorizationHeader;
}

Araucaria

Since you're using Shared Access Signature (SAS), you don't need authorization header. Can you tell me how you created the Shared Access Signature? I am interested in knowing about the permissions included in SAS. – Kuhlmann

Answer 1

Please consider the following code snippet. This should work for you.

void UploadBlobWithRestAPI() {

    string storageKey = "<your access key here>";
    string storageAccount = "<your storage account name here>";    
    string containerName = "<your container name here>";
    string blobName = "test.txt";

    string method = "PUT";
    string sampleContent = "This is sample text.";
    int contentLength = Encoding.UTF8.GetByteCount(sampleContent);

    string requestUri = $"https://{storageAccount}.blob.core.windows.net/{containerName}/{blobName}";

    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(requestUri);

    string now = DateTime.UtcNow.ToString("R");

    request.Method = method;
    request.ContentType = "text/plain; charset=UTF-8";
    request.ContentLength = contentLength;

    request.Headers.Add("x-ms-version", "2015-12-11");
    request.Headers.Add("x-ms-date", now);
    request.Headers.Add("x-ms-blob-type", "BlockBlob");
    request.Headers.Add("Authorization", AuthorizationHeader(method, now, request, storageAccount, storageKey, containerName, blobName));

    using (Stream requestStream = request.GetRequestStream()) {
        requestStream.Write(Encoding.UTF8.GetBytes(sampleContent), 0, contentLength);
    }

    using (HttpWebResponse resp = (HttpWebResponse)request.GetResponse()) {
        MessageBox.Show(resp.StatusCode.ToString());
    }

}

public string AuthorizationHeader(string method, string now, HttpWebRequest request, string storageAccount, string storageKey, string containerName, string blobName) {

    string headerResource = $"x-ms-blob-type:BlockBlob\nx-ms-date:{now}\nx-ms-version:2015-12-11";
    string urlResource = $"/{storageAccount}/{containerName}/{blobName}";
    string stringToSign = $"{method}\n\n\n{request.ContentLength}\n\n{request.ContentType}\n\n\n\n\n\n\n{headerResource}\n{urlResource}";

    HMACSHA256 hmac = new HMACSHA256(Convert.FromBase64String(storageKey));
    string signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));

    String AuthorizationHeader = String.Format("{0} {1}:{2}", "SharedKey", storageAccount, signature);
    return AuthorizationHeader;
}

And the traffic captured by Fiddler as follows:

Gayla

thanks a lot. i tried your code but i am getting a (403) Forbidden error – Araucaria

It works fine on my site. So could you edit your question with your latest code snippet? – Gayla

Hi, I have updated my code to fix my hard-code issue. Could you try again? – Gayla

Hi @kostas, glad to hear that, you can mark it as answer, which will help other communities who facing the similar issue with you. – Gayla

This solution works but only handles files up to 4MB. Robin Shahan from Red Gate wrote a great series on how to chunk data with the Windows Azure SDK library at red-gate.com/simple-talk/cloud/platform-as-a-service/… – Chop

Answer 2

The solution above only uploads files up to 4MB in size. I had need of a PowerShell version for a project requirement and went down the wrong trail with the solution above. For a custom function to chunk a BLOB to Azure RBS, I modified Robin Shahan's version from Red Gate at https://www.red-gate.com/simple-talk/cloud/platform-as-a-service/azure-blob-storage-part-4-uploading-large-blobs/.

$sdkPath = "C:/Program Files/Microsoft SDKs/Azure/.NET SDK/v2.9/bin/plugins/Diagnostics/Microsoft.WindowsAzure.Storage.dll"
[System.Reflection.Assembly]::LoadFrom($sdkPath);

Add-Type -AssemblyName System.Net.Http

function getMD5HashFromBytes([byte[]]$fileBytes){
    $md5 = [System.Security.Cryptography.MD5]::Create()
    [byte[]]$hash = $md5.ComputeHash($fileBytes)
    return [System.Convert]::ToBase64String($hash)
}

function setupBlobContainer($account, $secretKey, $container){
    $cs = [String]::Format("DefaultEndpointsProtocol=https;AccountName={0};AccountKey={1}", $account, $secretKey)
    $cloudStorageAccount = [Microsoft.WindowsAzure.Storage.CloudStorageAccount]::Parse($cs)
    $cloudBlobClient = [Microsoft.WindowsAzure.Storage.Blob.CloudBlobClient]$cloudStorageAccount.CreateCloudBlobClient()
    $cloudBlobContainer = [Microsoft.WindowsAzure.Storage.Blob.CloudBlobContainer]$cloudBlobClient.GetContainerReference($container)
    return $cloudBlobContainer
}

function chunkBlob([string]$filepath, [string]$filename, `
    [Microsoft.WindowsAzure.Storage.Blob.CloudBlobContainer]$cloudBlobContainer){

    #ref: https://www.red-gate.com/simple-talk/cloud/platform-as-a-service/azure-blob-storage-part-4-uploading-large-blobs/

    $blob = [Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob]$cloudBlobContainer.GetBlockBlobReference($filename)
    $blockSize = 256 * 1024; #256 kb
    $fileStream = [System.IO.FileStream]::new($filepath, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Read, [System.IO.FileShare]::ReadWrite)
    $fileSize = $fileStream.Length

    #block count is the number of blocks + 1 for the last one
    $blockCount = [int]([float]$fileSize / [float]$blockSize) + 1
 
    #List of block ids; the blocks will be committed in the order of this list 
    $blockIDs = [System.Collections.Generic.List[string]]::new()
 
    #starting block number - 1
    $blockNumber = 0
 
    try
    {
        $bytesRead = 0 #number of bytes read so far
        $bytesLeft = $fileSize; #number of bytes left to read and upload
 
        #do until all of the bytes are uploaded
        while($bytesLeft -gt 0){

            $blockNumber++;

            [int]$bytesToRead;
        
            if($bytesLeft -ge $blockSize){
                #more than one block left, so put up another whole block
                $bytesToRead = $blockSize
            }
            else{
                #less than one block left, read the rest of it
                $bytesToRead = [int]$bytesLeft
            }
 
            #create a blockID from the block number, add it to the block ID list
            #the block ID is a base64 string
            $blockId = [Convert]::ToBase64String([System.Text.ASCIIEncoding]::ASCII.GetBytes([String]::Format("BlockId{0}", $blockNumber.ToString("0000000"))))
            $blockIDs.Add($blockId)
        
            #set up new buffer with the right size, and read that many bytes into it 
            [byte[]]$bytes = [System.Byte[]]::new($bytesToRead)

            $fileStream.Read($bytes, 0, $bytesToRead)
 
            #calculate the MD5 hash of the byte array
            $blockHash = getMD5HashFromBytes $bytes
 
            #upload the block, provide the hash so Azure can verify it
            $blob.PutBlock($blockId, [System.IO.MemoryStream]::new($bytes), $blockHash)
 
            #increment/decrement counters
            $bytesRead += $bytesToRead
            $bytesLeft -= $bytesToRead

            $perc = [float][math]::Round( [float]$bytesRead/[float]($bytesRead + $bytesLeft) * 100, 2)

            Write-Progress -Activity "Writing '$($filename)'..." -PercentComplete $perc
        }
 
        #commit the blocks
        $blob.PutBlockList($blockIDs)
    }
    catch [System.Exception] {
        write-warning $_
    }
    finally{
        if($fileStream){
            $fileStream.Dispose()
        }
    }
}

Chop

Answer 3

Adding to @John Bonfardeci's answer, if you don't want to use the azure sdk, this will work for powershell.

function Add-FileToBlobStorage {
    param (
        [Parameter(Mandatory=$true,ValueFromPipeline=$true,ValueFromPipelineByPropertyName)]
        [ValidateScript({Test-Path $_ })]
        [Alias("FullName")]
        [string] $Path,
        [string] $TargetFileName,
        [Parameter(Mandatory=$true)]
        [ValidateScript({($_ -match "^https:\/\/(.)*\.blob.core.windows.net\/(.)*") -and ($_ -notmatch '\?')})]
        [string] $containerUrl,
        [Parameter(Mandatory=$true)]
        [ValidateScript({$_.StartsWith("?")})]
        [string]$sasToken
    )
    if (-not $TargetFileName) {
        $FileItem = Get-Item $Path -ErrorAction Stop
        $TargetFileName=$FileItem.Name
    }
    $HashArguments = @{
        uri = "$containerUrl/$($TargetFileName.Replace("\","/"))$sasToken"
        method = "Put"
        InFile = $Path
        headers = @{
            "x-ms-blob-type" = "BlockBlob"
        }
    }
    Write-Verbose "Uploading to $containerUrl/$($TargetFileName.Replace("\","/")) from $path"
    Invoke-RestMethod @HashArguments
    Write-Host "Upload to $containerUrl/$($TargetFileName.Replace("\","/")) from $path was successful"
}

# Usage example:
$url = "https://$storageAccountName.blob.core.windows.net/$containerName"
Get-Item $localFilePath | Add-FileToBlobStorage -containerUrl $url -sasToken $sasToken -TargetFileName 'a\test.html'

Baptism