(401) Unauthorized exception while downloading file from SharePoint

Question

I have generated an access token using OAuth mechanism for SharePoint Online server. I am using this token to create ClientContext using CSOM. While I am able to access all the sites, libraries, and folders seamlessly, I get error

The remote server returned an error: (401) Unauthorized.

while downloading the file from SharePoint Online. Below is the code that I am using for file download:

var clientContext = TokenHelper.GetClientContextWithAccessToken("https://adventurer.sharepoint.com/Subsite1", accessToken);
var list = clientContext.Web.Lists.GetByTitle("SubSite 1 Library 1");
string vquery = @"<View Scope='RecursiveAll'><Query><Where><Eq><FieldRef Name='UniqueId' /><Value Type='Lookup'>" + "6718053d-a785-489c-877f-5a4b88dcb2a7" + "</Value></Eq></Where></Query></View>";
CamlQuery query = new CamlQuery();
query.ViewXml = vquery;
var listItems = list.GetItems(query);
clientContext.Load(listItems, items => items.Take(1).Include(item => item.File));
clientContext.ExecuteQuery();

var fileRef = listItems[0].File.ServerRelativeUrl;
var fileInfo = Microsoft.SharePoint.Client.File.OpenBinaryDirect(clientContext, fileRef);

I don't understand the root cause of this error, as I am passing client context with right access token. I want to know if OpenBinaryDirect has a limitation to work with access tokens? If not, what is wrong with above code? Is there any other alternative that can be used to download using access token?

Sleep

Answer 1

After trying a lot of alternatives, I have come to conclusion that OpenBinaryDirect() cannot be used with OAuth tokens. I was able to download file from SharePoint Online using two other approaches. I am posting the answers here so that it might help someone:

Approach 1 (OpenBinaryStream):

var file = clientContext.Web.GetFileByServerRelativeUrl(fileRef);
clientContext.Load(file);
clientContext.ExecuteQuery();                    
ClientResult<Stream> streamResult = file.OpenBinaryStream();
clientContext.ExecuteQuery();

While this approach works perfectly, OpenBinaryStream is not available in Microsoft.SharePoint.Client.dll <= v 14.0.0.0.

Approach 2 (WebClient or Other Http Requests):

string downloadUrl = HostURL + "/_api/web/getfilebyserverrelativeurl('" + fileRef + "')/$value";
WebClient client = new WebClient(); 
client.Headers.Add("Authorization", "Bearer " + accessToken);
client.DownloadFile(downloadUrl, filePath);

Note the download URL that I have used for WebClient. Normal file URL will not work to download files from SharePoint Online.

Sleep

Answer 2

I am using C# and here is how I am currently retrieving documents from SharePoint Online. I am showing the user a list of their documents in a gridview, so I populate a DataTable with the documents. I am unsure of a way using an Access Token, but if you are able to use a Service Account like I am, then hopefully this helps you.

Namespaces

using Microsoft.SharePoint.Client;
using SP = Microsoft.SharePoint.Client;

Object Attributes

SecureString securePassword = new SecureString();
private string username = "";    
ClientContext context = new SP.ClientContext("https://<root>.sharepoint.com/<site collection (unless root)>/<site>");

Constructor (This is how I am authenticating)

public SharePoint()
{
    securePassword = convertToSecureString(System.Web.Configuration.WebConfigurationManager.AppSettings["O365PW"]);
    username = System.Web.Configuration.WebConfigurationManager.AppSettings["O365UN"];
    context.Credentials = new SharePointOnlineCredentials(username, securePassword);
}

Method to get documents

public DataTable GetDocuments(int changeID)
{
    DataTable dt = new DataTable("ChangeDocuments");
    DataRow dr = dt.NewRow();
    dt.Columns.Add("Title");
    dt.Columns.Add("URL");
    dt.Columns.Add("ChangeID");
    dt.Columns.Add("Modified");
    dt.Columns.Add("ID");


    // The SharePoint web at the URL.
    Web web = context.Web;

    // We want to retrieve the web's properties.
    context.Load(web);

    // We must call ExecuteQuery before enumerate list.Fields. 
    context.ExecuteQuery();

   // Assume the web has a list named "Announcements". 
   SP.List oList = context.Web.Lists.GetByTitle("Name of your document library");

   // This creates a CamlQuery that has a RowLimit of 100, and also specifies Scope="RecursiveAll" 
   // so that it grabs all list items, regardless of the folder they are in. 
   CamlQuery query = CamlQuery.CreateAllItemsQuery(100);
query.ViewXml = "<View><Query><Where><Eq><FieldRef Name='ChangeID'/>" +
    "<Value Type='Number'>" + changeID + "</Value></Eq></Where></Query><RowLimit>100</RowLimit></View>";
   SP.ListItemCollection items = oList.GetItems(query);

   // Retrieve all items in the ListItemCollection from List.GetItems(Query). 
   context.Load(items);
   context.ExecuteQuery();

foreach (Microsoft.SharePoint.Client.ListItem listItem in items)
{
    // We have all the list item data. For example, Title. 

    dr = dt.NewRow();

    dr["Title"] = listItem["FileLeafRef"];

    if (String.IsNullOrEmpty(listItem["ServerRedirectedEmbedUrl"].ToString()))
    {
        dr["URL"] = "<root>/<site>/<document library>" + listItem["FileLeafRef"].ToString();
    }
    else
    {
        dr["URL"] = listItem["ServerRedirectedEmbedUrl"];
    }

    dr["ChangeID"] = listItem["ChangeID"];
    dr["Modified"] = Convert.ToDateTime(listItem["Modified"]).ToString("MMM.dd,yyyy h:mm tt");
    dr["ID"] = listItem["ID"];
    dt.Rows.Add(dr);
 }

return dt;
}

Method to convert password to secure string

private SecureString convertToSecureString(string strPassword)
{
        var secureStr = new SecureString();
        if (strPassword.Length > 0)
        {
            foreach (var c in strPassword.ToCharArray()) secureStr.AppendChar(c);
        }
        return secureStr;
}

Hyperphagia

I am using OAuth mechanism to authenticate users to SharePoint Online. So, I am generating client context with access token. This code cannot be used in that regard: context.Credentials = new SharePointOnlineCredentials(username, securePassword); – Sleep