Bing Ads OAuth Automation using only .NET?

Question

How can I log onto Microsoft Live (with .NET WebClient?) and automate the OAuth process to get a token to make Bing Ads API calls?

My question is similar to How do I get an OAuth request_token from live.com?. However, I am building (C#, .NET 4.5.2) a headless Windows Service using the context of a Bing Ads super admin account that is linked to multiple other Bing Ads accounts. The idea is to authenticate, get the auth bits, and then make calls using the bits at 3:00am. Some of the accounts "compete" so for example group A should not see data from group B, so having an application get data for everyone and filter it and distribute it overnight solves many business problems.

I am concerned that if Live experiences problems, or our application is down for an extended time for any reason, we will have to re-authenticate manually to get data again. Maintenance and management of the credentials is now additional overhead (this is for an enterprise environment) that will have to take the form of an intranet web site/page to allow junior/uninitiated folks to do the work if needed (lets not forget testing and documentation). To contrast, Google provides an option to use key pairs for groups that need to work in a fully automated manner. It appears that Twitter's OAuth2 implementation can be automated without a GUI logon. It appears that other Bing services (eg Translation) can also be automated with WebClient.

I have the Microsoft account name and password already, and have a callback URL of "local-mydomain.com" set in the Bing Ads app GUI (and have a HOSTS entry for local-mydomain.com).

The Microsoft sample appears to work, but it automates the MS Web Browser Control, expects a user to input credentials in the GUI, and then the token is given. Giving the super admin account to users to do this is not an option. Expecting a user to get up at 3:00am to authenticate to upload/download data is not an option. Expecting a user to get desktop access to a server in the farm to "run something" is not an option.

All OAuth ideas appreciated.

Thanks.

Here is the launching code:

 partial class OAuthForm : Form
    {
        private static OAuthForm _form;
        private static WebBrowser _browser;

        private static string _code;
        private static string _error;

        // When you register your application, the Client ID is provisioned.

        private const string ClientId = "000redacted000";

        // Request-related URIs that you use to get an authorization code, 
        // access token, and refresh token.

        private const string AuthorizeUri = "https://login.live.com/oauth20_authorize.srf"; 
        private const string TokenUri = "https://login.live.com/oauth20_token.srf"; 
        private const string DesktopUri = "https://login.live.com/oauth20_desktop.srf"; 
        private const string RedirectPath = "/oauth20_desktop.srf";
        private const string ConsentUriFormatter = "{0}?client_id={1}&scope=bingads.manage&response_type=code&redirect_uri={2}";
        private const string AccessUriFormatter = "{0}?client_id={1}&code={2}&grant_type=authorization_code&redirect_uri={3}";
        private const string RefreshUriFormatter = "{0}?client_id={1}&grant_type=refresh_token&redirect_uri={2}&refresh_token={3}";

        // Constructor

        public OAuthForm(string uri)
        {
            InitializeForm(uri);
        }

        [STAThread]
        static void Main()
        {
            // Create the URI to get user consent. Returns the authorization
            // code that is used to get an access token and refresh token.

            var uri = string.Format(ConsentUriFormatter, AuthorizeUri, ClientId, DesktopUri);

            _form = new OAuthForm(uri);

            // The value for "uri" is 
            // https://login.live.com/oauth20_authorize.srf?client_id=000redacted000&scope=bingads.manage&response_type=code&redirect_uri=https://login.live.com/oauth20_desktop.srf



            _form.FormClosing += form_FormClosing;
            _form.Size = new Size(420, 580);

            Application.EnableVisualStyles();

            // Launch the form and make an initial request for user consent.
            // For example POST /oauth20_authorize.srf?
            //                 client_id=<ClientId>
            //                 &scope=bingads.manage
            //                 &response_type=code
            //                 &redirect_uri=https://login.live.com/oauth20_desktop.srf HTTP/1.1

            Application.Run(_form);  // <!---------- Problem is here. 
                                     //  I do not want a web browser window to show,
                                     // I need to automate the part of the process where
                                     // a user enters their name/password and are
                                     // redirected.

            // While the application is running, browser_Navigated filters traffic to identify
            // the redirect URI. The redirect's query string will contain either the authorization  
            // code if the user consented or an error if the user declined.
            // For example https://login.live.com/oauth20_desktop.srf?code=<code>



            // If the user did not give consent or the application was 
            // not registered, the authorization code will be null.

            if (string.IsNullOrEmpty(_code))
            {
                Console.WriteLine(_error);
                return;
            }

Mountaintop

The Microsoft sample appears to work, but it automates the MS Web Browser Control, expects a user to input credentials in the GUI Can you show that code snippet perhaps you are using the Credentials Enum incorrectly if one is needed.. – Wasting

@DJKRAZE, the problem is not an Enum, it's that a Winform+Web Browser control is required in the MS sample to get a token. I can't have a GUI COM object popping up on a server sitting in a rack just to get an auth token. My real problem is not knowing how to send credentials in pure code to get back a token - without a GUI. Thanks. – Mountaintop

can you show what you have thus far please.. this shouldn't seem that difficult normally most objects the use a token or a username\domain password for example have a propery PropertyName.UseDefaultCredentials for example can you show code – Wasting

Answer 1

Whatever you do, the "super admin" will have to log on at least once, using a browser. You can do that by hosting a simple web page in your service, or you could do it as part of the setup process. The Live samples show you how to do that.

Once the "super admin" has logged on using the code grant, you receive an access token and a refresh token. I'm not sure how long the Live access token is valid, but it is probably log enough for one nightly run. Save the refresh token in a safe place. The following night, you start by exchanging that refresh token by a new access token and a new refresh token. Again, you save this new refresh token for the following night.

You can keep this process running forever, as long as the "super admin" does not revoke the authorization he gave to your app.

UPDATE:

Some OAuth 2.0 servers support the "Resource Owner Password Credentials Grant", see the RFC at https://www.rfc-editor.org/rfc/rfc6749. If the Live server supports that, it would be an alternative to the Code Grant that does not require a browser. However, even of the server supports it, I would recommend against it for security reasons, as it requires storing your "super admin" password on the server. If someone grabs the password, they have full access to the account, and all resources protected by it. It will also break down if you change the password. The code grant does not have these problems.

Your question states that you want or need to run as this "super admin". An other option might be to use the "Client Credentials Grant". However, this also requires a client secret to be stored on the server (as with the password credentials grant). Furthermore, it still requires the super admin to authorize the client, and that in itself requires a code grant using a browser.

You ask why the code grant requires a browser, why you can not use some kind of screen scraping to simulate a browser interaction. First of all, you cannot predict the screens that will be shown to the user. These screens change without notice. More importantly, depending on user options and history, the server shows different screens. For example, the user may have turned on two-factor authentication. Last but not least, why do you object to opening a browser? It will probably be easier than trying to emulate it.

Finally, these "super admin" users might object to giving their password to your application, as they don't really know what you are doing with it (you might be sending to a server of your own, as far as they know). Using the Code Grant with a browser, they know your application never gets to see their password (kind of - you could listen in on browser events or something, unless the browser control is run in a separate process not under your control, such as the Windows 8 WebAuthenticationBroker). Your application only gets a token with the scopes they authorize.

Maebashi

Thanks Kris. One thing I don't understand is why the browser is required. I would think that WebClient could do anything a human can do (POSTing credentials outright, filling out a form, etc), and that server-server OAuth is not a new problem and has been solved. I thought there might be hope when I saw weblog.west-wind.com/posts/2013/Jun/06/… but maybe not. – Mountaintop

@Kris, Can I use single refresh token for all ad account which is generated from "Super Admin." – Animadvert

Answer 2

After spending a few hours on this problem for myself and finding absolutely no solution to automate connecting to Bing from a service. Here is what will work using the wonderful WatiN

First grab WatiN and add it to your solution via Nuget.

Then use the following code (my example works in a console application as an Example) to automate the whole grabbing of a token from Microsoft. It's not perfect as this is a sample but it will work.

You should double check the element ID's I'm using in case they changed, they are hard coded - generally remove all the hard coding if your going to use this in a production environment.

I didn't want anyone else to have to go through this.

First it grabs a Code that is then used to grab a Token, just like the OAuth 2.0 specification requires.

using System;
using System.Collections.Generic;
using System.Net;
using System.IO;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Json;
using System.Text;
using WatiN.Core.Native;
using WatiN.Core;

namespace LouiesOAuthCodeGrantFlow
{
    // Using access tokens requires that you register your application and that
    // the user gives consent to your application to access their data. This 
    // example uses a form and WebBrowser control to get the user's consent.
    // The control and form require a single-threaded apartment.

partial class LouiesBingOAuthAutomation 
{

    private static LouiesBingOAuthAutomation _form;

    private static string _code;
    private static string _error;

    //your going to want to put these in a secure place this is for the sample
    public const string UserName = "your microsoft user name";
    public const string Password = "<your microsoft account password";

    // When you register your application, the Client ID is provisioned.
    //get your clientid https://developers.bingads.microsoft.com/Account
    private const string ClientId = "<your client id>";

    // Request-related URIs that you use to get an authorization code, 
    // access token, and refresh token.

    private const string AuthorizeUri = "https://login.live.com/oauth20_authorize.srf";
    private const string TokenUri = "https://login.live.com/oauth20_token.srf";
    private const string DesktopUri = "https://login.live.com/oauth20_desktop.srf";
    private const string RedirectPath = "/oauth20_desktop.srf";
    private const string ConsentUriFormatter = "{0}?client_id={1}&scope=bingads.manage&response_type=code&redirect_uri={2}";//&displayNone
    private const string AccessUriFormatter = "{0}?client_id={1}&code={2}&grant_type=authorization_code&redirect_uri={3}";
    private const string RefreshUriFormatter = "{0}?client_id={1}&grant_type=refresh_token&redirect_uri={2}&refresh_token={3}";

    // Constructor

    public LouiesBingOAuthAutomation(string uri)
    {
        InitializeForm(uri);
    }

    [STAThread]
    static void Main()
    {

        var uri = string.Format(ConsentUriFormatter, AuthorizeUri, ClientId, DesktopUri);
        _form = new LouiesBingOAuthAutomation(uri);

        if (string.IsNullOrEmpty(_code))
        {
            Console.WriteLine(_error);
            return;
        }

        uri = string.Format(AccessUriFormatter, TokenUri, ClientId, _code, DesktopUri);
        AccessTokens tokens = GetAccessTokens(uri);

        Console.WriteLine("Access token expires in {0} minutes: ", tokens.ExpiresIn / 60);
        Console.WriteLine("\nAccess token: " + tokens.AccessToken);
        Console.WriteLine("\nRefresh token: " + tokens.RefreshToken);


        uri = string.Format(RefreshUriFormatter, TokenUri, ClientId, DesktopUri, tokens.RefreshToken);
        tokens = GetAccessTokens(uri);

        Console.WriteLine("Access token expires in {0} minutes: ", tokens.ExpiresIn / 60);
        Console.WriteLine("\nAccess token: " + tokens.AccessToken);
        Console.WriteLine("\nRefresh token: " + tokens.RefreshToken);
    }


    private void InitializeForm(string uri)
    {

        using (var browser = new IE(uri))
        {
            var page = browser.Page<MyPage>();
            page.PasswordField.TypeText(Password);
            try
            {
                StringBuilder js = new StringBuilder();
                js.Append(@"var myTextField = document.getElementById('i0116');");
                js.Append(@"myTextField.setAttribute('value', '"+ UserName + "');");
                browser.RunScript(js.ToString());
                var field = browser.ElementOfType<TextFieldExtended>("i0116");
                field.TypeText(UserName);
            }
            catch (Exception ex)
            {
                Console.Write(ex.Message + ex.StackTrace);
            }
            page.LoginButton.Click();
            browser.WaitForComplete();
            browser.Button(Find.ById("idBtn_Accept")).Click();
            var len = browser.Url.Length - 43;
            string query = browser.Url.Substring(43, len);

            if (query.Length == 50)
            {
                if (!string.IsNullOrEmpty(query))
                {
                    Dictionary<string, string> parameters = ParseQueryString(query, new[] { '&', '?' });

                    if (parameters.ContainsKey("code"))
                    {
                        _code = parameters["code"];
                    }
                    else
                    {
                        _error = Uri.UnescapeDataString(parameters["error_description"]);
                    }
                }
            }

        }

    }

    // Parses the URI query string. The query string contains a list of name-value pairs 
    // following the '?'. Each name-value pair is separated by an '&'.

    private static Dictionary<string, string> ParseQueryString(string query, char[] delimiters)
    {
        var parameters = new Dictionary<string, string>();

        string[] pairs = query.Split(delimiters, StringSplitOptions.RemoveEmptyEntries);

        foreach (string pair in pairs)
        {
            string[] nameValue = pair.Split(new[] { '=' });
            parameters.Add(nameValue[0], nameValue[1]);
        }

        return parameters;
    }

    // Gets an access token. Returns the access token, access token 
    // expiration, and refresh token.

    private static AccessTokens GetAccessTokens(string uri)
    {
        var responseSerializer = new DataContractJsonSerializer(typeof(AccessTokens));
        AccessTokens tokenResponse = null;

        try
        {
            var realUri = new Uri(uri, UriKind.Absolute);

            var addy = realUri.AbsoluteUri.Substring(0, realUri.AbsoluteUri.Length - realUri.Query.Length);
            var request = (HttpWebRequest)WebRequest.Create(addy);

            request.Method = "POST";
            request.ContentType = "application/x-www-form-urlencoded";

            using (var writer = new StreamWriter(request.GetRequestStream()))
            {
                writer.Write(realUri.Query.Substring(1));
            }

            var response = (HttpWebResponse)request.GetResponse();

            using (Stream responseStream = response.GetResponseStream())
            {
                if (responseStream != null)
                    tokenResponse = (AccessTokens)responseSerializer.ReadObject(responseStream);
            }
        }
        catch (WebException e)
        {
            var response = (HttpWebResponse)e.Response;

            Console.WriteLine("HTTP status code: " + response.StatusCode);
        }

        return tokenResponse;
    }

 }



 public class MyPage : WatiN.Core.Page
 {
    public TextField PasswordField
    {
        get { return Document.TextField(Find.ByName("passwd")); }
    }

    public WatiN.Core.Button LoginButton
    {
        get { return Document.Button(Find.ById("idSIButton9")); }
    }
 }

 [ElementTag("input", InputType = "text", Index = 0)]
 [ElementTag("input", InputType = "password", Index = 1)]
 [ElementTag("input", InputType = "textarea", Index = 2)]
 [ElementTag("input", InputType = "hidden", Index = 3)]
 [ElementTag("textarea", Index = 4)]
 [ElementTag("input", InputType = "email", Index = 5)]
 [ElementTag("input", InputType = "url", Index = 6)]
 [ElementTag("input", InputType = "number", Index = 7)]
 [ElementTag("input", InputType = "range", Index = 8)]
 [ElementTag("input", InputType = "search", Index = 9)]
 [ElementTag("input", InputType = "color", Index = 10)]
 public class TextFieldExtended : TextField
 {
    public TextFieldExtended(DomContainer domContainer, INativeElement element)
        : base(domContainer, element)
    {
    }

    public TextFieldExtended(DomContainer domContainer, ElementFinder finder)
        : base(domContainer, finder)
    {
    }

    public static void Register()
    {
        Type typeToRegister = typeof(TextFieldExtended);
        ElementFactory.RegisterElementType(typeToRegister);
    }
 }


 // The grant flow returns more fields than captured in this sample.
 // Additional fields are not relevant for calling Bing Ads APIs or refreshing the token.

 [DataContract]
 class AccessTokens
 {
    [DataMember]
    // Indicates the duration in seconds until the access token will expire.
    internal int expires_in = 0;

    [DataMember]
    // When calling Bing Ads service operations, the access token is used as  
    // the AuthenticationToken header element.
    internal string access_token = null;

    [DataMember]
    // May be used to get a new access token with a fresh expiration duration.
    internal string refresh_token = null;

    public string AccessToken { get { return access_token; } }
    public int ExpiresIn { get { return expires_in; } }
    public string RefreshToken { get { return refresh_token; } }
 }
}

Wymore