Hooking to Zapier using .Net WebHooks as RESThooks

Took a bit of research, but I finally have the Zapier Rest Hooks working. Not as straight forward as I might have hoped (more likely I am a bit slow on the uptake). The customer support was excellent and friendly, so don't hesitate to email them your questions. Also once you get this working it is VERY powerful, though their normal webhook mechanism also works and does not require you to create a Zap App. At the time of this writing I have not pushed out the application, though it is working locally. This assumes you have started to create your own Zapier Application in the Developer Dashboard. It is pretty straightforward so I will not cover it here.

This explanation will only cover creating a single Trigger (though you create 1 more private one to support user Authentication, and I created another for a dynamic pulldown purposes) and only as a RESThook with Basic Auth. The basics are:

1. Create webhooks to allow Zapier to add, update and remove subscriptions to your Actions. By "subscribing" Zapier does not have to poll your webhook, rather when the subscribed action happens on your side, you respond to a URL that Zapier has given you during the subscription process.

2. Create a DB table that logs these subscriptions, storing the data that you need to be able to then Post a response back to a Zapier supplied URL when the action is triggered on your side.

3. When the action is triggered, recognize that and post the data you told zapier that you would be sending. Zapier is pretty clever and will map the data (JSON or XML) for you, so when wiring up to another app the user can map between the two.

So a few more details. This was done in C# on .Net, but I think the concepts should work just as well in any other language or platform.

First the RESTHooks. What follows is an example of the RESTHook methods. Please note that I spent days trying to figure out the Zapier scripting side, so I am not entirely happy with the naming conventions, but hopefully you will get the idea.

In this scenario there is the concept of a "form" which is a chunk of JSON holding data that I care about, a User and Account that user belongs to. All of them have a unique ID in the System. Finally the subscription itself has an id. When you are subscribing, a specific User in a specific Account is subscribing a specific Form to be sent to Zapier when a specific Trigger (a submission of that form) is accomplished.

RESTHooks:

First the RouteConfig which maps the path to the method to execute. You can see all the methods I implemented. Some of them are not used and just included for possible future use (like updating a subscription).

    // ZAPIER Webhooks
    routes.MapRoute(
       "User_Form_List",
       "api/zapier/user/formlist",
        new { controller = "Zapier", action = "RetrieveFormListForUser" },
        new { httpMethod = new HttpMethodConstraint("GET") }
        );
    routes.MapRoute(
       "Authenticate_Subscription",
       "api/zapier/authenticate",
        new { controller = "Zapier", action = "WebhookAuthenticate" },
        new { httpMethod = new HttpMethodConstraint("GET") }
        );
    routes.MapRoute(
        "Test_Subscription",
        "api/zapier/subscription/testdata",
        new { controller = "Zapier", action = "TestData" },
        new { httpMethod = new HttpMethodConstraint("GET") }
        );
    routes.MapRoute(
        "Create_Submission",
        "api/zapier/subscription/create",
        new { controller = "Zapier", action = "CreateSubscription" },
        new { httpMethod = new HttpMethodConstraint("GET") }
        );
    routes.MapRoute(
       "List_Subscriptions",
       "api/zapier/subscription",
       new { controller = "Zapier", action = "ListSubscriptions" },
       new { httpMethod = new HttpMethodConstraint("GET") }
       );
    routes.MapRoute(
       "Get_Subscriptions",
       "api/zapier/subscription/{id}",
       new { controller = "Zapier", action = "GetSubscription", id = 0 },
       new { httpMethod = new HttpMethodConstraint("GET") }
       );
    routes.MapRoute(
       "Update_Subscription",
       "api/zapier/subscription/{id}",
       new { controller = "Zapier", action = "UpdateSubscription", id = 0 },
       new { httpMethod = new HttpMethodConstraint("PUT") }
       );
    routes.MapRoute(
        "Delete_Subscription",
        "api/zapier/subscription/{id}",
        new { controller = "Zapier", action = "DeleteSubscription", id = 0 },
        new { httpMethod = new HttpMethodConstraint("DELETE") }
    );

The code corresponding to this is (Also I have pulled out the error handling to reduce the code size):

    public class ZapierController : BaseController //(this inherits from Controller)
    {
        private readonly IMyRepository _DBrepository;

        public ZapierController(IMyRepository repository, ...lots of other autowiring you don't need or care about)
            : base(logger)
        {
            _DBrepository = repository;
         }

        #region Zapier Subscriptions

        // api/zapier/subscription/create  : Creates a subscription
        [HttpGet]
        public ActionResult CreateSubscription()
        {
            ApiResult authresult = Authenticate();
            if (authresult.code != 201)
            {
                return JsonApiResult(authresult);
            }

            // Get the request parameters
            var reqParams = GetParameters();

            // Create the subscription so long as it does not already exist
            WebhookSubscription sub = new WebhookSubscription();
            // _currentUser and _currentAccount are set as part of the authenticate and stored in our base controller
            sub.AccountId = _currentAccount.Id;
            sub.UserId = _currentUser.UserId;
            sub.TargetURL = reqParams["target_url"];
            sub.EventType = reqParams["target_event"];
            sub.FormId = Int32.Parse(reqParams["form_id"]);
            sub.IsActive = true;

            ObjectResult workflowActionRecord = _DBrepository.createWebhookSubscription(sub);
            sub.Id = workflowActionRecord.objectId;

            // return the subscription back to Zapier in the result. Zapier will remember it
            var result = new ApiResult();
            result.data.id = workflowActionRecord.objectId;
            result.data.subscription = sub;
            result.code = 201;
            return JsonApiResult(result);
        }


        // api/zapier/authenticate  : used to test authentication
        [HttpGet]
        public ActionResult WebhookAuthenticate()
        {
            ApiResult authresult = Authenticate();

            var result = new ApiResult();
            result.code = 201;
            return JsonApiResult(result);
        }

        // api/zapier/user/formlist  : returns list of forms for this user
        [HttpGet]
        public ActionResult RetrieveFormListForUser()
        {
            ApiResult authresult = Authenticate();

            var result = new ApiResult();

            List<Form> forms = _DBRepository.FormListRetrieveByUser(_currentUser, false);

            JsonSerializer serializer = new JsonSerializer();
            serializer.Converters.Add(new JavaScriptDateTimeConverter());
            serializer.NullValueHandling = NullValueHandling.Ignore;

            // Again Zapier likes arrays returned
            JArray objarray = JArray.FromObject(forms);
            return JsonApiResultDynamic(objarray);
        }

        // api/zapier/subscription/testdata  : returns test data for zapier
        [HttpGet]
        public ActionResult TestData()
        {

            ApiResult authresult = Authenticate();

            var result = new ApiResult();

            JsonSerializer serializer = new JsonSerializer();
            serializer.Converters.Add(new JavaScriptDateTimeConverter());
            serializer.NullValueHandling = NullValueHandling.Ignore;

            // Get the request parameters
            var reqParams = GetParameters();
            int chosenFormId = -1;
            // We need the form Id to proceed
            if (reqParams != null && reqParams["form_id"] != null)
                chosenFormId = Int32.Parse(reqParams["form_id"]);
            else
                return  JsonApiResult(new ApiResult() { code = 403, error = "Form Id Not Found" });

            // Get the form by Form Id, and return the JSON...I have removed that code, but make sure the result is place in an Array
            var resultdata = new[] { myFinalFormJSON };

            JArray objarray = JArray.FromObject(resultdata);
            return JsonApiResultDynamic(objarray);
        }


        // api/zapier/subscription  : returns list of subscriptions by account
        [HttpGet]
        public ActionResult ListSubscriptions()
        {
            ApiResult authresult = Authenticate();

            // Get a list all subscriptions for the account
            List<WebhookSubscription> actionData = _DBrepository.accountWebhookSubscriptions(_currentAccount.Id);

            var result = new ApiResult();
            result.code = 201;
            result.data.subscriptions = actionData;
            return JsonApiResult(result);
        }

        // api/zapier/subscription/{id}  : Creates a subscription
        [HttpGet]
        public ActionResult GetSubscription(int id)
        {
            ApiResult authresult = Authenticate();

            // Get a list all subscriptions for the account
            WebhookSubscription actionData = _DBrepository.getWebhookSubscription(id);

            var result = new ApiResult();
            result.data.subscription = actionData; ;
            result.code = 201;
            return JsonApiResult(result);
        }

        // api/zapier/subscription/{id}  : updates a subscription
        [HttpPut]
        public ActionResult UpdateSubscription(int id)
        {
            ApiResult authresult = Authenticate();

            // get target url and eventy type from the body of request
            string jsonString = RequestBody();
            var json = CommonUtils.DecodeJson(jsonString);

            // Create the subscription so long as it does not already exist
            WebhookSubscription sub = _DBrepository.getWebhookSubscription(id);

            var result = new ApiResult();
            if (sub != null)
            {
                sub.TargetURL = json.target_url; ;
                sub.EventType = json.eventType;

                ObjectResult objResult = _DBrepository.updateWebhookSubscription(sub);
                result.code = 201;
            }

            return JsonApiResult(result);
        }



        // api/zapier/subscription/{id}  : deletes a subscription
        [HttpDelete]
        public ActionResult DeleteSubscription(int id)
        {
            ApiResult authresult = Authenticate();

            // Delete a subscription
            _DBrepository.deleteWebhookSubscription(id);

            var result = new ApiResult();
            result.code = 201;
            return JsonApiResult(result);
        }

        // We need to Basic Auth for each call to subscription
        public ApiResult Authenticate()
        {
            // get auth from basic authentication header
            var auth = this.BasicAuthHeaderValue();

            // parse credentials from auth
            var userCredentials = Encoding.UTF8.GetString(Convert.FromBase64String(auth));
            var parts = CommonUtils.SplitOnFirst(userCredentials, ":");
            var username = parts[0];
            var password = parts[1];

            // authenticate user against repository
            if (!_DBrepository.UserAuthenticate(username, password))
            {
                _logger.Info("Invalid Authentication: " + username);
                return new ApiResult() { code = 401, error = "invalid authentication" };
            }

            return new ApiResult() { code = 201, error = "successful authentication" };

        }
   }

The DB table that will hold the subscriptions looks like this. I will leave out the reading and writing side, since you might have different mechanism.

   Create.Table("WebhookSubscription")
        .WithColumn("Id").AsInt32().Identity().PrimaryKey().NotNullable()
        .WithColumn("AccountId").AsInt32().NotNullable()
        .WithColumn("UserId").AsInt32().NotNullable()
        .WithColumn("EventType").AsString(256).NotNullable()
        .WithColumn("TargetURL").AsString(1000).NotNullable()
        .WithColumn("IsActive").AsBoolean().NotNullable()
        .WithColumn("CreatedOn").AsDateTime().Nullable()
        .WithColumn("FormId").AsInt32().NotNullable().WithDefaultValue(0);
        .WithColumn("UpdatedAt").AsDateTime().Nullable();

Just to be clear, the meaning/use for the following columns:

Id - Unique Subscription ID. Will be used to unsubscribe
AccountId - Account Id for the user subscribing. If you wanted all of these to work on an Account level could just do that instead
UserId - Id of user subscribing
EventType - The type of event your Action is to respond to, such as "new_form_submission"
TargetURL - Target URL that zapier gave you when subscribing. Will be where you post your JSON when Action initiated
FormId - Id of the form the user wants an action for when submitted

So that is the code needed for subscription (obviously you can't just throw it up there and have it work - left out a good amount to save space)...

Trigger Code

Only code left is the actual Trigger code - the code you execute when the event being looked for is encountered in your code. So for example when the user submits a "form" we want to send that form JSON to Zapier. Now that we have all the other code set up, this part is pretty simple. First the code to detect we had a submission arrive that needs a Zapier response:

The actual code that looks to see if the form submission is registered/subscribed to Zapier:

public BusinessResult FormSubmitted(string jsonString)
{
    var json = CommonUtils.DecodeJson(jsonString);
    var account = _DBrepository.AccountRetrieveById(_currentUser.AccountId.Value); // Assumes user has bee authenticated


    // inject additional meta data into json and retrieve submission/alert settings
    var form = _DBformRepository.FormRetrieveById((int)json.formId);

    // Lookup Subscription Webhooks
    List<WebhookSubscription>  subscriptions = _DBrepository.accountWebhookSubscriptions(account.Id);
    if (subscriptions != null && subscriptions.Count > 0)
    {
        foreach (WebhookSubscription sub in subscriptions)
        {
            if (sub.EventType.Equals("new_form_submission") && sub.FormId == form.Id)
            {
                _webhookService.NewFormSubmission(sub, jsonString, form.Name, account.Name, account.Id);
            }
        }
    }
}

And Finally the code to post that response back to Zapier, who will parse the JSON and send it to the appropriate parties:

public class WebhookService : IWebhookService
{
    protected readonly IRepository _DBrepository;

    public WebhookService(IRepository repository)
    {
        _DBrepository = repository;
    }

    public void NewFormSubmission(string formResultJSON)
    {
        throw new NotImplementedException();
    }

    public void NewFormSubmission(WebhookSubscription subscription, string formResultJSON, string formName, string accountName, int accountId)
    {
        // Now post to webhook URL
        string response; 
        using (var client = new WebClient())
        {
            client.Headers[HttpRequestHeader.ContentType] = "application/json";
            // Needs to be an array sent to Zapier
            response = client.UploadString(subscription.TargetURL, "POST", "[" + formResultJSON + "]");
        }
    }
}

Ok, that should get you most of the way there. But the wiring of the code/webhooks into Zapier is where is gets a bit more difficult. The idea now is to wire code above into your Zapier application using Development Dashboard. You will have to start creating a Zapier application. There are 2 main triggers you need - The basic action you are trying to implement (in this case "New Form Submission") and the authenticate so Zapier can authenticate a user as they are creating a Zap (in this case "Test Auth"). I am using Basic Auth, but others are supported (OAuth, etc). Additionally I added a trigger that will return the list of Forms the user has access to. Since it is not required I will not show that implementation in the screen captures:

I will not show the wiring for the "Test Auth" since that went pretty smoothly (I'll add it if someone requests it - heaven knows if anyone will even read this). So here is the wiring, page by page for the "New Form Submission":

Page 1

Page 2

This is where I wire up the Form Listing, which provides a list of forms the user creating a Zap can select from. You probably can skip this (leave it empty) unless you have dynamic data you want to display. I have included it for completeness:

Page 3

Here you wire up the test data

Page 4

This page allows you to enter Sample Data. Skipping this since it it pretty straight forward.

Scripting API

So now you have wired up your first Zap Trigger! But wait, we are not done. To make the Subscription process work we need to add some script. This was the hardest part of the entire process and was not very intuitive. So on the original main screen, a bit down you will see Scripting API:

Now you have to have the script for the RESTHook subscription. I am not going to go into great detail since Zapier does have documentation on this, but it is good to know that Zapier does store data as part of the subscription. Also we will need to do one more wiring step after this...

var Zap = {
    pre_subscribe: function(bundle) {
        bundle.request.method = 'GET';
        bundle.request.headers['Content-Type'] = 'application/x-www-form-urlencoded';
        bundle.request.params = {
            target_url: bundle.subscription_url,
            target_event:bundle.event,
            form_id:bundle.trigger_fields.form_id
        };
        bundle.request.data = $.param({
            target_url: bundle.subscription_url,
            target_event:bundle.event,
            form_id:bundle.trigger_fields.form_id
        });
        return bundle.request;
    },
    post_subscribe: function(bundle) {
        // must return a json serializable object for use in pre_unsubscribe
        var data = JSON.parse(bundle.response.content);
        // we need this in order to build the {{webhook_id}}
        // in the rest hook unsubscribe url
        return {webhook_id: data.id};
    },
    pre_unsubscribe: function(bundle) {
        bundle.request.method = 'DELETE';
        bundle.request.data = null;
        return bundle.request;
    },
    new_form_submission_pre_poll: function(bundle) { 
        bundle.request.method = 'GET';
        bundle.request.headers['Content-Type'] = 'application/x-www-form-urlencoded';
        bundle.request.params = bundle.trigger_fields;
        bundle.request.data = $.param({
            form_id:bundle.trigger_fields.form_id
        });
        return bundle.request;
    }
};

There is a bit to this...but look at the Zapier documentation and it should help. Or post questions here and I will try to answer them...this is getting larger than I expected!

Manage Trigger Settings

Finally, we need to finish up the wiring for the subscriptions...

We then set up the RESTHook methods we created a while back:

And that is it. Hopefully this will save someone some time and learning lessons!