Make browser submit additional HTTP-Header if click on hyperlink

Question

Is there a way to make the webbrowser submit additional HTTP header if the user clicks on a link?

Background: In our environment every http-request has a unique ID on the server side. See https://serverfault.com/questions/797609/apache-x-request-id-like-in-heroku

If your web application receives a http-request, I would like to know which page was the page before. The http referrer is not enough, since the user could use several tabs in his browser.

I would like to avoid to put the ugly request-id into every GET request which gets send from the browser to the server. Up to now our URLs are nice.

My prefered solution would be some JavaScript magic which adds the request-id of the current page into the next http request.

Steps in detail:

1. browser access URL http://example.com/search
2. web server receives http request with request ID 123
3. web server sends content of the URL to the browser (a search page). The page includes the request ID 123 somewhere
4. the user searches for "foobar".
5. the web browser submits a http request to the server and includes the previous request id somehow.
6. web server receives second http request (ID 456) and can access the value of the first request (ID 123) somehow.
7. Web server can store the relation "123 --> 456" in a database for later analysis.

My goal is to track the relations "123 --> 456". Above solution is just a strategy to get to the goal. Other strategies are welcome.

We use the web framework django. But AFAIK this does matter in this context.

the user could use several tabs in his browser

I elaborate what that means for a matching solution. The sequence of requests which come from one user does not solve the issue.

One use with several tabs:

1. user looks at page A in tab1
2. user looks at page B in tab2
3. user follows a link on page A to page C
4. user follows a link on page C to page D
5. user follows a link on page B (tab2) to page E.

I want to know see two sequences:

A -> C -> D

And

B -> E

Erotic

just append the id to all link (GET param) or use ajax? – Evolutionary

@appleapple I would like to avoid to put the ugly request-id into every GET request which gets send from the browser to the server. Up to now our URLs are nice. – Erotic

oh, miss that sentence. BTW, I don't think a nice URL is important. at least you can redirect to a cleaner one. – Evolutionary

Answer 1

The only modern 'sane' option here is to use a ServiceWorker.

A ServiceWorker can intercept HTTP requests for a domain you control and decorate it with more headers.

A ServiceWorker works 'outside' of a browser tab, and if multiple tabs are open with the same website, the same serviceworker will be used for all of them.

A full tutorial on how to accomplish that is definitely too much for this answer box, but intercepting and doing stuff with HTTP requests is a big use-case, so off-site sources will usually have this as an example.

I would say that this is kind of a bad idea. If you think you need this, maybe you can handle this in a different way. A common way to do this might be using cookies instead.

Spoilfive

Thanks this is thorough! – Sympathy

Is this still a solution in 2021? Could you share some information or links on how to achieve this using cookies? – Judyjudye

@PaulM you probably still shouldn't do this. If you have questions about how cookies work, open a new question. – Spoilfive

Thanks @Evert, I had a signed URL from S3 which I wanted to be downloaded with a different name. I just used AJAX to fetch and made the download with different filename. The original idea was to send a header with filename to S3 to download with the different name. Anyways, different use case than the question posted by OP. – Judyjudye

@PaulM that solution makes sense! Might be a bit tougher with large files though – Spoilfive

Answer 2

We can modify request headers using:

• .setRequestHeader() method of XMLHttpRequest() object (in same or allowed origins).
• Editing the headers in browser console or using some complement (it is not practical).
• Performing the request from the server side e.g using CURL, wget, or some library (client->serverProxy->url with custom headers ).

It is not possible (using javascript) to change the headers sent by browser in a request like <a href=""></a> because at least now, the http content negotiation is a browser's inner capability (except in part using XMLHttpRequest in same or allowed origins).

Then, in my opinion, as @Evert said you have two practical ways (a third in fact) to achieve your goal, performing a server proxy or using cookies. Here you have a very simple way using window.localStorage:

LocalStorage example

if (!localStorage.getItem("ids")) {//<-- the place in which we store the behavior
  localStorage.setItem("ids", 'somevalue')
} else {
  var ids = JSON.parse(localStorage.getItem("ids"));
  ids.ids.push(id);//<-- we add some value
  localStorage.setItem("ids", JSON.stringify(ids));  
}

Full example here: https://jsfiddle.net/hy4rzob9/ press run several times and you'll see that we store each visit, of course, in your implementation you have to replace the random number for a unique identifier of each page.

LocalStorage example with several tabs

Taking into account the update, we could store the history using also document.referrer with localStorage with something like this:

var session = Math.random();

if(!localStorage.getItem("routes")){//<-- first time
    var routes = {};
routes[session] = [document.location.href];

localStorage.setItem("routes", JSON.stringify(routes))

}else{

    var routes = JSON.parse(localStorage.getItem("routes"));

    if(!document.referrer){
        routes[session] = [document.location.href];//<-- new root
    }else{

        for(let ses in routes){
             if(routes[ses].includes(document.referrer)){
                routes[ses].push(document.location.href); 
             }
        }
    }

    localStorage.setItem("routes", JSON.stringify(routes))


}

var r = JSON.parse(localStorage.getItem("routes"));

console.log(r);

Full example here https://codesandbox.io/s/qk99o4vy7q, to emulate your example open this https://qk99o4vy7q.codesandbox.io/a.html (represents A) and open in a new tab https://qk99o4vy7q.codesandbox.io/b.html (represents B), navigate in both tabs and see the console. This example won't work if we share some referrer, because we can't differentiate between referrers if we attach nothing in the URL. A -> C -> D and B -> E will work, but A -> C -> D and B -> E -> A won't.

Ping example

There is other way, that is easy but has a limitation in browser compatibility, that is using ping attribute of <a> like this:

<a href="https://www.google.com/" ping="trackPing.py">Link to track</a>

ping Contains a space-separated list of URLs to which, when the hyperlink is followed, POST requests with the body PING will be sent by the browser (in the background). Typically used for tracking.

Open the console -> network, delete all, run the snippet and click in the link, if your browser supports it, you will see that the browser send a POST request to trackPing.py (I guess doesn't exist in SO), that post is void but you could track the environmental variables such as request.environ['REMOTE_ADDR'] or something.

Howlyn

Does this work is the user has several browser tabs? I elaborated the question to illustrate my concerns. – Erotic

@Erotic I've updated the answer, see this this codesandbox.io/s/qk99o4vy7q I've also specified how to emulate your case. – Howlyn

Answer 3

First of all, sorry for my english.

Edit:

After reading your edit, I realised that my answer didn't fit at all, because of the tabs.

It is not possible to modify directly the way the browser makes a get request. Knowing that, your posibilities are:

• Use GET parameters. I know you try to avoid this.
• As @Evert said, use ServiceWorkers. It is the cleanest way to modify a request before it leaves the browser.
• The last approach (an an easy one) is similar to @Emeeus's, but instead of using localStorage, whose values are shared between tabs, you should use sessionStorage, whose values are tab-independant. Also, instead of store the entire route, you should store just a random ID. This ID will work as the identification of the chain of requests for an specific tab. Then, once your webserver returns each Request-ID for example using <meta name="request_id" content="123" /> you just need to make a request via ajax to an specific tracking endpoint and store:
  • chain_id (stored in sessionStorage)
  • request_id (stored in head > meta)
  • timestamp (generated in webserver)
  • session_id (accesible from webserver). You can avoid this, but it is still useful for checking purposes.

The request to store the route is made after you page is loaded, instead of before. This approach is quite similar to how Analytics works.

// generate an unique code and store it in sessionStorage.
if (!sessionStorage.getItem('chain_id')) { 
    sessionStorage.setItem('chain_id', 'a7835e0a-3ee9-e981-...');
}

// Then, if you use JQuery:
$(document).ready(function() {
    $.ajax({
        type: "POST",
        url: 'your/tracking/endpoint/',
        data: {
            'chain_id': sessionStorage.getItem('chain_id'),
            'request_id': document.querySelector("meta[name='request_id']").getAttribute('content'),
        }
    });
});

Note: It is preferable to don't use JQuery to handle tracking requests neither wait until document is fully loaded. It is just an example.

And that's all. You have the relation between user-agent, the chain, the request and the timestamp of the request, so if you need to know what request was made before or after a given one, you just need to lookup in the database using the Chain-ID and the timestamp as filters.

The django model for your requests could be.

from django.db import models
from django.contrib.sessions.models import Session


class Request(models.Model):
    session = models.ForeignKey(Session)
    chain_id = models.Charfield(max_length=100)
    request_id = models.WhatEverField...
    request_url = models.URLField(max_length=200)
    created = models.DateTimeField(auto_now_add=True)

I hope it helps.

Commandment

The solution needs to work for several browser tabs. I elaborated the question. Please tell me if something is not clear. Thank you. – Erotic

The need to use several browser tabs independently, complicates your question a little. I'm going to edit my answer. – Commandment

Answer 4

I don't know if this will help, but I think maybe Ajax will do, like set additional header inside onclick event listener, as for request id, if it's not something that sensitive then you could use cookie for the container, or maybe something much better ...

Viewless

I am unsure if AJAX will can intercept the http request if a user follows the href of a link. This creates a GET request from the browser to the server. I know no way to add additional headers via AJAX. – Erotic

Send it using POST reuest like : onclick(function(){ //set header //post to destination //retrive next data, attach to cookie, then redirect user to clicked url }) – Viewless