Update WebExtension webRequest.onBeforeRequest listener URL settings from separate script
Asked Answered
S

2

1

I am currently creating a WebExtension in which I register a listener on the web requests being made, as such:

main.js:

chrome.webRequest.onBeforeRequest.addListener(main_function, {urls: sites}, ["blocking"]);

where sites is an array containing a list of URLs loaded from a settings page.

Upon changing these settings, which live in separate HTML and JavaScript files, I want to update the aforementioned listener to now contain the new list of sites.

I'm having trouble getting access to this listener from the Settings JavaScript file, as

 onBeforeRequest.removeListener(callback)

requires the original callback as an argument.

Does anyone know how I can, from a different JavaScript file, update this listener?

Example of problem (3 files):

manifest.json:

{
    ...
    "permissions": ["webRequest", "webRequestBlocking", "storage"],
    "background: { "scripts": ["main.js"] },
    "options_ui": { "page":"settings.html" }
}

where settings.html in turn loads settings.js.

main.js:

/* I wont type out the code to load this array from settings. You will have to imagine it being loaded */
all_urls = ["http://www.google.com", "http://www.stackoverflow.com"];

function example_callback() {
    console.log("Hello World!");
}

chome.webRequest.onBeforeRequest.addListener(example_callback, {urls: all_urls}, ["blocking"]);

settings.js:

/* Again, imagine that the settings are edited by this file and saved to storage. */
all_urls = ["https://www.google.com"];

/* This is where the problem lies. Listener cannot be removed, as callback is not available in this file */
chrome.webRequest.onBeforeRequest.removeListener(); // Wont work
chrome.webRequest.onBeforeRequest.addListener(example_callback, {urls: all_urls}, ["blocking"]);
Symphonious answered 14/8, 2016 at 8:10 Comment(5)
Please provide a complete minimal reproducible example. While we can guess (with, I believe, reasonable accuracy) the exact problem you are having, it is not possible to be certain what your problem is without complete code.Resht
I added more code in hope that you will get a more complete picture of the problem. As stated in my answer however, I have found what I believe to be the solution. I will edit my answer as to reflect how the messaging pertains to the problem.Symphonious
Thank you for the additional code. Just to check my (current) understanding: By "settings page" you mean an options page that is loaded using the manifest.json options_ui key. Is that correct?Resht
I'm assuming your main.js is loaded from the manifest.json background key. If I am not correct in believing that settings.js is loaded from the HTML file pointed to by a manifest.json options_ui key, then how is the separate script loaded?Resht
That is correct. I'll provide a sample manifest aswell.Symphonious
R
2

Communicating between your options page (or panel) JavaScript and background scripts

There are two general methods of communicating between your options page or a panel, and your background scripts.

  1. You can directly access variables and functions that are in your background scripts from your options page JavaScript by first obtaining the Window object for your background scripts by using extension.getBackgroundPage(). If you want to directly access other pages from your background script you can use extension.getViews() to get the Window object for the background page and, if defined, any popup/panel, option page, or tab containing content which is packaged with the extension.

    For the code that is in the question you could do:

let backgroundPage = chrome.extension.getBackgroundPage();
chrome.webRequest.onBeforeRequest.removeListener(backgroundPage.example_callback);
  1. You can send messages back an forth between the pages using runtime.sendMessage(), runtime.onMessage, and/or runtime.connect().

    If you send messages back and forth, then you need to make choices as to what those messages will be used for and their contents. Do you send all the data, or just a message that the data was updated? Are you going to use messages for multiple purposes? If so, how are your listener(s) going to determine what message is for which part of your script. You will need to impose some type of format on the messages. The more things that you need to accomplish with these messages, the more complex the format that you need to impose.

Example Code:

The following extension logs web requests to the console. Depending on the users choice, it will log

  1. Nothing
  2. All requests to mozilla.org
  3. All web requests

It implements the same page as both an options_ui page and a default_popup for a browser_action button. The user can select from the above 3 logging options and how the option data is communicated to the background page:

  1. Options are stored to storage.local in the options.js code. Then, the options.js directly invokes the getOptions() function in the background.js file to have the background script re-read the options.
  2. Options are stored to storage.local in the options.js code. Then, the options.js sends a optionsUpdated message to the background script that the options have been updated. The background script then re-reads the options.
  3. A optionsData message is sent from the options.js code to the background page when the options are change which contains a data payload with all of the options. The options are then stored to storage.local in the background script. Once the options are stored, the background script sends a optionsStored message back to the options.js code. The options.js code then indicates to the user that the options have been saved.

Messages that are sent between the background.js and options.js are an object that has the following format:

{
    type: //String describing the type of message:
          //  'optionsUpdated' 'optionsData', or 'optionsStored'
    data: //Object containing the options data
}

//Options data object:
{
    loggingUrls: //Array of URL match strings for webRequest requestFilter
    useDirect: //Number: 0, 1, 2 indicating the method of communication between
               // options.js and background.js
               // 0 = Directly invoke functions in background script from options/panel code
               // 1 = Send a message that data was updated
               // 2 = Send a message with all options data
}

The extension has been testing in both Firefox and Google Chrome:

manifest.json:

{
    "description": "Demonstrate Changing webRequest.RequestFilter",
    "manifest_version": 2,
    "name": "webrequest.requestfilter-demo",
    "version": "0.1",

    "applications": {
        "gecko": {
             //Firefox: must define id to use option_ui:
            "id": "[email protected]",
            "strict_min_version": "42.0",
            "strict_max_version": "51.*"
        }
    },

    "permissions": [
        "storage",
        "webRequest",
        "webRequestBlocking",
        "<all_urls>" //Required for Google Chrome. Not, currently, needed for Firefox.
    ],

    "background": {
        "scripts": [
            "background.js"
        ]
    },

    "browser_action": {
        "default_icon": {
            "48": "myIcon.png"
        },
        "default_title": "Currently NOT logging. Click to start logging only mozilla.org",
        "browser_style": true,
        "default_popup": "options.html"
    },

    "options_ui": {
      "page": "options.html",
      "chrome_style": true
    }
}

background.js:

var webRequestExtraInfo = ["blocking"];

var useDirect=0; //Holds the state of how we communicate with options.js
const useDirectTypes=[ 'Directly invoke functions in background script'
                      ,'Send a message that data was updated'
                      ,'Send a message with all options data'];


//Register the message listener 
chrome.runtime.onMessage.addListener(receiveMessage);

function receiveMessage(message,sender,sendResponse){
    //Receives a message that must be an object with a property 'type'.
    //  This format is imposed because in a larger extension we may
    //  be using messages for multiple purposes. Having the 'type'
    //  provides a defined way for other parts of the extension to
    //  both indicate the purpose of the message and send arbitrary
    //  data (other properties in the object).
    console.log('Received message: ',message);
    if(typeof message !== 'object' || !message.hasOwnProperty('type')){
        //Message does not have the format we have imposed for our use.
        //Message is not one we understand.
        return;
    }
    if(message.type === "optionsUpdated"){
        //The options have been updated and stored by options.js.
        //Re-read all options.
        getOptions();
    }
    if(message.type === "optionsData"){
        saveOptionsSentAsData(message.data,function(){
            //Callback function executed once data is stored in storage.local
            console.log('Sending response back to options page/panel');
            //Send a message back to options.js that the data has been stored.
            sendResponse({type:'optionsStored'});
            //Re-read all options.
            getOptions();
        });
        //Return true to leave the message channel open so we can 
        //  asynchronously send a message back to options.js that the
        //  data has actually been stored.
        return true;
    }
}

function getOptions(){
    //Options are normally in storage.sync (sync'ed across the profile).
    //This example is using storage.local.
    //Firefox does not currently support storage.sync.
    chrome.storage.local.get({
        loggingUrls: [''],
        useDirect: 0
    }, function(items) {
        if(typeof items.useDirect !== 'number' || items.useDirect<0 || items.useDirect>2) {
            items.useDirect=0;
        }
        useDirect = items.useDirect;
        updateLogging(items.loggingUrls);
        console.log('useDirect=' + useDirectTypes[useDirect]);
    });
}

function saveOptionsSentAsData(data,callback) {
    //Options data received as a message from options.js is 
    //  stored in storeage.local.
    chrome.storage.local.set(data, function() {
        //Invoke a callback function if we were passed one.
        if(typeof callback === 'function'){
            callback();
        }
    });
}

function updateLogging(urlArray){
    //The match URLs for the webRequest listener are passed in as an 
    //  array.  Check to make sure it is an array, and forward to
    //  function that adds the listener as a requestFilter.
    if(typeof urlArray === "object" &&  Array.isArray(urlArray)
        && urlArray[0].length>0){
        startListeningToWebRequests({urls: urlArray});
    }else{
        //The argument was not an array
        stopListeningToWebRequests();
    }
}

function logURL(requestDetails) {
    //Log the webRequest to the Console.
    console.log("Loading: " + requestDetails.url);
    return {}; //Return object in case this is a  blocking listener
}

function stopListeningToWebRequests() {
    if(chrome.webRequest.onBeforeRequest.hasListener(logURL)) {
        //Don't really need to check for the listener, as removeListener for a 
        //  function which is not listening does nothing (no-op).
        chrome.webRequest.onBeforeRequest.removeListener(logURL);
        console.log("STOPPED logging all Web Requests");
    }
}

function startListeningToWebRequests(requestFilter) {
    stopListeningToWebRequests();
    //Start listening to webRequests
    chrome.webRequest.onBeforeRequest
                     .addListener(logURL,requestFilter,webRequestExtraInfo);
    //Log to the console the requestFilter that is being used
    console.log("Logging Web Requests:", requestFilter, "-->", requestFilter.urls);
}

//Read the options stored from prior runs of the extension.
getOptions();

//On Firefox, open the Browser Console:
//To determine if this is Chrome, multiple methods which are not implemented
//  in Firefox are checked. Multiple ones are used as Firefox will eventually 
//  support more APIs.
var isChrome = !!chrome.extension.setUpdateUrlData
               && !!chrome.runtime.reload
               && !!chrome.runtime.restart;
if(!isChrome) {
    //In Firefox cause the Browser Console to open by using alert()
    window.alert('Open the console. isChrome=' + isChrome);
}

options.js:

// Saves options to chrome.storage.local.
// It is recommended by Google that options be saved to chrome.storage.sync.
// Firefox does not yet support storage.sync.
function saveOptions(data, callback) {
    chrome.storage.local.set(data, function() {
        if(typeof callback === 'function'){
            callback();
        }
        // Update status to let user know options were saved.
        notifyOptionsSaved();
    });
}

function optionsChanged() {
    //Get the selected option values from the DOM
    let loggingUrls = document.getElementById('loggingUrls').value;
    let useDirectValue = document.getElementById('useDirect').value;
    useDirectValue = +useDirectValue; //Force to number, not string
    //Put all the option data in a single object
    let optionData = {
        loggingUrls: [loggingUrls],
        useDirect: useDirectValue
    }
    if(useDirectValue == 0 ) {
        //We save the options in the options page, or popup
        saveOptions(optionData, function(){
            //After the save is complete:
            //The getOptions() functon already exists to retrieve options from
            //  storage.local upon startup of the extension. It is easiest to use that.
            //  We could remove and add the listener here, but that code already
            //  exists in background.js. There is no reason to duplicate the code here.
            let backgroundPage = chrome.extension.getBackgroundPage();
            backgroundPage.getOptions();
        });
    } else if (useDirectValue == 1) {
        //We save the options in the options page, or popup
        saveOptions(optionData, function(){
            //Send a message to background.js that options in storage.local were updated.
            chrome.runtime.sendMessage({type:'optionsUpdated'});
        });
    } else {
        //Send all the options data to background.js and let it be dealt with there.
        chrome.runtime.sendMessage({
            type:'optionsData',
            data: optionData
        }, function(message){
            //Get a message back that may indicate we have stored the data.
            if(typeof message === 'object' && message.hasOwnProperty('type')){
                if(message.type === 'optionsStored') {
                    //The message received back indicated the option data has
                    //  been stored by background.js.
                    //Notify the user that the options have been saved.
                    notifyOptionsSaved();
                }
            }
        });
    }
}

// Restores select box using the preferences
// stored in chrome.storage.
function useStoredOptionsForDisplayInDOM() {
    chrome.storage.local.get({
        loggingUrls: [''],
        useDirect: 0
    }, function(items) {
        //Store retrieved options as the selected values in the DOM
        document.getElementById('loggingUrls').value = items.loggingUrls[0];
        document.getElementById('useDirect').value = items.useDirect;
    });
    //notifyStatusChange('Option read');
}

function notifyOptionsSaved(callback){
    //Notify the user that the options have been saved
    notifyStatusChange('Options saved.',callback);
}

function notifyStatusChange(newStatus,callback){
    let status = document.getElementById('status');
    status.textContent = newStatus;
    //Clear the notification after a second
    setTimeout(function() {
        status.textContent = '';
        if(typeof callback === 'function'){
            callback();
        }
    }, 1000);
}

document.addEventListener('DOMContentLoaded', useStoredOptionsForDisplayInDOM);
document.getElementById('optionsArea').addEventListener('change',optionsChanged);

options.html:

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8">
    <title>WebRequest Logging Options</title>
    <style>
        body: { padding: 10px; }
    </style>
</head>

<body>
    <div id="optionsArea">
        Log Web Requests from:
        <select id="loggingUrls">
            <option value="">None</option>
            <option value="*://*.mozilla.org/*">Mozilla.org</option>
            <option value="<all_urls>">All URLs</option>
        </select>
        <br/>
        Communication with background page:
        <select id="useDirect">
            <option value="0">Direct</option>
            <option value="1">Message Updated</option>
            <option value="2">Message all Data</option>
        </select>
    </div>
    <div id="status" style="top:0px;display:inline-block;"></div>

    <script src="options.js"></script>
</body>
</html>

The code in this answer was combined and modified from that in the question I answered here, and my answer here, which is based on code the OP for that question provided in a GitHub repository. The options.js and options.html files had their start with code found on developer.chrome.com.

Resht answered 14/8, 2016 at 18:4 Comment(2)
@VonL, I have updated this answer with how to directly access the code in your background scripts from the code in your options page.Resht
Cool! It looks like your solution employs the message API I mentioned in my answer, so I'd say we are both correct in our answers. Which answer do I check as correct for this question? I'm assuming that I can only check one. EDIT: I see that you added another route to solving the problem. Since that is the case, I consider your answer to be the more correct one.Symphonious
S
0

The solution is to use the WebExtension message API, as such:

settings.js:

...
/* Settings now outdated */
chrome.runtime.sendMessage(message);
...

main.js

...
chrome.runtime.onMessage.addListener( (message) => {
    /* Update listener */
    chrome.webRequest.onBeforeRequest.removeListener(example_callback);
    chrome.webRequest.onBeforeRequest.addListener(example_callback, {urls: all_urls}, ["blocking"]);
});
...

Relevant section from documentation: https://developer.mozilla.org/en-US/Add-ons/WebExtensions/Content_scripts#Communicating_with_background_scripts

Symphonious answered 14/8, 2016 at 10:19 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.