AngularJS SEO for 404 Status for Document Similar to This Website

Question

I'm trying to figure out how to get 404 document status error for Page Not Found on my AngularJS App to maintain good SEO. I would like to do something similar to how the Red Bull Sound Select website does it, but I'm not sure how are they doing it?

example 404 URL

https://www.redbullsoundselect.com/no-page-here

As you can see in the above example, the URL changes to /404 and you get a 404 document status error for the original path in the URL i.e no-page-here

---

---

On my AngularJS app I just have:

.otherwise({
    class: 'page-not-found',
    title: '404 Page Not Found',
    description: '404 Page Not Found',
    templateUrl: '/app/static/404.html',
    controller: 'mainController as mainCtrl'
});

I could use something like:

otherwise({
    redirectTo: '/404'
});

This is similar to what the redbull site have but it still doesn't give the 404 document status error. ( also changing the URL to 404 rather than maintaining the original URL, which isn't great for SEO either but at least the bad URL would not be indexed by Google.)

I thought I could try making a non existing http request in Angular to get a 404 xhr status but I don't think this would be enough.

I've read some answers that suggested using prerender.io but this is something you have to pay for, which seems a bit much just to get a 404 page working correctly.

Perhaps, there is something we can do in our .htaccess to handle 404 pages diferently? At the moment we have rewrite rules so that any requested resource which does not exist will use /index.html. Perhaps we can make an exception for how 404 pages are handled in Apache.

UPDATE

I found on the redbullsoundselect.com that they are using the following in their app.js

$stateProvider.state("404", {
  url: "/404",
  templateUrl: "/views/site/404.html",
  controller: "NotFoundController"
});

Although, I still don't understand how this logic is working. I looked in their NotFoundController but not much seems to be happening in it.

Could it have something to do with them using ui.router instead of ngRoute as I am?

2nd Update

This is my .htaccess setup:

RewriteEngine On
  # If an existing asset or directory is requested go to it as it is
  RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -f [OR]
  RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -d
  RewriteRule ^ - [L]

# If the requested resource doesn't exist, use index.html RewriteRule ^ /index.html

Inter

While AngularJS and its included routing is handled by the client you can not return a 404 not found by using AngularJS. Thats what you already suggested. Sure, you can create a .htaccess file and return a 404 for all unconfigured AngularJS routes. While the routing of AngularJS is configured in your client application you need to manage the URLs in your .htaccess manually or with a custom script. – Peptidase

@Peptidase thanks, do you how redbullsoundselect.com are doing it? – Inter

I don't think there is special logic behind the scene. I guess their server knows all of the routes, and returns the same index file with the same path, but the only thing it does, it is changing the status code to 404. and the app run as usual to the 404 state (otherwise case). And I'm sure they hate that logic, because it make them to change the url anytime they are add/change states in the app, but google is forcing you to do so you won't have junk urls in google results that redirect to 404 – Lidalidah

@Dvir, but how do they change the status code on a SPA – Inter

@Holy They are doing it from the server. For example, If you are using server with angularjs SPA, you will have to redirect any request to the index.html file, and keep the querystring as is. The server get the request with the url, and he knows that the url doesn't exist so he's changing the response status code to 404 and redirect to the index.html with the querystring (original request: domain/some-invalid-page). Then angular playing by itself, he can't find the state so it will redirect to the 404 state. The only thing that changing in here is the status code. – Lidalidah

@Holy: i have the same problem. Did you resolve your problem and how did you do ? Thanks you so much. – Winegrower

Possible solution to your problem: https://mcmap.net/q/321369/-in-a-single-page-app-what-is-the-right-way-to-deal-with-wrong-urls-404-errors – Triumphal

Answer 1

What RedBull Sound Select is doing

Valid URL: https://www.redbullsoundselect.com/artists/gherbo

Invalid URL: https://www.redbullsoundselect.com/artists/gherbo2

Open DevTools, select "Preserve log" in Network tab, open invalid URL.

1. Angular app is loaded with 200 or 304 (if in cache)
2. API call is made to /api/v1/groups/gherbo2?type=artist
3. API responds with 404
4. Angular code executes window.location.assign("/404"); artists-profile.js:577

Their Express app which serves Angular files knows about valid URL patterns and send 404 if URL doesn't match defined patterns. However, if URL matches a pattern like above invalid URL then Angular uses window.location.assign to navigate to new page.

How will crawler (GoogleBot) behave

1. Let's say Google visits the invalid URL
2. First response code it gets is 200
3. Now Google may or may not execute JS. Source
4. If it executes JS, Angular tries to navigate it to new page. (Here not sure how crawler will react to it)

Overall, in my opinion this is not good setup for SEO (For users, it is good though). We are integrating self-hosted Prerender in our Angular deployment for SEO.

Prerender is open source under MIT license.

We host this as a service at prerender.io but we also open sourced it because we believe basic SEO is a right, not a privilege!

Gaberdine

thanks that's a really good answer. So they have configured their express server with the routes to send to the soundSelect angular app? But when you got to a undefined route you're sent to the 404 page, which is part of the ng-app="soundSelect", as seen in the html tag. Is this correct, the 404 page is part of their angular JS app but giving a 404 document status response, I still don't get it fully. – Inter

And the /artists/gherbo2 URL behaves differently from /no-page-here. The no-page-here is not a configured router for the angularJS app while the gherbo2 is being redirected to the 404 page with window.location.assign("/404") (I think window.location.replace("/404") would be better). – Inter

Interestingly, this redbull site does not seem to have much indexed by google so I get your point about Prerender, it seems to be the only reliable solution and is free if self hosted? – Inter

Their express server is configured to return Angular index html with 200 status when a known URL pattern is requested. Same html file is served with 404 code, when /404 is hit – Gaberdine

one last thing how did you know they were using express for server? – Inter

Response header has x-powered-by:Express – Gaberdine

Answer 2

You can achieve 404 document status error for Page Not Found on your AngularJS App whenever you will get 404 xhr status.

angular.module('app', ['ui.router'])
       .config(config)
       .run(run);

config.$inject = ['$httpProvider', '$stateProvider', '$urlRouterProvider'];

function config($httpProvider, $stateProvider, $urlRouterProvider) {

    $httpProvider.interceptors.push('404');

    // For any unmatched url, redirect to /state1
    $urlRouterProvider.otherwise("/login");

    // Now set up the states
    $stateProvider

    // This will be helpfull if in your application you want to redirect it to state '404' based on some false condition occured.
    .state('404', {
       url: '/404',
       templateUrl: '/views/site/404.html'
    })
}

//This peace of code will be called when you will get http response 404 OR 403 xhr status.
angular.module('app').factory('404', unautherizedService);

unautherizedService.$inject = ['$rootScope', '$q', '$state'];

function unautherizedService($rootScope, $q, $state) {
    return {
        responseError: function(response) {
            if (response.status === 404 || response.status === 403) {
                $state.go('404'); // OR you can fire a broadcast event to logout from app and redirect to 404 page.
                event.preventDefault();
                return $q.reject(response);
            }
            return $q.reject(response);
        }
    };
};

Dolan

thanks this looks interesting. I'll check it out. Currently I'm using ngRoute. Is this only possible with ui.router? – Inter

No this approach is possible with ngRoute also. I prefer ui.router more over ngRoute as ui.router provides more flexibility and advantages. – Dolan

Here are some common reason ui-router is chosen over ngRoute: - ui-router allows for nested views and multiple named views. - ui-router allows for you to have strong-type linking between states based on state names. - There is also the concept of the decorator - states allow you to map different states and you can easily pass information between states via $stateParams. - You can easily determine if you are in a state or parent of a state within your templates via $state. – Dolan

Can I use otherwise({ redirectTo: '/404' }); in a way similar to ng-route – Inter

You can use it but that will give users a wrong msg. As 404 is used only when page is not found or we are trying to access a page which is not available due to some server related issues. otherwise({ redirectTo: '/urlPath' }); is generally used when authentication fails or url path which is not valid. – Dolan

I tried this solution and couldn't get it working. I tested it on this Plunker and it doesn't seem to work, when I go to a bad url it just get's redirected to the homepage. NOTE: You need to download the plunker and run it locally as plunker naturally gives a 404. – Inter

@Varsha: your solution does not work. I think we need to put something on server, something like this bscalable.com/blog-galapagos/2015/6/27/… – Winegrower