Determine nearest mirror PHP
Asked Answered
I

7

8

I maintain a site with a bunch of downloadable files. It is currently hosted on a server in the U.S., but I have recently acquired a new server in Germany. I would like to mirror the downloads to the server in Germany, and have a PHP script on the first server (hosting the website) detect which file mirror to use based on the user's location. For instance, if the user is in Canada, they should download the file from my current server in the U.S. If they're in France, they should get the file from Germany, rather than downloading across the Atlantic. How, then, can I determine which country they are closer to?

I know about MaxMind GeoIP, and have it installed, but that just gives me a country, and AFAIK, there is no way to automatically determine which of my two mirror countries the given country is closest to. I suppose what I could do is go by continent: have users in Asia, Europe, Africa, and Australia get the content from Germany, and have visitors from North and South America get the file from the U.S. If anyone can think of a better solution, I'm open to suggestions.


Well, I guess I am going to go go with my original idea of checking by continents. For others looking to do this sort of thing, that will be a good place to start. The problem will come when I have multiple mirrors in Europe, but the continent idea will have to work for now.

Interregnum answered 28/1, 2012 at 21:3 Comment(9)
You could call Google maps through the google API and calculate the distance between the country the IP-user live in, and your two server destinations, based on which of them is closest, set the download mirror. Then you only have two static variables. One for each server location.Sarthe
I'm waaay outside my area with this, but I think there's a way to use dns to handle this...? A little searching yields this: #1041045Sebastiansebastiano
@Jonasm: not a bad idea, but I believe you can't just use the Maps API to calculate distance, you actually have to show a map, which is definitely unsatisfactory in this case.Interregnum
You can always hide the map inside a non-visible container and just pull the required data. That will be the fastest and easiest way for you i believeSarthe
@Jonasm you have to actually display the map. See code.google.com/apis/maps/terms.html, section 10.1.1 part (g).Interregnum
Instead of creating your own mirrors, have you looked into amazon cloudfront, or akamai, or one of the many other content distribution networks. They would take care of all of this for you.Liebowitz
@Liebowitz In my situation this is not feasible. It is a good idea for those who can afford it, but I'm running this site on a very bare bones budget. Really, I could do without a location-aware mirror system entirely, but I feel like this is something that should be doable and I hope this question (and hopefully a good answer) will be able to be referenced by people needing to do this. It seems a fairly common scenario.Interregnum
Using anycast dns is the best way to do this, to cut a long story short, all your servers get the same ip address and users are seamlessly routed to the closest one without any code being required. This has the advantage of also giving your fall over protection if a server goes down and the ability to add new servers without worrying about a load of changes to your application every time.Forfeiture
PHP.net has mirror selection / offering by country, php.net/source.php?url=/include/ip-to-country.incPike
W
2

Seems there is a lot of developer overhead in the proposed solutions thus far. If this was a problem I had to address in my own applications, I might save me a few hours of work by opting not to reinvent the wheel on this one.

Determining the Closest Mirror (Using Zip Codes)

  1. Maintain a list of postal codes in an array for those mirror servers available.
  2. Determine the postal code of the user agent (e.g. user input or PHP library)
  3. Calculate the distance between the two postal codes (e.g. PHP library)
  4. Proceed with mirror selection based on distance returned

Please keep in mind that a closer distance does not necessarily constitute a faster response time. In the context of your scenario, however, a mirror in one country will obviously be faster than a mirror in another, assuming both mirrors are up. Continue reading for what I consider to be a more "robust" solution.

Resources & Links

The "Maverick" Approach

In my opinion, Mavericks are also known as those innovators, problem solvers, and inventors of these great libraries and frameworks we all use today. Sometimes mistakenly associated with "hackish" ideas, but we embrace the complement :)

  1. Create your own API service on either one of your mirror servers that will accept either a $_GET or $_POST request.

  2. This API service will take an IP address it is given and ping() it, calculating the response times and then taking the average, returning it to the requesting interface (e.g. your frontend portal through which clients are connecting and/or the server trying to determine the closest mirror). The server that responds with the lowest average ought to be your quickest responding server, albeit not necessarily closest. Which is more important to you? See Ping site and return result in PHP for a working ping() function that does not rely on executing shell commands locally (e.g. platform independent).

  3. Last step, derive the IP address of the requesting client and pass it to your API service running on either mirror server in the background. And we all know how to derive the IP, but not as well as you think we might. If you're load balanced or behind a proxy, you may want to first check to see if any of these headers came through (HTTP_FORWARDED, HTTP_FORWARDED_FOR, HTTP_X_FORWARDED, HTTP_X_FORWARDED_FOR, HTTP_CLIENT_IP). If so, that's probably the real IP address of the user agent.

It is at this point (Step 3) where you would compare the averages of response times that each mirror replied with when they went to ping the user agent. Then proceed with selecting which mirror the user agent should download from. The service flow you will have created then resembles something like this:

  1. User agent visits portal
  2. Portal forwards user agent's IP address to API service running separately on both mirrors using a background AJAX/jQuery request (or traditional POST and redirect).
  3. API service running on mirrors pings the IP address it receives and returns an average of the total number of responses it is configured to fetch.
  4. Portal reads the returned averages and compares them.

Hope that helps and happy coding!

Wiseman answered 3/5, 2012 at 16:21 Comment(3)
That is the first really good and easy to implement idea I have seen so far. I like it a lot. One question: how much overhead will this cause? Will the bunch of little requests end up being hard for the server to deal with, figuring about 500 visitors/day?Interregnum
PHP is certainly capable to do above and beyond this, so it is up to the programmer to benchmark and create test units for their code to evaluate its performance. One possibility that comes to mind surrounding your concern is coming up with a caching solution where server selection has been predetermined. Determine the network portion of your user agent's IP address and assign the server to that network (e.g. 192.168.0.0). Then, if an incoming IP address matches a network portion in your database, your portal can skip performing the API call and jump right to server provisioning.Wiseman
Also, another thing to point out - some people configure their routers not to reply to ICMP pings, while others still have no available ports to connect to test (per the ping() reference in my post). It is entirely possible that you will get a request timed out. In such cases, I would update your API service to fall back to an IP to Postal Code distance calculation function which instead of returning the average ping response as a numerical figure, returns the distance as a numerical figure. Again, with the understanding that distance does not necessarily mean a faster connection.Wiseman
D
1

If you have just two mirrors, kick off AJAX requests in your browser that download a 50K file from each server. This is small enough not to represent a huge delay for the user, but large enough to make timer measurement differences significant - though of course you should play with that figure a bit.

Then, once you've got a 'best time', set a JS cookie and redirect to the preferred mirror whenever a download is required. The measurement can be kicked off from a download page in the background, so the user probably won't notice the delay (whilst they are selecting the file they want).

You could even reply with a 'server load' in each AJAX op too, and select the best server not just on response time but on current load also. So, a UK user would use the US server, even though the closest server is in Germany, if the load on the latter is significantly higher than the first.

Diocesan answered 3/5, 2012 at 10:55 Comment(1)
I actually like this ideaSandysandye
P
0

I dont remember any library that can do this. But instead of build a system, If I have an idea, that might be able to help you out.

Calculate the distance between two IP's using this distance calculator. Or find out the latitude and longitude of the two IP address (one server) and (one guest) and calculate the distance. Here is a pseudocode to do that

distance = ( 3956 *2 * ASIN( SQRT( POWER( SIN( ( 34.1012181 - ABS( latitude ) ) * PI( ) /180 /2 ) , 2 ) + COS( 34.1012181 * PI( ) /180 ) * COS( ABS( latitude ) * PI( ) /180 ) * POWER( SIN( ( ABS( - 118.325739 ) - ABS( longitude ) ) * PI( ) /180 /2 ) , 2 ) ) ))
Platitudinous answered 1/5, 2012 at 2:6 Comment(0)
P
0

Do a traceroute (configure the traceroute client to not resolve hostnames and with a small timeout).

Based on the number of hops and the location of the traceroute client (I supose it's the same as the PHP script) select between U.S. and Germany.

Geographical distance has nothing to do with network distance and network speed, or bandwidth costs.

Alternatively to the traceroute (since its a hackish, small code solution), I recommend you to use the $_SERVER["REMOTE_ADDR"] and look it up in a geo ip database to get the country code. If the country code is not one of the countries on the American continents, to avoid crossing a crowded internet backbone, fallback to Germany (additionaly you could condition the country code to be from Europe).

Once you setup the geo ip database, I recommend you convert the IP addresses in the ranges from dotted format to integer format for speed and ease of querying.

From my experience with the above geo ip database, it misses so rarely it doesn't matter.

Pero answered 3/5, 2012 at 10:43 Comment(0)
W
0

its not more easy use some library like geoip as u said and use the latitude and longitude to compare the distance betwen the mirrors and the user ?

I think its less complicated and its much easy to implemen,works for N mirrors and u dont need to ask for Zip or another kind of data to make the references

Wristwatch answered 3/5, 2012 at 17:10 Comment(0)
D
0

I'm pretty late to the party but wouldn't be the simplest solution to just ping booth and see which is faster.

Deon answered 1/10, 2022 at 18:3 Comment(0)
L
0

Sorry: Also quite late to the party. :-)

With access to the Maxmind GeoLite2 City Database you can optain latitude and longitude of your mirrors and your visitors and redirect accordingly.

I've created a small script that tells you which server it would redirect you too based on the nearest server to you. Instead of the echo statements you would just do a header("Location: .....); redirect.

Here is the complete exmaple:

<?php
/**
 * Credit for the function: https://stackoverflow.com/a/10054282
 * Calculates the great-circle distance between two points, with
 * the Vincenty formula.
 * @param float $latitudeFrom Latitude of start point in [deg decimal]
 * @param float $longitudeFrom Longitude of start point in [deg decimal]
 * @param float $latitudeTo Latitude of target point in [deg decimal]
 * @param float $longitudeTo Longitude of target point in [deg decimal]
 * @param float $earthRadius Mean earth radius in [m]
 * @return float Distance between points in [m] (same as earthRadius)
 */
function vincentyGreatCircleDistance(
  $latitudeFrom, $longitudeFrom, $latitudeTo, $longitudeTo, $earthRadius = 6371000)
{
  // convert from degrees to radians
  $latFrom = deg2rad($latitudeFrom);
  $lonFrom = deg2rad($longitudeFrom);
  $latTo = deg2rad($latitudeTo);
  $lonTo = deg2rad($longitudeTo);

  $lonDelta = $lonTo - $lonFrom;
  $a = pow(cos($latTo) * sin($lonDelta), 2) +
    pow(cos($latFrom) * sin($latTo) - sin($latFrom) * cos($latTo) * cos($lonDelta), 2);
  $b = sin($latFrom) * sin($latTo) + cos($latFrom) * cos($latTo) * cos($lonDelta);

  $angle = atan2(sqrt($a), $b);
  return $angle * $earthRadius;
}

$download_servers = array(      0 => array(     'hostname'  => "ftp.bu.edu",
                                                'longitude' => null,
                                                'latitude'  => null,
                                                'city'      => null,
                                                'distance'  => null),
                                1 => array(     'hostname'  => "www.softlayer.com",
                                                'longitude' => null,
                                                'latitude'  => null,
                                                'city'      => null,
                                                'distance'  => null),
                                2 => array(     'hostname'  => "download.nust.na",
                                                'longitude' => null,
                                                'latitude'  => null,
                                                'city'      => null,
                                                'distance'  => null),
                                3 => array(     'hostname'  => "mirror.isoc.org.il",
                                                'longitude' => null,
                                                'latitude'  => null,
                                                'city'      => null,
                                                'distance'  => null),
                                4 => array(     'hostname'  => "download.nus.edu.sg",
                                                'longitude' => null,
                                                'latitude'  => null,
                                                'city'      => null,
                                                'distance'  => null),
                                5 => array(     'hostname'  => "mirror.yandex.ru",
                                                'longitude' => null,
                                                'latitude'  => null,
                                                'city'      => null,
                                                'distance'  => null),
                                6 => array(     'hostname'  => "ftp.iij.ad.jp",
                                                'longitude' => null,
                                                'latitude'  => null,
                                                'city'      => null,
                                                'distance'  => null)
                          );

require_once( __DIR__ . "/maxmind/geoip2.phar");
use GeoIp2\Database\Reader;

// City DB
$reader = new Reader( __DIR__ . '/maxmind/GeoLite2-City.mmdb');
$record = $reader->city($_SERVER['REMOTE_ADDR']);

$i = 0;
foreach ($download_servers as $server) {
   //$record2 = $reader->city($server['ip']);
   $record2 = $reader->city(gethostbyname($server['hostname']));
   if (!isset($download_servers[$i]['longitude'])) {
      $download_servers[$i]['longitude'] = $record2->location->longitude;
      $download_servers[$i]['latitude'] = $record2->location->latitude;
   }
   $download_servers[$i]['city'] = $record2->city->name;
   $download_servers[$i]['distance'] = vincentyGreatCircleDistance(     $record->location->latitude, $record->location->longitude,
                                                                        $download_servers[$i]['latitude'], $download_servers[$i]['longitude']);
   $i++;
}

$dists = array_column($download_servers, 'distance');
$min = array_search(min($dists), $dists, true);

echo "Nearest download server is: " . $download_servers[$min]['hostname'] . " in " . $download_servers[$min]['city']  . "<br>";
echo "Distance to you (" . $record->city->name . ") is: " . $download_servers[$min]['distance'] / 1000 . " km";
Loutish answered 13/3, 2023 at 15:21 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.