[Ilugc] Tool to identify nearest & fastest sarge rsync mirror

From: karra@xxxxxxxxxxxxxxxxxxxx (Sriram Karra)
Date: Mon Apr 4 19:02:14 2005

On Mar 24, 2005 5:29 PM, Bhuvaneswaran A <bhuvaneswaran@xxxxxxxxx> wrote:

Hi,

I wish to mirror sarge locally for my internal users. I wish to
mirror it using rsync. Is there any tool available to identify the
nearest & fastest sarge rsync mirror ?

I found searching for a similar tool myself, and hacked up the
following script.  The comments should be sufficient to explain what
it does and how it does it.

-karra

P.S.: Note that the script does not consider packet drop rate.  ...
If that is important.. well you can fix the script ;)

#!/usr/bin/perl

##
## Created : Sun Apr  3 20:25:32 2005
##
## Copyright (C) 2005, Sriram Karra <karra@xxxxxxxxxxxxxxxxxxxx>
##
## Distributed under The "Do As You Please (without nuking the copyright
## notice)" License

##
## This perl script compiles a list of all official Debian mirrors, and
## estimates the 'ping times' to each host in the list.  This is likely
## to be useful for admins looking for a suitably parent mirror site for
## their use.
##
## Recommended Usage:
##
##   ./ping-mirrors [ --gen-only ] > ping-times.txt
##   sort -g --key=4 ping-times.txt
##
## The above sort command will sort based on the Avg. times change to
## --key=2 to sort based on Min. ping times and so on.
##
## Sample output of this script:
##
##     [@Spock mirror-speed]$./doit > out
##     min: 524.6 avg: 542.8 max: 572.1 Hostname: debian.logiclinux.com
##     min: 556.8 avg: 564.5 max: 578.0 Hostname: ftp.at.debian.org
##     min: 523.5 avg: 526.1 max: 530.8 Hostname: debian.atnet.at
##     min: 523.5 avg: 527.2 max: 529.4 Hostname: www.at.debian.org
##
##     [@Spock mirror-speed]$sort -g --key=4 out
##     min: 523.5 avg: 526.1 max: 530.8 Hostname: debian.atnet.at
##     min: 523.5 avg: 527.2 max: 529.4 Hostname: www.at.debian.org
##     min: 524.6 avg: 542.8 max: 572.1 Hostname: debian.logiclinux.com
##     min: 556.8 avg: 564.5 max: 578.0 Hostname: ftp.at.debian.org
##
##
## Tips:
##
## - The official mirror list has about 570 hosts.  That is one helluva
##   large list.  It is very much advisable to use the --gen-only flag
##   first, and prune the mirrors.txt file by hand before rerunning the
##   script.
##
## - The script pings each host name with 5 packets.  If you think it is
##   too much, change the $ping_cnt variable below.
##

$ping_cnt=5;

if ($ARGV[0] eq "--gen-only") {
    $gen_only = 1;
} else {
    $gen_only = 0;
}

# Fetch the complete mirror list if not present locally.  If you want
# to regenerate the mirror list, simply delete the old one.
unless (-f "mirrors.txt" ) {
    unless (-f "list-full") {
        !system("wget http://www.debian.org/mirror/list-full ;> /dev/null") or
            die "Unable to fetch list-full from debian.org";
    }
    # Extract the host names from the html page that is `list-full'
    !system("lynx -dump list-full | grep Site: | sed 's/^Site: //' | sed 's/,
/\\n/g' > mirrors.txt") or
        die "Could not extract mirrors.txt";
}

if ($gen_only) {
    exit;
}

$sites = `wc -l mirrors.txt | sed 's/ mirrors.txt//'`;
chomp($sites);
$sec = $sites * 5;

printf stderr "Total Mirror sites = $sites\n";
printf stderr "Pinging mirror site now.  ".
    "This can take REALLY LONG (Approx $sec seconds).\n".
    "Please stand by...\n";

# Now go through each host and ping it and gather some data.

open FIL, "mirrors.txt";
while (<FIL>) {
    chomp;

    # Silently ignore errors like unresolvable host names and such.
    $out = `(ping -q -c $ping_cnt $_ | grep 'min/avg/max')2>/dev/null`;
    $out =~ s/^round-trip.*= //;
    $out =~ s/ ms$//;

    chomp($out);
    if ($out) {
        ($min, $avg, $max) = split(/\//, $out);
        printf("min: %6s avg: %6s max: %6s Hostname: $_\n",
               $min, $avg, $max);
    }
}

--
Well, all's well that ends.

[Ilugc] Tool to identify nearest & fastest sarge rsync mirror

Other related posts: