Sunday, June 29, 2008

Dataone Account Usage Checker in PERL

The following Script in PERL is a Dataone account usage checker.


#!/usr/bin/perl
# http://digitalpbk.blogspot.com/2008/06/free-dataone-account-usage-checker-perl.html

use WWW::Mechanize;
use HTTP::Cookies;
use HTTP::Request::Common;

my $verbose = 0;
my $username;
my $password;
my $url="http://10.240.64.195/weblogin.jsp";

for(my $i=0;$i<=$#ARGV;$i++)
{
if($ARGV[$i] eq "-v" or $ARGV[$i] eq "--verbose"){ $verbose = 1; }
elsif($ARGV[$i] eq "-u" or $ARGV[$i] eq "--user"){ $username = $ARGV[++$i]; }
elsif($ARGV[$i] eq "-p" or $ARGV[$i] eq "--pass"){ $password = $ARGV[++$i]; }
elsif($ARGV[$i] eq "-i" or $ARGV[$i] eq "--url"){ $url = $ARGV[++$i]; }
}
unless($username)
{
print "Username : ";chomp($username = <STDIN>);
}
unless($password)
{
print "Password : ";
`stty -echo`;
chomp($password = <STDIN>);
`stty echo`;
print "\n";
}
$cj = HTTP::Cookies->new(file => "cookie.jar",autosave=>1);
$mech=WWW::Mechanize->new(cookie_jar=>$cj);

print "GET $url\n" if($verbose);
$mech->get($url);
exit unless($mech->success());

my $cnt = $mech->response->as_string;

$cnt =~ s[action="../(.*?)"][action="$1"]is;
$mech->update_html($cnt);

$mech->form_number(1);
$mech->field("username",$username);
$mech->field("password",$password);
print "LOGIN\n" if($verbose);
$mech->submit();

exit unless($mech->success());
$cnt = $mech->response->as_string;

$|=1;
my $uri;my $maxre=0;
while($cnt =~ m/location.replace\('(.*?)'\)/is)
{
print "FOLLOWING .. " if($verbose);

$uri = $1;
if($uri =~ m/loginerr/g)
{
print "\nLOGIN Failed !\n";
exit;
}

$mech->get($uri);
exit unless($mech->success());
print $uri,"\n" if($verbose);
$cnt = $mech->response->as_string;
last if($maxre++ > 4);
}

exit if($maxre > 4);
print "GET Service Records\n" if($verbose);
$mech->get("serviceRecords.jsp");
exit unless($mech->success());
$cnt = $mech->response->as_string;

$mech->form_number(1);
$mech->submit();
print "POST Form\n" if($verbose);

$cnt = $mech->response->content;
$cnt = $1 if($cnt =~ m:<body[^>]*>(.*?)</body>:sig);

$cnt =~ s/<[^>]+>//sig;
$cnt =~ s/&nbsp;/ /sig;
$cnt =~ s/([\n\r\t ]){2,}/;/sig;

$cnt = $1 if($cnt =~ m/(Total Send Volume([^;]+;){10})/);

@sl = split /;/,$cnt;

$sl[4] = "Excluding Night";
for($i=0;$i<=($#sl/2);$i++)
{
printf("%25s : %s\n",$sl[$i],$sl[$i+5]);
}

$mech->get("../logout.jsp");



Installation

On Linux
Copy paste to a new file data1.pl
Install WWW::Mechanize if not already installed.

chmod 755 data1.pl


On Windows
Copy paste to a new file data1.pl
Get Active Perl and install if PERL is not installed.
Install Module WWW::Mechanize.

Usage
Linux
./data1.pl Options


Windows, not TESTED
perl data1.pl Options

PS : Comment `stty` lines when using on windows or ignore the errors
perl data1.pl -u username -p password


Options
No need any if your not good at the Command line.
-v to turn on Verbose, describe what is being done.
-u username to specify username
-p password to specify password

...
Any more doubts ?
Use the link that says comment
;)


100th

Post

Tuesday, June 24, 2008

500 I won't open a connection to 192.168.1.232 (only to 59.93.9.97)

FTP Problem

It says it wont open a connection to the host !!
Dont know what the problem is .

Solution is to use Passive mode


ftp> passive
Passive mode on.



Not of much of a post.
But just felt it might be useful to me atleast :D

Solved TV in Black and White Problem in Ubuntu 8.04 with Compaq Laptop

?
Finally after a lot of tweakin with the xorg.conf got Full Colour on S-Video Out on my Compaq Presario V3155(3000) AU, with an nVidia GeForce 6150 GPU on Ubuntu 8.04. The notable sections in the xorg.conf are the device section where you have to put the Option's right. You have to choose the right output SVIDEO or COMPOSITE and the TV Format,and in the Monitor section VertRefresh to 50hz also helped.

Please check your TV Manual for the correct VertRefresh Rate.
And the following table for the format






PAL-B

Australia, Belgium, Denmark, Finland, Germany, Guinea, Hong Kong, India, Indonesia, Italy, Malaysia, The Netherlands, Norway, Portugal, Singapore, Spain, Sweden, and Switzerland

PAL-D

China and North Korea

PAL-G

Denmark, Finland, Germany, Italy, Malaysia, The Netherlands, Norway, Portugal, Spain, Sweden, and Switzerland

PAL-H

Belgium

PAL-I

Hong Kong, Ireland, and United Kingdom

PAL-K1

Guinea


PAL-M

Brazil

PAL-N


France, Paraguay, and Uruguay

PAL-NC

Argentina

NTSC-J

Japan

NTSC-M

Canada, Chile, Colombia, Costa Rica, Ecuador, Haiti, Honduras, Mexico, Panama, Puerto Rico, South Korea, Taiwan, United States of America, and Venezuela

HD480i

480 line interlaced

HD480p

480 line progressive

HD720p

720 line progressive

HD1080i

1080 line interlaced

HD1080p

1080 line rogressive

HD576i

576 line interlace


HD576p

576 line progressive



Here is the whole xorg.conf, I guess you can find out the relevant parts from it, else just comment .. I will help you.

Xorg.conf


# nvidia-xconfig: X configuration file generated by nvidia-xconfig
# nvidia-xconfig: version 1.0 (buildmeister@builder26) Thu Feb 14 18:13:41 PST 2008

# nvidia-settings: X configuration file generated by nvidia-settings
# nvidia-settings: version 1.0 (buildd@yellow) Tue Mar 4 20:28:57 UTC 2008
# xorg.conf (X.Org X Window System server configuration file)
#
# This file was generated by dexconf, the Debian X Configuration tool, using
# values from the debconf database.
#
# Edit this file with caution, and see the xorg.conf manual page.
# (Type "man xorg.conf" at the shell prompt.)
#
# This file is automatically updated on xserver-xorg package upgrades *only*
# if it has not been modified since the last upgrade of the xserver-xorg
# package.
#
# If you have edited this file but would like it to be automatically updated
# again, run the following command:
# sudo dpkg-reconfigure -phigh xserver-xorg

Section "ServerLayout"
Identifier "Default Layout"
Screen 0 "Screen0" 0 0
Screen 1 "Screen1" RightOf "Screen0"
InputDevice "Generic Keyboard" "CoreKeyboard"
InputDevice "Configured Mouse"
InputDevice "Synaptics Touchpad"
EndSection

Section "Module"
Load "glx"
EndSection

Section "ServerFlags"
Option "Xinerama" "0"
EndSection

Section "InputDevice"
Identifier "Generic Keyboard"
Driver "kbd"
Option "XkbRules" "xorg"
Option "XkbModel" "pc105"
Option "XkbLayout" "us"
EndSection

Section "InputDevice"
Identifier "Configured Mouse"
Driver "mouse"
Option "CorePointer"
EndSection

Section "InputDevice"
Identifier "Synaptics Touchpad"
Driver "synaptics"
Option "SendCoreEvents" "true"
Option "Device" "/dev/psaux"
Option "Protocol" "auto-dev"
Option "HorizEdgeScroll" "0"
EndSection

Section "Monitor"
Identifier "Configured Monitor"
EndSection

Section "Monitor"
Identifier "Monitor0"
VendorName "Unknown"
ModelName "Nvidia Default Flat Panel"
HorizSync 29.0 - 49.0
VertRefresh 0.0 - 61.0
EndSection

Section "Monitor"
Identifier "Monitor1"
VendorName "Unknown"
ModelName "TV-0"
HorizSync 28.0 - 33.0
VertRefresh 50.0
Option "dpms"
EndSection

Section "Device"
Identifier "Configured Video Device"
Driver "nvidia"
Option "NoLogo" "True"
EndSection

Section "Device"
Identifier "Videocard0"
Driver "nvidia"
VendorName "NVIDIA Corporation"
BoardName "GeForce Go 6150"
BusID "PCI:0:5:0"
Screen 0
EndSection

Section "Device"
Identifier "Videocard1"
Driver "nvidia"
VendorName "NVIDIA Corporation"
BoardName "GeForce Go 6150"
BusID "PCI:0:5:0"
Screen 1
Option "TVOutFormat" "SVIDEO"
Option "TVStandard" "PAL-B"
EndSection

Section "Screen"
Identifier "Default Screen"
Device "Configured Video Device"
Monitor "Configured Monitor"
DefaultDepth 24
EndSection

Section "Screen"
Identifier "Screen0"
Device "Videocard0"
Monitor "Monitor0"
DefaultDepth 24
Option "TwinView" "0"
Option "metamodes" "DFP: nvidia-auto-select +0+0"


SubSection "Display"
Depth 24
Modes "nvidia-auto-select"
EndSubSection
EndSection

Section "Screen"
Identifier "Screen1"
Device "Videocard1"
Monitor "Monitor1"
DefaultDepth 24
Option "TwinView" "0"
Option "metamodes" "TV: nvidia-auto-select +0+0"
SubSection "Display"
Depth 24
Modes "nvidia-auto-select"
EndSubSection
EndSection



Update
Enable xinerema to use extended monitors that span across screens

Those in the dark follow the light jump out of the windows!

Saturday, June 14, 2008

Perl Download Manager

Intro
This is my PERL downloader program, with resume.
Only works if SERVER has Parts enabled.
Well pretty much thats it



#!/usr/bin/perl

use strict;
use threads;
use threads::shared;
use HTTP::Request;
use HTTP::Response;
use Math::Round qw(nearest);
use LWP;



exit unless(defined $ARGV[0] && -e $ARGV[0]);

open HND,$ARGV[0];
NEXT:
while(my $url = <HND>)
{
my $file = substr($url,rindex($url,"/")+1);
$file =~ s/%20/_/g;

chomp $url;

exit if($url eq "" || $file eq "");

print "Downloading $url \n to $file \n";


our @stats:shared = (0,0,0,0,0,0,0,0,0,0,0,0,0,0);
our $exit:shared = 0;
our $tdl:shared = 0;
our $tgl:shared = 0;

our $lastime:shared = time();
our $lasdl:shared = 0;

my $folder = "/home/arun/download/";

select STDOUT; $| = 1;

sub catchintr()
{
my $who = shift;
print "\n$who is Bugging me\nDownload Paused\nExit message dispatched";
$exit = 1;
}

$SIG{INT} = \&catchintr;
$SIG{HUP} = \&catchintr;

sub dloader
{
select STDOUT; $| = 1;

my ($inx,$url,$localfile,$blockstart,$blockend) = @_;

my $length = $blockend - $blockstart + 1;

my $stbit = 0;
if(-f $localfile)
{
open HND, $localfile or die $!;
binmode HND;
seek HND,0,2;
$stbit = tell HND;
close HND;
$tdl += $stbit;
}
else
{
open HND,">$localfile" or die $!;
print HND "";
close HND;
}

my $ua = LWP::UserAgent->new;

my $pcksize = 1024*64;

my $enbit = $stbit+$pcksize-1;
my $parts = int($length / $pcksize)+1;
$enbit = $length - 1 if($enbit > $length);
my $err =0;
my $i = int($stbit / $pcksize);

my $req = HTTP::Request->new('GET',$url);

if($stbit >= $length)
{
print "$localfile Part Complete! $length <=> $stbit\n";
$stats[$inx] = 1;
return;
}
while($stbit < ($length-1) && $err < 100)
{
my $stbit2 = $stbit + $blockstart;
my $enbit2 = $enbit + $blockstart;

my $range = "bytes=$stbit2-$enbit2";
$req->header('Range' => $range);

$i++;

print "\n $inx ... $i / $parts ";
my $res = $ua->request($req);

threads->yield();

if($res->code == 206)
{

open HND, ">>$localfile";
binmode HND;
print HND $res->content;
close HND;

$stbit = $stbit + $pcksize;
$enbit = $stbit + $pcksize - 1;
$enbit = $length - 1 if($enbit > $length);

$tdl += $pcksize;


if($tdl < 1024)
{
print "[$tdl] bytes";
}
elsif($tdl < 1024 * 1024)
{
print "[".nearest(0.01,$tdl/1024,)." KB]";
}
else
{
print "[".nearest(0.01,$tdl/1048576)." MB]";
}

if((time()-$lastime) > 0)
{
print " [ ".nearest(0.01,($tdl-$lasdl) / (1024 * (time() - $lastime) ) ). " kB/s ] ";
$lasdl = $tdl;
$lastime = time();
}

}
elsif($res->code == 416)
{
print "\n\nSeems complete";
exit;
}
else
{
print "\nGlitch $err : ";
print $res->code;
print "\nResuming...\n";
sleep 1;
$i--;
$err++;
}

if($exit)
{
print "\n$inx is exiting ";
$stats[$inx] = 1;
return 0;
}

}
if($err == 100)
{
print "\nToo many errors in $inx, killing everyone\n";
$exit = 1;
}

$stats[$inx] = 1;
return $err;
}
my $errls = 0;

REDO:
my $req = HTTP::Request->new('HEAD',$url);
print $req->as_string;
my $ua = LWP::UserAgent->new;

my $res = $ua->request($req);

print $res->as_string;
if($res->code != 200)
{
print "\n\n";
goto REDO if($errls++ < 10);
}


my $length = $res->header('Content-Length');
$tgl = $length;
my $req = HTTP::Request->new('HEAD',$url);
$req->header('Range' => 'bytes=0-99');
my $res = $ua->request($req);
print $res->as_string;
if($res->code != 206)
{
print "Parts cant be used!\n:(\n";
goto NEXT;
}

if(!defined $length)
{
$length=-1;
print "Missing Length
Any Idea? : ";
$length = <>;
print "\n";
}
my $blocks = 8;
my $blocksize = ($length + $blocks - $length % $blocks) / $blocks;
if($blocksize < 1024 * 512)
{
$blocks = 4;
$blocksize = ($length + $blocks - $length % $blocks) / $blocks;
}
#print $blocksize," ",$blocksize*$blocks," ",$length;

my $blockstart = 0;

my @thread = ();

print "\nthreading Gatherers ...\n";

my ($blockend);
for(my $i=1;$i<=$blocks;$i++)
{
$blockend = $blockstart + $blocksize;
$blockend = $length if($blockend > $length);
$thread[$i] = threads->new(sub{dloader($i,$url,$folder."$file.$i.part",$blockstart,$blockend)});
$blockstart = $blockend + 1;
}


my $flag = 0;

do
{
$flag = 0;
for(my $i=1;$i<=$blocks;$i++)
{
$flag = 1 if($stats[$i] == 0);
}
sleep 1;
}
while($flag);

if($exit)
{
open STAT,">>$folder"."$file.stat";
print STAT "".localtime;
print STAT " : Downloaded $tdl / $tgl bytes ".($tdl*100/$tgl)."%\n";
close STAT;
exit;
}

print "\n\ngathering Gatherers...\n";


print "\nMerging ... ";

open OUT,">$file";
binmode OUT;
for(my $i=1;$i<=$blocks;$i++)
{
print "\n+ ".$folder."$file.$i.part";
open IN, $folder."$file.$i.part";
binmode IN;
my $buffer;
print OUT $buffer while(read(IN,$buffer,65536));
close IN;

unlink $folder."$file.$i.part";
}
close OUT;
print "\nDone :)\n";
open STAT,">>$folder$file.stat";
print STAT "".localtime;
print STAT " : Finished Downloading $tdl / $tgl bytes \n";
close STAT;
}

close HND;