Matt Feenstra is an Architect and Developer, living and working in Palm Desert, CA
Tuesday, April 23, 2013
getzippy.pl usage example
#!/usr/bin/perl -w
use strict;
system("rm -rf ./index.html*");
system("wget http://muzeno.pl");
open(IN,"<index.html");
my @zippyurls;
foreach(<IN>) {
if($_ =~ /a\shref=\"(.+file\.html)?\"\s/) {
push(@zippyurls, $1);
}
}
close(IN);
foreach(@zippyurls) {
system("./getzippy.pl $_");
}
system("rm -rf ./index.html*");
z1ppy sh4re automatic downloader (getzippy.pl)
Disclaimer: Code is for educational use only.
command line syntax: getzippy.pl http://www13.zippyshare.com/v/83632345/file.html
#!/usr/bin/perl -w
use strict;
my $useragent = "Mozilla/5.0";
if(!defined($ARGV[0])) { die "no args. requires url\n"; }
system("mkdir files 2>/dev/null");
my $zippyurl = $ARGV[0];
chomp($zippyurl);
my $magic_b;
my $url;
system("wget -O zippy.page --keep-session-cookies --save-cookies zippy.cookies \"$zippyurl\"");
print "\n\n";
my @filepage = `cat zippy.page`;
foreach(@filepage) {
# looks like
# document.getElementById('dlbutton').href = "/d/83632345/"+(a/a+b+b%10)+"/Deadmau5%20-%20Ghost%20N%20Stuff%20%28Original%20Mix%29.mp3";
my($prefix, $postfix, $zippyserv);
if($zippyurl =~ /(http:\/\/ww.+?\.zippyshare\.com)/) {
$zippyserv = $1;
}
if($_ =~ /var\sb\s=\s(.+)?\;/) {
$magic_b = $1 * 1;
print "magic b =>\t$magic_b\n";
}
#if(!defined($magic_b)) { print "ERROR: couldn't find magic var numbers\n"; }
if($_ =~ /.+?dlbutton.+?\s=\s\"(.+?)\"(\+.+?)\"(.+?)\"/) {
$prefix = $1;
my $magicnumformula = $2;
$postfix = $3;
print "prefix ->\t$prefix\n";
print "formula ->\t$magicnumformula\n";
print "postfix ->\t$postfix\n\n";
my $temp1 = 1 + $magic_b;
my $temp2 = $magic_b % 10;
my $magicmath = $temp1 + $temp2;
$url = "$zippyserv" . "$prefix" . "$magicmath" . "$postfix";
print "\nurl ->\t$url\ngetting file..\n\n";
system("mkdir ./files 2>>/dev/null");
system("wget --directory-prefix=./files --load-cookies ./zippy.cookies --user-agent \"$useragent\" $url");
print "\n";
}
}
# cleanup temp files
system("rm zippy.cookies");
system("rm zippy.page");
command line syntax: getzippy.pl http://www13.zippyshare.com/v/83632345/file.html
#!/usr/bin/perl -w
use strict;
my $useragent = "Mozilla/5.0";
if(!defined($ARGV[0])) { die "no args. requires url\n"; }
system("mkdir files 2>/dev/null");
my $zippyurl = $ARGV[0];
chomp($zippyurl);
my $magic_b;
my $url;
system("wget -O zippy.page --keep-session-cookies --save-cookies zippy.cookies \"$zippyurl\"");
print "\n\n";
my @filepage = `cat zippy.page`;
foreach(@filepage) {
# looks like
# document.getElementById('dlbutton').href = "/d/83632345/"+(a/a+b+b%10)+"/Deadmau5%20-%20Ghost%20N%20Stuff%20%28Original%20Mix%29.mp3";
my($prefix, $postfix, $zippyserv);
if($zippyurl =~ /(http:\/\/ww.+?\.zippyshare\.com)/) {
$zippyserv = $1;
}
if($_ =~ /var\sb\s=\s(.+)?\;/) {
$magic_b = $1 * 1;
print "magic b =>\t$magic_b\n";
}
#if(!defined($magic_b)) { print "ERROR: couldn't find magic var numbers\n"; }
if($_ =~ /.+?dlbutton.+?\s=\s\"(.+?)\"(\+.+?)\"(.+?)\"/) {
$prefix = $1;
my $magicnumformula = $2;
$postfix = $3;
print "prefix ->\t$prefix\n";
print "formula ->\t$magicnumformula\n";
print "postfix ->\t$postfix\n\n";
my $temp1 = 1 + $magic_b;
my $temp2 = $magic_b % 10;
my $magicmath = $temp1 + $temp2;
$url = "$zippyserv" . "$prefix" . "$magicmath" . "$postfix";
print "\nurl ->\t$url\ngetting file..\n\n";
system("mkdir ./files 2>>/dev/null");
system("wget --directory-prefix=./files --load-cookies ./zippy.cookies --user-agent \"$useragent\" $url");
print "\n";
}
}
# cleanup temp files
system("rm zippy.cookies");
system("rm zippy.page");
Monday, April 15, 2013
Installing Chef Server on CentOS 5 / 6
Requires bootstrap method, starting with chef-solo /
chef-client
First steps, system prep
vi /etc/sysconfig/network
vi /etc/selinux/config
chkconfig iptables off
/etc/init.d/iptables stop
(reboot)
vi /etc/hosts – add ip
yum update
rpm -Uvh http://rbel.frameos.org/rbel5
wget -O /etc/yum.repos.d/aegisco.repo http://rpm.aegisco.com/aegisco/el5/aegisco.repo
rpm -Uvh http://dl.fedoraproject.org/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
or
rpm -Uvh http://rbel.frameos.org/rbel6
rpm -Uvh
http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
yum install java-1.6.0-openjdk gecode-devel telnet ruby
rubygems ruby-devel ruby-ri ruby-rdoc ruby-shadow gcc gcc-c++ automake autoconf
make curl dmidecode
curl -O
http://production.cf.rubygems.org/rubygems/rubygems-1.8.10.tgz
tar -zxvf rubygems-1.8.10.tgz
ruby setup.rb --no-format-executable
Second steps, install chef client and chef solo
gem install chef –no-ri –no-rdoc
Create chef-solo (bootstrap)
mkdir /etc/chef
vi /etc/chef/solo.rb
file_cache_path "/tmp/chef-solo"
cookbook_path "/tmp/chef-solo/cookbooks"
vi /etc/chef/chef.json
{
"chef_server": {
"server_url": "http://localhost:4000",
"webui_enabled": true
},
"run_list": [
"recipe[chef-server::rubygems-install]" ]
}
If you find the error message
/usr/lib/ruby/1.8/openssl/cipher.rb:22: Cipher is not a module (TypeError)
then download the ruby source files (for your version!) from e.g.,
ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.7-p352.tar.bz2,
untar it and change dir to ruby-XXXXX/ext/openssl. There, run
ruby extconf.rb,
make,
and make install.
Afterwards, the bootstrap should overcome this error.
Running into library dependency errors on running chef-solo for CentOS 5 ...
yum install openssl-devel
yum install glibc glibc-devel glibc-headers
export LD_LIBRARY_PATH=/usr/local/lib
(configure, make, make install - in order)
ftp://gcc.gnu.org/pub/gcc/infrastructure/gmp-4.3.2.tar.bz2
ftp://gcc.gnu.org/pub/gcc/infrastructure/mpfr-2.4.2.tar.bz2
ftp://gcc.gnu.org/pub/gcc/infrastructure/mpc-0.8.1.tar.gz
ftp://gcc.gnu.org/pub/gcc/releases/gcc-4.7.2/gcc-4.7.2.tar.gz
yum install openssl-devel
yum install glibc glibc-devel glibc-headers
#yum install gcc44
#yum install gcc44-c++
#yum install gcc44-c++
export LD_LIBRARY_PATH=/usr/local/lib
(configure, make, make install - in order)
ftp://gcc.gnu.org/pub/gcc/infrastructure/gmp-4.3.2.tar.bz2
ftp://gcc.gnu.org/pub/gcc/infrastructure/mpfr-2.4.2.tar.bz2
ftp://gcc.gnu.org/pub/gcc/infrastructure/mpc-0.8.1.tar.gz
ftp://gcc.gnu.org/pub/gcc/releases/gcc-4.7.2/gcc-4.7.2.tar.gz
chef-solo -c /etc/chef/solo.rb -j /etc/chef/chef.json -r http://s3.amazonaws.com/chef-solo/bootstrap-latest.tar.gz
Known CentOS error ??
Recipe: gecode::source
* execute[ldconfig] action run
- execute ldconfig
[2013-03-09T14:01:15-08:00] ERROR: Running exception handlers
[2013-03-09T14:01:15-08:00] ERROR: Exception handlers complete
Chef Client failed. 6 resources updated
[2013-03-09T14:01:15-08:00] FATAL: Stacktrace dumped to /tmp/chef-solo/chef-stacktrace.out
[2013-03-09T14:01:15-08:00] FATAL: NoMethodError: gem_package[chef-server-api] (chef-server::rubygems-install line 83) had an error: NoMethodError: undefined method `full_name' for nil:NilClass
Try -
gem install chef-server
Throws -
(::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::)
Fetching: merb-assets-1.1.3.gem (100%)
Fetching: merb-helpers-1.1.3.gem (100%)
Fetching: merb-param-protection-1.1.3.gem (100%)
Fetching: dep_selector-0.0.8.gem (100%)
Building native extensions. This could take a while...
ERROR: Error installing chef-server:
ERROR: Failed to build gem native extension.
/usr/bin/ruby extconf.rb
checking for main() in -lgecodesearch... no
=========================================================================================
Gecode >3.5 must be installed (http://www.gecode.org/).
OSX:
brew install gecode
For convenience, we have built Gecode for Debian/Ubuntu (<release> is lucid or maverick):
Add the following two lines to /etc/apt/sources.list.d/opscode.list:
deb http://apt.opscode.com <release> main
deb-src http://apt.opscode.com <release> main
Then run:
curl http://apt.opscode.com/packages@opscode.com.gpg.key | sudo apt-key add -
sudo apt-get update
sudo apt-get install libgecode-dev
Other distributions can build from source.
Try installing from source -
http://www.gecode.org/download/gecode-3.7.3.tar.gz
(configure, make, make install)
Try -
gem install chef-server
Throws -
needs gecode > 3.5
Try -
chef-solo -c /etc/chef/solo.rb -j /etc/chef/chef.json -r http://s3.amazonaws.com/chef-solo/bootstrap-latest.tar.gz
same
Try -
yum erase gecode gecode-devel
cd gecode-3.7.3
make install
gem install chef-server
Known CentOS error ??
Recipe: gecode::source
* execute[ldconfig] action run
- execute ldconfig
[2013-03-09T14:01:15-08:00] ERROR: Running exception handlers
[2013-03-09T14:01:15-08:00] ERROR: Exception handlers complete
Chef Client failed. 6 resources updated
[2013-03-09T14:01:15-08:00] FATAL: Stacktrace dumped to /tmp/chef-solo/chef-stacktrace.out
[2013-03-09T14:01:15-08:00] FATAL: NoMethodError: gem_package[chef-server-api] (chef-server::rubygems-install line 83) had an error: NoMethodError: undefined method `full_name' for nil:NilClass
Try -
gem install chef-server
Throws -
(::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::) (::)
Fetching: merb-assets-1.1.3.gem (100%)
Fetching: merb-helpers-1.1.3.gem (100%)
Fetching: merb-param-protection-1.1.3.gem (100%)
Fetching: dep_selector-0.0.8.gem (100%)
Building native extensions. This could take a while...
ERROR: Error installing chef-server:
ERROR: Failed to build gem native extension.
/usr/bin/ruby extconf.rb
checking for main() in -lgecodesearch... no
=========================================================================================
Gecode >3.5 must be installed (http://www.gecode.org/).
OSX:
brew install gecode
For convenience, we have built Gecode for Debian/Ubuntu (<release> is lucid or maverick):
Add the following two lines to /etc/apt/sources.list.d/opscode.list:
deb http://apt.opscode.com <release> main
deb-src http://apt.opscode.com <release> main
Then run:
curl http://apt.opscode.com/packages@opscode.com.gpg.key | sudo apt-key add -
sudo apt-get update
sudo apt-get install libgecode-dev
Other distributions can build from source.
Try -
yum install gecode
yum install gecode-devel
Throws -
Package gecode-3.5.0-1.el5.x86_64 already installed and latest version
Package gecode-devel-3.5.0-1.el5.x86_64 already installed and latest version
Try installing from source -
http://www.gecode.org/download/gecode-3.7.3.tar.gz
(configure, make, make install)
Try -
gem install chef-server
Throws -
needs gecode > 3.5
Try -
chef-solo -c /etc/chef/solo.rb -j /etc/chef/chef.json -r http://s3.amazonaws.com/chef-solo/bootstrap-latest.tar.gz
same
Try -
yum erase gecode gecode-devel
cd gecode-3.7.3
make install
gem install chef-server
Verify components installed and running
Name
|
Listen Port
|
Example Program Name in
ps (Erlang programs truncated) |
---|---|---|
Chef Server
|
4000
| merb : chef-server (api) : worker (port 4000) |
Chef Server WebUI
|
4040
| merb : chef-server-webui : worker (port 4040) |
CouchDB
|
5984
| beam.smp -Bd -K true – -root /usr/local/lib/erlang -progname erl – -noshell -noinput -couch_ini /usr/local/etc/couchdb/default.ini /usr/local/etc/couchdb/local.ini -s couch |
RabbitMQ
|
5672
|
{{beam.smp -W w -K true -A30 – -root /usr/local/lib/erlang -progname erl – -noshell -noinput -s rabbit -sname
rabbit -rabbit tcp_listeners [{"0.0.0.0", 5672}]}} |
Chef Solr
|
8983
| /usr/bin/java -Xmx250M -Xms250M -Dsolr.data.dir=/opscode/chef/features/data/solr/data -Dsolr.solr.home=/opscode/chef/features/data/solr/home -jar /opscode/chef/features/data/solr/jetty/start.jar |
Chef Expander
|
none
| ruby ./chef-solr/bin/chef-expander -c /etc/chef/solr.rb -l debug |
Subscribe to:
Posts (Atom)