R is the superglue language

We all know that scripting languages are often used as glueing unix commands together.

But the ultimate glueing language is R. By using R you can glue shared object libraries together in a seamless way that makes perl or python look pale. Here is a set of slides how you can use it together with all the R goodies:

R as the superglue language

Using R you can glue C/C++/ObjectiveC, FORTRAN, opengl, CUDA, web, MPI, threads and you name it... together.

Installing bioperl without root access

In order to give our users the maximum freedom, we are not installing perl modules systemwide, but the modules have to be installed on a per user basis, unless the modules are of such importance and wide use by our user basis, that it justifies a system wide installation. Here is a tutorial how to install perl modules by yourself without having root access:

1) start cpan and config. defaults are ok.
   downloadsites set: 11 9 4
   takes a while -> quit

2) create local perl module directory:
 > mkdir ~/myperl
 
3) start cpan and set the following options:

 > cpan
 cpan> o conf makepl_arg "LIB=~/myperl/lib \
  INSTALLMAN1DIR=~/myperl/man/man1 \
  INSTALLMAN3DIR=~/myperl/man/man3 \
  INSTALLSCRIPT=~/myperl/bin \
  INSTALLBIN=~/myperl/bin"

 cpan> o conf mbuildpl_arg "--lib=~/myperl/lib \
  --installman1dir=~/myperl/man/man1 \
  --installman3dir=~/myperl/man/man3 \
  --installscript=~/myperl/bin \
  --installbin=~/myperl/bin"

 cpan> o conf mbuild_install_arg "--install_path lib=~/myperl"
 cpan> o conf prerequisites_policy automatically
 cpan> o conf commit
 cpan> quit

then you can install whatever modules you like in your local directory.
For example, for bioperl you would need:

 cpan>d /bioperl/
 CPAN: Storable loaded ok
 Going to read /home/bosborne/.cpan/Metadata
 Database was generated on Mon, 20 Nov 2006 05:24:36 GMT
 
 ....
 
 Distribution B/BI/BIRNEY/bioperl-1.2.tar.gz
 Distribution B/BI/BIRNEY/bioperl-1.4.tar.gz
 Distribution C/CJ/CJFIELDS/BioPerl-1.6.0.tar.gz

Now install:

 cpan> force install C/CJ/CJFIELDS/BioPerl-1.6.0.tar.gz

in case the download is slow, then edit the file ~/.cpan/CPAN/MyConfig.pm and insert the following line
into $CPAN::Config :

'dontload_hash' => { "Net::FTP" => 1, "LWP" =>1 },

in case something goes wrong you can delete ~/.cpan and start over again.

very helpful is the perl shell which you can obtain by installing:
cpan> install Psh
cpan> install IO::String

it is then available under ~/myperl/bin/psh
e.g. try out 
psh% use Bio::Perl;
psh% $seq_object = get_sequence('genbank',"ROA1_HUMAN");
psh% write_sequence(">roa1.fasta",'fasta',$seq_object);

HPC and Visualisation

LRZ is giving a course on "Visualisation of Large Data Sets on Supercomputers". Here are some of the slides:

• Hardware @ LRZ
• VAPOR @ LRZ
• R as superglue language

Intel announces accelerator card

Today at 12:00 at the International Supercomputing Conference ISC2010 here in Hamburg, Intel announced a new accelerator card featuring 32 cores and up to 2 GB RAM which can run 128 threads via hyperthreading. Looks like the accelerator wars have begun. They showed a LU decomposition test running on the card and claimed it to be the fastest decomposition at the moment with more than 500 GFlop/s. Pretty impressive. I was attending the keynote from Kirk B. Skaugen
Vice President, Intel Architecture Group & General Manager, Data Center Group, Intel, USA who demoed the accelerator card and presented a complete software stack including C and FORTRAN compilers.

R on the SGI Altix 4700

Here at LRZ we have a large SGI Altix 4700 Supercomputer. Lately we have compiled an MPI-enabled version of R for this beast. The machine sports 9728 cores connected with a ccNUMA link and maxes out at 62 TFlop/s. With R we were able to use a whooping 4000 cores on the machine and still get a decent performance.

MPI programming with R is a breeze. You can use the foreach packages from Revolutions and just transfer your serial code to the supercomputer, replace all %do% with %dopar% and submit your jobs and you are done.

Here is a short tutorial how to start programming on supercomputers with R.