[Bigjob-users] bigjob on teragrid/xsede

Wed Jan 4 16:27:45 CST 2012

So this is what we are doing.

Andre M you have until 9:30 central tonight to update bigjob installations
(easy_install -U bigjob) in the python installations of both ranger and
lonestar CSA's. If you don't I am going to do it myself from your account.

I fixed a minor permissions issue for the readme.

Regards
Yaakoub El Khamra

On Wed, Jan 4, 2012 at 3:47 PM, Yaakoub El Khamra <yelkhamra at gmail.com>wrote:

>
> Actually Andre this works. You just have to use a 3rd freaking python:
>
> which python
>
>
> /scratch/projects/tg/SAGA/external/python/2.7.1/gcc-4.4.5/bin/python
>
>  python
>
>
> Python 2.7.1 (r271:86832, Nov 25 2011, 04:51:00)
> [GCC 4.4.5] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from saga import *
> >>>
>
> That's how I got it to work. Ok we need to clean this up. Easiest way to
> do it would be for Andre M/L to install BigJob in the SAGA python with
> easy_install and keep it up to date. We also need to make this uniform
> across all machines.
>
>
> Regards
> Yaakoub El Khamra
>
>
>
>
> On Wed, Jan 4, 2012 at 3:42 PM, Yaakoub El Khamra <yelkhamra at gmail.com>wrote:
>
>>
>> Andre M, your thoughts?
>>
>> This was working late nov for me. What happened?
>>
>> Regards
>> Yaakoub El Khamra
>>
>>
>>
>>
>> On Wed, Jan 4, 2012 at 3:38 PM, Andre Luckow <aluckow at cct.lsu.edu> wrote:
>>
>>> Hi Yaakoub,
>>> we have SAGA and Python in CSA space that works fine with BJ:
>>>
>>>
>>> https://svn.cct.lsu.edu/repos/saga-projects/deployment/tg-csa/doc/README.saga-1.6.gcc-4.4.5.ranger
>>>
>>> Unfortunately, it's broken :-( - will open a ticket:
>>>
>>> tg804093 at login3 ~$ python
>>> Python 2.7.1 (r271:86832, Nov 25 2011, 04:51:00)
>>> [GCC 4.4.5] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>> >>> import saga
>>> Traceback (most recent call last):
>>>   File "<stdin>", line 1, in <module>
>>>  File
>>> "/scratch/projects/tg/SAGA/saga/1.6/gcc-4.4.5/lib/python2.7/site-packages/saga/__init__.py",
>>> line 28, in <module>
>>>    from _engine import *         # import all classes from SAGA.engine
>>> ImportError: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.9' not
>>> found (required by
>>>
>>> /scratch/projects/tg/SAGA/saga/1.6/gcc-4.4.5/lib/libsaga_package_advert.so.1.6)
>>>
>>> Best,
>>> Andre
>>>
>>> On Wed, Jan 4, 2012 at 10:28 PM, Yaakoub El Khamra <yelkhamra at gmail.com>
>>> wrote:
>>> >
>>> > python bigjob-bootstrap.py $WORK/ECSS/bigjob_install
>>> >
>>> >      Traceback (most recent call last):
>>> >   File "bigjob-bootstrap.py", line 23, in ?
>>> >     import subprocess
>>> > ImportError: No module named subprocess
>>> >
>>> > SAGA on ranger was compiled with python 2.3.4, the system python.
>>> Andre M,
>>> > j'accuse!
>>> >
>>> > You have to load the python module to get bigjob. This can be better
>>> > coordinated IMHO.
>>> >
>>> > Regards
>>> > Yaakoub El Khamra
>>> >
>>> >
>>> >
>>> >
>>> > On Wed, Jan 4, 2012 at 3:22 PM, Andre Luckow <aluckow at cct.lsu.edu>
>>> wrote:
>>> >>
>>> >> Hi Yaakoub,
>>> >> the bootstrap script in variant c) installs virtualenv automatically
>>> >> for the user. Option d) should work. We just migrated to Github
>>> >> yesterday - so there might be some issues. Not sure why your git
>>> >> version doesn't accept the Github certificate. But, you can also clone
>>> >> the repository via HTTP.
>>> >>
>>> >> git clone http://github.com/drelu/BigJob.git
>>> >>
>>> >> or the native git protocol
>>> >>
>>> >> git clone git://github.com/drelu/BigJob.git
>>> >>
>>> >> Or alternatively, the good old SVN.
>>> >>
>>> >> I think we have many options to install BJ (maybe to many for a normal
>>> >> user). I personally prefer the virtualenv alternative since this is a
>>> >> clean way to manage all your Python libraries in user space.
>>> >>
>>> >> Best,
>>> >> Andre
>>> >>
>>> >>
>>> >> On Wed, Jan 4, 2012 at 10:02 PM, Yaakoub El Khamra <
>>> yelkhamra at gmail.com>
>>> >> wrote:
>>> >> >
>>> >> > Also can I point out something I should have pointed out a while
>>> ago:
>>> >> > users have to install virtualenv on almost all supercomputers. At
>>> least
>>> >> > the
>>> >> > top 5 on XSEDE, all the ones we have allocations on.
>>> >> >
>>> >> > This means, users have to install virtualenv to install bigjob.
>>> That's
>>> >> > not
>>> >> > practical. d) is what most users will want to use since they already
>>> >> > have to
>>> >> > add to the pythonpath from the saga installation.
>>> >> >
>>> >> > Regards
>>> >> > Yaakoub El Khamra
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > On Wed, Jan 4, 2012 at 2:39 PM, Yaakoub El Khamra <
>>> yelkhamra at gmail.com>
>>> >> > wrote:
>>> >> >>
>>> >> >> Cloning into BigJob...
>>> >> >> error: SSL certificate problem, verify that the CA cert is OK.
>>> Details:
>>> >> >> error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate
>>> >> >> verify
>>> >> >> failed while accessing
>>> https://github.com/drelu/BigJob.git/info/refs
>>> >> >>
>>> >> >> fatal: HTTP request failed
>>> >> >>
>>> >> >> Regards
>>> >> >> Yaakoub El Khamra
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> On Wed, Jan 4, 2012 at 2:33 PM, Andre Luckow <aluckow at cct.lsu.edu>
>>> >> >> wrote:
>>> >> >>>
>>> >> >>> Hi Yaakoub,
>>> >> >>> with what error does d) fail for you?
>>> >> >>>
>>> >> >>> Best,
>>> >> >>> Andre
>>> >> >>>
>>> >> >>> On Wed, Jan 4, 2012 at 9:18 PM, Yaakoub El Khamra
>>> >> >>> <yelkhamra at gmail.com>
>>> >> >>> wrote:
>>> >> >>> >
>>> >> >>> > FYI d installation method fails. Sorry I should have told you
>>> guys
>>> >> >>> > about
>>> >> >>> > this a very long time ago when I first discovered it.
>>> >> >>> >
>>> >> >>> > Regards
>>> >> >>> > Yaakoub El Khamra
>>> >> >>> >
>>> >> >>> >
>>> >> >>> >
>>> >> >>> >
>>> >> >>> > On Wed, Jan 4, 2012 at 3:03 AM, Andre Luckow <
>>> aluckow at cct.lsu.edu>
>>> >> >>> > wrote:
>>> >> >>> >>
>>> >> >>> >> Hi Melissa,
>>> >> >>> >> the recommended way is to install BJ in user space. The CSA BJ
>>> >> >>> >> version
>>> >> >>> >> is a bit outdated and is already known to have a couple of
>>> bugs:
>>> >> >>> >>
>>> >> >>> >> http://faust.cct.lsu.edu/trac/bigjob/wiki/install
>>> >> >>> >>
>>> >> >>> >> Installation type b or c should be the best for you.
>>> >> >>> >>
>>> >> >>> >> Once you have setup your virtualenv, you can easily update to
>>> a new
>>> >> >>> >> version of BJ with:
>>> >> >>> >>
>>> >> >>> >> easy_install -U bigjob
>>> >> >>> >>
>>> >> >>> >> Since, I expect that we will discover some bugs on the way to
>>> >> >>> >> XSEDE,
>>> >> >>> >> this will be an important feature.
>>> >> >>> >>
>>> >> >>> >> Best,
>>> >> >>> >> Andre
>>> >> >>> >>
>>> >> >>> >> On Wed, Jan 4, 2012 at 4:24 AM, Melissa Romanus
>>> >> >>> >> <melissa.romanus at rutgers.edu> wrote:
>>> >> >>> >> > Hi all-
>>> >> >>> >> >
>>> >> >>> >> > I need to start either compiling information or playing
>>> around
>>> >> >>> >> > with
>>> >> >>> >> > bigjob
>>> >> >>> >> > on xsede/teragrid, specifically kraken, ranger, and possibly
>>> >> >>> >> > lonestar.
>>> >> >>> >> > Last
>>> >> >>> >> > semester, I was only working on the futuregrid. What
>>> >> >>> >> > documentation
>>> >> >>> >> > currently
>>> >> >>> >> > exists for getting bj running on the XSEDE machines? Also, I
>>> am
>>> >> >>> >> > sorry I
>>> >> >>> >> > have
>>> >> >>> >> > not been super up on email traffic, but what is the general
>>> >> >>> >> > consensus of
>>> >> >>> >> > 'how to use bj now' (in other words, install into user space
>>> vs
>>> >> >>> >> > CSA)? I
>>> >> >>> >> > am
>>> >> >>> >> > asking because I will soon be working closely with the
>>> chemists
>>> >> >>> >> > and
>>> >> >>> >> > want
>>> >> >>> >> > to
>>> >> >>> >> > have an easy/smooth process for them to follow.
>>> >> >>> >> >
>>> >> >>> >> > Thank you,
>>> >> >>> >> > Melissa
>>> >> >>> >> >
>>> >> >>> >> > _______________________________________________
>>> >> >>> >> > Bigjob-users mailing list
>>> >> >>> >> > Bigjob-users at mail.cct.lsu.edu
>>> >> >>> >> > https://mail.cct.lsu.edu/mailman/listinfo/bigjob-users
>>> >> >>> >> >
>>> >> >>> >> _______________________________________________
>>> >> >>> >> Bigjob-users mailing list
>>> >> >>> >> Bigjob-users at mail.cct.lsu.edu
>>> >> >>> >> https://mail.cct.lsu.edu/mailman/listinfo/bigjob-users
>>> >> >>> >
>>> >> >>> >
>>> >> >>
>>> >> >>
>>> >> >
>>> >
>>> >
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cct.lsu.edu/pipermail/bigjob-users/attachments/20120104/91d47fc8/attachment.html 