[Bigjob-users] bigjob on teragrid/xsede

Wed Jan 4 15:47:41 CST 2012

Actually Andre this works. You just have to use a 3rd freaking python:

which python

/scratch/projects/tg/SAGA/external/python/2.7.1/gcc-4.4.5/bin/python

 python

Python 2.7.1 (r271:86832, Nov 25 2011, 04:51:00)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from saga import *
>>>

That's how I got it to work. Ok we need to clean this up. Easiest way to do
it would be for Andre M/L to install BigJob in the SAGA python with
easy_install and keep it up to date. We also need to make this uniform
across all machines.

Regards
Yaakoub El Khamra

On Wed, Jan 4, 2012 at 3:42 PM, Yaakoub El Khamra <yelkhamra at gmail.com>wrote:

>
> Andre M, your thoughts?
>
> This was working late nov for me. What happened?
>
> Regards
> Yaakoub El Khamra
>
>
>
>
> On Wed, Jan 4, 2012 at 3:38 PM, Andre Luckow <aluckow at cct.lsu.edu> wrote:
>
>> Hi Yaakoub,
>> we have SAGA and Python in CSA space that works fine with BJ:
>>
>>
>> https://svn.cct.lsu.edu/repos/saga-projects/deployment/tg-csa/doc/README.saga-1.6.gcc-4.4.5.ranger
>>
>> Unfortunately, it's broken :-( - will open a ticket:
>>
>> tg804093 at login3 ~$ python
>> Python 2.7.1 (r271:86832, Nov 25 2011, 04:51:00)
>> [GCC 4.4.5] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> import saga
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>  File
>> "/scratch/projects/tg/SAGA/saga/1.6/gcc-4.4.5/lib/python2.7/site-packages/saga/__init__.py",
>> line 28, in <module>
>>    from _engine import *         # import all classes from SAGA.engine
>> ImportError: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.9' not
>> found (required by
>>
>> /scratch/projects/tg/SAGA/saga/1.6/gcc-4.4.5/lib/libsaga_package_advert.so.1.6)
>>
>> Best,
>> Andre
>>
>> On Wed, Jan 4, 2012 at 10:28 PM, Yaakoub El Khamra <yelkhamra at gmail.com>
>> wrote:
>> >
>> > python bigjob-bootstrap.py $WORK/ECSS/bigjob_install
>> >
>> >      Traceback (most recent call last):
>> >   File "bigjob-bootstrap.py", line 23, in ?
>> >     import subprocess
>> > ImportError: No module named subprocess
>> >
>> > SAGA on ranger was compiled with python 2.3.4, the system python. Andre
>> M,
>> > j'accuse!
>> >
>> > You have to load the python module to get bigjob. This can be better
>> > coordinated IMHO.
>> >
>> > Regards
>> > Yaakoub El Khamra
>> >
>> >
>> >
>> >
>> > On Wed, Jan 4, 2012 at 3:22 PM, Andre Luckow <aluckow at cct.lsu.edu>
>> wrote:
>> >>
>> >> Hi Yaakoub,
>> >> the bootstrap script in variant c) installs virtualenv automatically
>> >> for the user. Option d) should work. We just migrated to Github
>> >> yesterday - so there might be some issues. Not sure why your git
>> >> version doesn't accept the Github certificate. But, you can also clone
>> >> the repository via HTTP.
>> >>
>> >> git clone http://github.com/drelu/BigJob.git
>> >>
>> >> or the native git protocol
>> >>
>> >> git clone git://github.com/drelu/BigJob.git
>> >>
>> >> Or alternatively, the good old SVN.
>> >>
>> >> I think we have many options to install BJ (maybe to many for a normal
>> >> user). I personally prefer the virtualenv alternative since this is a
>> >> clean way to manage all your Python libraries in user space.
>> >>
>> >> Best,
>> >> Andre
>> >>
>> >>
>> >> On Wed, Jan 4, 2012 at 10:02 PM, Yaakoub El Khamra <
>> yelkhamra at gmail.com>
>> >> wrote:
>> >> >
>> >> > Also can I point out something I should have pointed out a while ago:
>> >> > users have to install virtualenv on almost all supercomputers. At
>> least
>> >> > the
>> >> > top 5 on XSEDE, all the ones we have allocations on.
>> >> >
>> >> > This means, users have to install virtualenv to install bigjob.
>> That's
>> >> > not
>> >> > practical. d) is what most users will want to use since they already
>> >> > have to
>> >> > add to the pythonpath from the saga installation.
>> >> >
>> >> > Regards
>> >> > Yaakoub El Khamra
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On Wed, Jan 4, 2012 at 2:39 PM, Yaakoub El Khamra <
>> yelkhamra at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Cloning into BigJob...
>> >> >> error: SSL certificate problem, verify that the CA cert is OK.
>> Details:
>> >> >> error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate
>> >> >> verify
>> >> >> failed while accessing
>> https://github.com/drelu/BigJob.git/info/refs
>> >> >>
>> >> >> fatal: HTTP request failed
>> >> >>
>> >> >> Regards
>> >> >> Yaakoub El Khamra
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Wed, Jan 4, 2012 at 2:33 PM, Andre Luckow <aluckow at cct.lsu.edu>
>> >> >> wrote:
>> >> >>>
>> >> >>> Hi Yaakoub,
>> >> >>> with what error does d) fail for you?
>> >> >>>
>> >> >>> Best,
>> >> >>> Andre
>> >> >>>
>> >> >>> On Wed, Jan 4, 2012 at 9:18 PM, Yaakoub El Khamra
>> >> >>> <yelkhamra at gmail.com>
>> >> >>> wrote:
>> >> >>> >
>> >> >>> > FYI d installation method fails. Sorry I should have told you
>> guys
>> >> >>> > about
>> >> >>> > this a very long time ago when I first discovered it.
>> >> >>> >
>> >> >>> > Regards
>> >> >>> > Yaakoub El Khamra
>> >> >>> >
>> >> >>> >
>> >> >>> >
>> >> >>> >
>> >> >>> > On Wed, Jan 4, 2012 at 3:03 AM, Andre Luckow <
>> aluckow at cct.lsu.edu>
>> >> >>> > wrote:
>> >> >>> >>
>> >> >>> >> Hi Melissa,
>> >> >>> >> the recommended way is to install BJ in user space. The CSA BJ
>> >> >>> >> version
>> >> >>> >> is a bit outdated and is already known to have a couple of bugs:
>> >> >>> >>
>> >> >>> >> http://faust.cct.lsu.edu/trac/bigjob/wiki/install
>> >> >>> >>
>> >> >>> >> Installation type b or c should be the best for you.
>> >> >>> >>
>> >> >>> >> Once you have setup your virtualenv, you can easily update to a
>> new
>> >> >>> >> version of BJ with:
>> >> >>> >>
>> >> >>> >> easy_install -U bigjob
>> >> >>> >>
>> >> >>> >> Since, I expect that we will discover some bugs on the way to
>> >> >>> >> XSEDE,
>> >> >>> >> this will be an important feature.
>> >> >>> >>
>> >> >>> >> Best,
>> >> >>> >> Andre
>> >> >>> >>
>> >> >>> >> On Wed, Jan 4, 2012 at 4:24 AM, Melissa Romanus
>> >> >>> >> <melissa.romanus at rutgers.edu> wrote:
>> >> >>> >> > Hi all-
>> >> >>> >> >
>> >> >>> >> > I need to start either compiling information or playing around
>> >> >>> >> > with
>> >> >>> >> > bigjob
>> >> >>> >> > on xsede/teragrid, specifically kraken, ranger, and possibly
>> >> >>> >> > lonestar.
>> >> >>> >> > Last
>> >> >>> >> > semester, I was only working on the futuregrid. What
>> >> >>> >> > documentation
>> >> >>> >> > currently
>> >> >>> >> > exists for getting bj running on the XSEDE machines? Also, I
>> am
>> >> >>> >> > sorry I
>> >> >>> >> > have
>> >> >>> >> > not been super up on email traffic, but what is the general
>> >> >>> >> > consensus of
>> >> >>> >> > 'how to use bj now' (in other words, install into user space
>> vs
>> >> >>> >> > CSA)? I
>> >> >>> >> > am
>> >> >>> >> > asking because I will soon be working closely with the
>> chemists
>> >> >>> >> > and
>> >> >>> >> > want
>> >> >>> >> > to
>> >> >>> >> > have an easy/smooth process for them to follow.
>> >> >>> >> >
>> >> >>> >> > Thank you,
>> >> >>> >> > Melissa
>> >> >>> >> >
>> >> >>> >> > _______________________________________________
>> >> >>> >> > Bigjob-users mailing list
>> >> >>> >> > Bigjob-users at mail.cct.lsu.edu
>> >> >>> >> > https://mail.cct.lsu.edu/mailman/listinfo/bigjob-users
>> >> >>> >> >
>> >> >>> >> _______________________________________________
>> >> >>> >> Bigjob-users mailing list
>> >> >>> >> Bigjob-users at mail.cct.lsu.edu
>> >> >>> >> https://mail.cct.lsu.edu/mailman/listinfo/bigjob-users
>> >> >>> >
>> >> >>> >
>> >> >>
>> >> >>
>> >> >
>> >
>> >
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cct.lsu.edu/pipermail/bigjob-users/attachments/20120104/bec1f19f/attachment-0001.html 