[Saga-devel] Re: Tests for Saga-Cloud paper

Wed Jan 14 15:07:40 CST 2009

Hey Andre,

On Wed, 14 Jan 2009 09:07:28 +0100
Andre Merzky <andremerzky at gmail.com> wrote:

> Quoting [Katerina Stamou] (Jan 14 2009):
> > 
> > Dear Shantenu, Andre,
> > 
> > This is an update (yeah almost 24 hrs before the paper submission) of
> > the progress regarding the experiments we intended to conduct with
> > Saga-MapReduce in GumboCloud and LONI clusters.
> > 
> > In a quick chat with Shantenu yesterday afternoon, I've explained the
> > necessity of turning from GumboCloud to Amazon EC2 in order to conduct
> > the experiments, mainly due to the fact that the current installation
> > and configuration of Eucaliptus in GumboCloud (and although Andre
> > tried to fix things even at this very last moment) hasn't and still
> > doesn't allow to actually utilize it even to run a simple test. The
> > causes are out of my concern, and I think it was really useful that at
> > least both Andre and me investigated and possibly helped to improve the
> > current status of this service. Since, yesterday morning, I've been
> > trying to set up an image for the Amazon EC2 in order to conduct the
> > tests there. This resulted to be a more difficult effort than initially
> > thought, mainly due to the fact that I had to setup everything from
> > scratch, and overcome various installation hurdles with mismatching
> > software versions and the like.
> 
> Katarina,
> 
> I created an Amazon image with the id ami-8bb057e2, which
> should be publicly visible.  That is similar to the one
> which is listed in the .saga.ini which I sent you, and used
> by the SAGA aws adaptor.  That image has boost, SAGA, sshsf
> etc installed (in /usr/local/packages/saga).
>
> You should be able to use that image - so simply creating an
> instance with that ami id should work.  Please let me know
> if not.  (Note that I am updating that image now and then -
> mostly to improve default configuration etc.)
> 

I'm unable to find your AMI, (doing ec2-describe-images -a | grep -i ami-8bb057e2).
Perhaps we're using a different availability zone, and images are not global ?
My $EC2_URL points to http://us-east-1.ec2.amazonaws.com, what are you using ?

> 
> > I'm currently at the point where everything should be set for running,
> > but for some reason when the master worker launches the agents on the
> > ec2 instances, it just waits there forever with no output. From the
> > other side of things (inside the vm), the worker appears to be having
> > some trouble. Looking more into it right now, and will send out a mail
> > later or tomorrow morning for the debugging details.
> 
> Yes, please.  thanks for checking that out!
> 

So, the situation is the following, I have the mapreduce config xml set for
EC2, and using postgres as the advert backend (set to fortytwo.cct.lsu.edu).
The worker/agent running in the instance dies with the following error:

INFO       : engine.cpp                : loading dynamic adaptors
Created exception: SAGA(NoAdaptor): adaptor_selector.cpp(184): Could not select any matching adaptor for: advert_directory_cpi::__init__
Created exception: SAGA(NoAdaptor): proxy.cpp(227): No adaptor succeeded in executing constructor for advert_directory_cpi
Created exception: SAGA(NoAdaptor): proxy.cpp(227): No adaptor succeeded in executing constructor for advert_directory_cpi
Created exception: SAGA(NoAdaptor): proxy.cpp(227): No adaptor succeeded in executing constructor for advert_directory_cpi
FAILED (SAGA(NoAdaptor): proxy.cpp(227): No adaptor succeeded in executing constructor for advert_directory_cpi)
MapReduceBase::init : Exception caught : SAGA(NoAdaptor): proxy.cpp(227): No adaptor succeeded in executing constructor for advert_directory_cpi

As a side note, I had troubles getting SAGA detecting my postgres, I'm running
8.0.15, and digging into your config scripts it appears that 8.0.X doesn't define
PG_VERSION_NUM on the pg_config.h. The same is NOT true for postgres 8.2.X.

Best,
Katerina

> 
> > As I had stated from the very beginning of this effort last week, I
> > couldn't promise for the accomplishment of our goal, since I am really
> > in Cloud Computing, as well as the relevant Saga implementations
> > (Saga-MapReduce, relevant adaptors, etc) are really new and of course
> > not well tested also.
> 
> Understood! :-)
> 
> 
> > To sum up, I would like to first of all say that I'm really sorry for
> > not being able to sufficiently contribute in the current effort. I
> > really wish I had more time, and I know that I will have a lot of time
> > in the future not only to conduct such trivial tests but also offer to
> > the Saga-Cloud and MapReduce efforts regardless of this paper.
> > Although, I ll keep continue setting up this image in EC2, it would be
> > mythical to say that I'll manage in time to run the experiments and
> > comment on them in the paper (as human-weak being I also have to sleep,
> > and go to school ;-) ), and I don't know if it would be valuable for
> > further efforts to report what I've done so far.
> 
> Everything helps, and is appreciated!  
> 
> 
> > It will be a quite tough day for me today to schedule a possible
> > telecon through Skype, as I have to attend several classes and decide
> > eventually which one to participate, but I'll be online on aim and I'll
> > keep reading emails.
> > 
> > Shantenu, Andre -again my apologies for missing the our final goal, I
> > really did the best I could in such little amount of time.
> 
> Well, I have very bad news for you: Shantenu managed to get
> us an extension until Sunday night...  So, not all is lost.
> 
> I agree with your assessement of GumboCloud: unless there is
> significant improvement (IP numbers assigned, some idea why
> the new image won't run, etc), we should not put more effort
> in it - it has eaten up too much time already.
> 
> Best, Andre.
> 
> > 
> > Best,
> > Katerina
> > 
> 
> 
> 
> -- 
> Nothing is ever easy.
> 

-- 
Katerina Stamou <kstamou at cct.lsu.edu>