[Bigjob-users] Attempt to run example_manyjob_affinity.py

[Andre Luckow]

Hi Paula,
there is a small bug in the example:

Please change numer_nodes to number_of_processes in line 51:

resource_list.append( {"resource_url" : "fork://localhost/",
"number_of_processes" : "2", "allocation" : "myAllocation", ...

Also, the BJ part of CSA is a bit old and contains some bugs. If
possible, try to install BJ in userspace (as outlined on the Wiki
page).

It's fixed in BigJob-0.3.31 and SVN.

Best,
Andre


On Thu, Dec 8, 2011 at 9:42 PM, Paula Sanematsu <psanem1 at tigers.lsu.edu> wrote:
> Hi,
>
> I'm trying to run example_manyjob_affinity.py on Sierra, but it doesn't
> complete (see my output below). I'm submitting the job from Sierra and would
> like to use Hotel as my second machine. Could you please advise me on how to
> proceed?
>
> In addition, is there anything wrong with the cct advert service? I could
> run example_local_single.py, but now it's not working.
>
> Thanks,
>
> Paula
>
> ManyJob load test with 8 jobs.
> Create manyjob service
> DEBUG:root:start bigjob at: fork://localhost/
> DEBUG:root:init BigJob w/: advert://advert.cct.lsu.edu:8080
> DEBUG:root:['/N/u/paulasoo/HW06_E3/examples/manySJ_2BJ_diff/../',
> '/N/u/paulasoo/HW06_E3/examples/manySJ_2BJ_diff',
> '/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg',
> '/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/BigJob-0.3.2-py2.7.egg',
> '/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/redis-2.2.4-py2.7.egg',
> '/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/virtualenv-1.6.4-py2.7.egg',
> '/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/threadpool-1.2.7-py2.7.egg',
> '/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/uuid-1.30-py2.7.egg',
> '/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg',
> '/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/redis-2.2.4-py2.7.egg',
> '/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/threadpool-1.2.7-py2.7.egg',
> '/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/uuid-1.30-py2.7.egg',
> '/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages',
> '/N/soft/SAGA/external/python/2.7.1/gcc-4.1.2/lib/2.7/site-packages',
> '/N/u/paulasoo/HW06_E3/examples/manySJ_2BJ_diff',
> '/N/soft/SAGA/external/python/2.7.1/gcc-4.1.2/lib/python27.zip',
> '/N/soft/SAGA/external/python/2.7.1/gcc-4.1.2/lib/python2.7',
> '/N/soft/SAGA/external/python/2.7.1/gcc-4.1.2/lib/python2.7/plat-linux2',
> '/N/soft/SAGA/external/python/2.7.1/gcc-4.1.2/lib/python2.7/lib-tk',
> '/N/soft/SAGA/external/python/2.7.1/gcc-4.1.2/lib/python2.7/lib-old',
> '/N/soft/SAGA/external/python/2.7.1/gcc-4.1.2/lib/python2.7/lib-dynload',
> '/N/soft/SAGA/external/python/2.7.1/gcc-4.1.2/lib/python2.7/site-packages',
> '/N/soft/SAGA/external/python/2.7.1/gcc-4.1.2/lib/2.7/site-packages',
> '/N/soft/SAGA/external/python/2.7.1/gcc-4.1.2/lib/2.7/site-packages',
> '/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/BigJob-0.3.2-py2.7.egg/bigjob',
> '/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/BigJob-0.3.2-py2.7.egg/bigjob_dynamic',
> '/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/BigJob-0.3.2-py2.7.egg/bigjob_dynamic']
> DEBUG:root:Utilizing ADVERT Backend
> DEBUG:root:Parsing URL: advert://advert.cct.lsu.edu:8080
> DEBUG:root:Server: advert.cct.lsu.edu Port 8080 server_connect_url: None
> DEBUG:root:initialized BigJob: bigjob:6638ce9e-21d6-11e1-ac76-002215124496
> Traceback (most recent call last):
>   File "example_manyjob_affinity.py", line 61, in <module>
>     mjs = many_job_affinity_service(resource_list, COORDINATION_URL)
>   File
> "/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/BigJob-0.3.2-py2.7.egg/bigjob_dynamic/many_job_affinity.py",
> line 19, in __init__
>     super(many_job_affinity_service, self).__init__(bigjob_list,
> advert_host)
>   File
> "/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/BigJob-0.3.2-py2.7.egg/bigjob_dynamic/many_job.py",
> line 59, in __init__
>     self.__init_bigjobs()
>   File
> "/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/BigJob-0.3.2-py2.7.egg/bigjob_dynamic/many_job.py",
> line 74, in __init_bigjobs
>     self.__start_bigjob(i)
>   File
> "/N/soft/SAGA/saga/1.6/gcc-4.1.2/lib/python2.7/site-packages/BigJob-0.3.2-py2.7.egg/bigjob_dynamic/many_job.py",
> line 98, in __start_bigjob
>     bj_dict["number_of_processes"],
> KeyError: 'number_of_processes'
> Cancel Pilot Job
> stop pilot job:
> DEBUG:root:create advert entry: advert://advert.cct.lsu.edu:8080/
> DEBUG:root:update state of pilot job to: Done Stopped: True
> DEBUG:root:delete pilot job:
>
> _______________________________________________
> Bigjob-users mailing list
> Bigjob-users at mail.cct.lsu.edu
> https://mail.cct.lsu.edu/mailman/listinfo/bigjob-users
>