Please take our Survey
logo       

Choosing A Webhost:
A web hosting service is a type of Internet hosting service that allows individuals and organizations to provide their own website accessible via the World Wide Web. Web hosts are companies that provide space on a server they own for use by their clients as well as providing Internet connectivity, typically in a data center. Web hosts can also provide data center space and connectivity to the Internet for servers they do not own to be located in their data center, called colocation. more...

job in R state but not running and can't be deleted from queue: msg#00127

clustering.torque.user

Subject: job in R state but not running and can't be deleted from queue

Hi,

I observed the following behavior today, on an otherwise stable setup based on torque-2.0.0p2 and maui-3.2.6p13 (I know, I have to upgrade).

- some jobs being reported by qstat to be in R state with an exec_host defined, were found to have no corresponding processes on the respective nodes
- the nodes did have the usual files in the spool/, aux/ and mom_prov/jobs/ subdirectories
- a qdel on the jobs, came back with exit-status=0, but the job status did not change
- pbs_server logs showed that SIGTERM and later SIGKILL were sent to the job, but the usual final message with the job's exit-status and used resources was found absent
- in the mom logs, there were messages for job start, "scan_for_terminated" and "Terminated" for each job. But the job state was still R

- other jobs on the same nodes worked fine during this time

- restarting mom on the nodes helped - the jobs could then be qdel-ed

Has anyone seen this behavior before? Are there any other patterns I should be looking for? Is there a better/cleaner way in such cases to clear the queues?


<Prev in Thread] Current Thread [Next in Thread>
Google Custom Search

Recently Viewed:
hardware.arm.at...    cms.citadel.dev...    video.gstreamer...    java.facelets.u...    misc.basics.qna...    web.wiki.instik...    network.uip.use...    xdg.devel/2003-...    tex.bibtex.bibd...    finance.quotesp...    ietf.zeroconf/2...    redhat.blinux.g...    suse.db2/2003-0...    php.phpesp/2004...    uml.devel/2003-...    gnome.labyrinth...    qnx.openqnx.dev...    boot-loaders.gr...    db.dataperfect....    audio.audacity....    linux.uclinux.m...    editors.j.devel...    os.openbsd.tech...    kde.users.multi...   
Home | advertise | OSDir is an inevitable website. super tiny logo

Free Magazines

Cisco News
Receive a free quarterly e-newsletter with exclusive articles on how Cisco IT uses its own products and solutions to enable the business.
subscribe

Systems Management News, the newspaper for IT systems administration and data center managers! Each issue of Systems Management News is chock-full of news and analysis to help you understand what's happening in your field.
subscribe

The Enterprise Newsweekly eWeek is the essential technology information source for builders of e-business.
subscribe

Oracle Magazine Oracle Magazine contains technology strategy articles, sample code, tips, Oracle and partner news, how to articles for developers and DBAs, and more. Oracle (NASDAQ: ORCL) is the world's largest enterprise software company.
subscribe

Total Telecom Total Telecom is "The Economist of the communications industry".
subscribe

Navigation