|
|
Choosing A Webhost: |
job in R state but not running and can't be deleted from queue: msg#00127clustering.torque.user
Hi, I observed the following behavior today, on an otherwise stable setup based on torque-2.0.0p2 and maui-3.2.6p13 (I know, I have to upgrade). - some jobs being reported by qstat to be in R state with an exec_host defined, were found to have no corresponding processes on the respective nodes - the nodes did have the usual files in the spool/, aux/ and mom_prov/jobs/ subdirectories - a qdel on the jobs, came back with exit-status=0, but the job status did not change - pbs_server logs showed that SIGTERM and later SIGKILL were sent to the job, but the usual final message with the job's exit-status and used resources was found absent - in the mom logs, there were messages for job start, "scan_for_terminated" and "Terminated" for each job. But the job state was still R - other jobs on the same nodes worked fine during this time - restarting mom on the nodes helped - the jobs could then be qdel-ed Has anyone seen this behavior before? Are there any other patterns I should be looking for? Is there a better/cleaner way in such cases to clear the queues?
|
|
| <Prev in Thread] | Current Thread | [Next in Thread> |
|---|---|---|
| Previous by Date: | Re: job in R state but not running and can't be deleted from queue, Garrick Staples |
|---|---|
| Next by Date: | Re: Changing filename, Joel Perras |
| Previous by Thread: | Changing filename, Rafal Masztalerz |
| Next by Thread: | Re: job in R state but not running and can't be deleted from queue, Garrick Staples |
| Indexes: | [Date] [Thread] [Top] [All Lists] |
Free MagazinesCisco NewsReceive a free quarterly e-newsletter with exclusive articles on how Cisco IT uses its own products and solutions to enable the business. subscribe Systems Management News, the newspaper for IT systems administration and data center managers! Each issue of Systems Management News is chock-full of news and analysis to help you understand what's happening in your field. subscribe The Enterprise Newsweekly eWeek is the essential technology information source for builders of e-business. subscribe Oracle Magazine Oracle Magazine contains technology strategy articles, sample code, tips, Oracle and partner news, how to articles for developers and DBAs, and more. Oracle (NASDAQ: ORCL) is the world's largest enterprise software company. subscribe Total Telecom Total Telecom is "The Economist of the communications industry". subscribe |