 Terminate CUDA kernel which got stuck in an endless loop?, Is that possible under linux? |
  |
 Dec 16 2008, 02:28 PM
Post
#1
|
|
 Group: Members Posts: 34 Joined: 31-October 08 Member No.: 124,108  |
Sometimes kernel goes into an endless loop (by a mistake in a kernel of course). And I'm unable to somehow terminate it.
kill -9 of the hosting process fails, process remains (still the process is not zombie, it is marked as just running). Have to reboot. Definitely unusable approach. Driver: 180.60 Linux: Debian/lenny Monitor is attached to another video card, so there is no CUDA timeout. Is there any solution to terminate such process/kernel? This post has been edited by euk: Dec 16 2008, 03:01 PM |
 |
 
|
|
Dec 16 2008, 03:21 PM
Post
#2
|
|
|
Group: Extranet Users Posts: 2,501 Joined: 23-March 07 Member No.: 46,425 Org.: University of Michigan |
Odd, I usually have luck just pressint Ctrl-C. Sometimes it takes ~10 seconds to take effect, but it usually works. The only times a reboot has been necessary for me is with horribly buggy kernels that wrote all over device memory, probably messing up the driver.
|
|
|
|
|
Dec 16 2008, 04:25 PM
Post
#3
|
|
|
Group: Members Posts: 34 Joined: 31-October 08 Member No.: 124,108 |
I have just tested with while(true) {}. No memory use...
Just found that Ctrl-C really helps, but in about 30 minutes  .Maybe there can help an unload of some module (nvidia driver?) Any ideas? This post has been edited by euk: Dec 16 2008, 04:25 PM |
|
|
|
|
Dec 16 2008, 04:41 PM
Post
#4
|
|
 Group: Moderators Posts: 2,685 Joined: 3-June 08 From: Santa Clara, CA Member No.: 106,363 Club SLI Member: No Org.: NVIDIA |
A fix for that is coming, but not until after 2.1 is out.
|
|
|
|
|
Dec 16 2008, 04:55 PM
Post
#5
|
|
|
Group: Members Posts: 34 Joined: 31-October 08 Member No.: 124,108 |
QUOTE (tmurray @ Dec 16 2008, 10:41 PM)  A fix for that is coming, but not until after 2.1 is out. Thank you. But are there any tweaks for the present moment? Such as driver unload or something like that? |
|
|
|
|
Dec 17 2008, 02:05 AM
Post
#6
|
|
|
Group: Members Posts: 990 Joined: 29-August 07 Member No.: 67,724 |
QUOTE (tmurray @ Dec 16 2008, 12:41 PM) A fix for that is coming A fix in what form? Quicker return after Ctrl+C, or some larger-scale solution? Will it work on Windows? -------------------- CUDA consulting/programming available: cuda@almson.net. Pay after results.
|
|
|
|
|
Dec 17 2008, 04:58 AM
Post
#7
|
|
 Group: Members Posts: 1,907 Joined: 23-November 07 From: Bangalore Member No.: 79,873 Org.: HCL Technologies |
QUOTE (euk @ Dec 16 2008, 10:25 PM) Thank you. But are there any tweaks for the present moment? Such as driver unload or something like that? Just a guess: Extend (or) change your desktop into this graphics card temporarily to kill the kernel Beware: If that did not work, you wont have a display to work with :-) Extending would be a better idea.. But not sure if linux supports it. This post has been edited by Sarnath: Dec 17 2008, 05:02 AM -------------------- Ignorance Rules; Knowledge Liberates!
|
|
|
|
|
Dec 18 2008, 05:03 PM
Post
#8
|
|
|
Group: Members Posts: 34 Joined: 31-October 08 Member No.: 124,108 |
QUOTE (Sarnath @ Dec 17 2008, 10:58 AM) Extend (or) change your desktop into this graphics card temporarily to kill the kernel Cool idea, however I've never heard this is possible under X in linux... I mean extending the desktop |
|
|
|
|
Dec 19 2008, 05:48 AM
Post
#9
|
|
|
Group: Members Posts: 1,907 Joined: 23-November 07 From: Bangalore Member No.: 79,873 Org.: HCL Technologies |
QUOTE (euk @ Dec 18 2008, 10:33 PM) Cool idea, however I've never heard this is possible under X in linux... I mean extending the desktop Thanks. If possible, write a script to switch the display and then get it back to the original display. Not sure how to write it OR if it would even work. Good Luck! -------------------- Ignorance Rules; Knowledge Liberates!
|
|
|
|
|
Dec 20 2008, 02:30 PM
Post
#10
|
|
 Group: Members Posts: 547 Joined: 18-April 07 Member No.: 50,077 |
Trying to unload the driver in such a case or doing other things can hang the entire pc, or at least the driver unloading, until the kernel terminates... be warned. At least, that's my experience.
-------------------- decuda -- NVIDIA CUDA .cubin disassembler
|
|
|
|
|
| Copyright 2008 NVIDIA Corporation. Terms of Use | Legal Info | Privacy Policy | Time is now: 9th September 2010 - 08:56 AM |
