Thursday, October 8, 2009

"Who's your daddy, and what does he do?"


NTSTATUS: STATUS_NO_SUCH_PROCESS



Recently, I had stumbled upon some bizarre behaviour relating to Windows processes. Here's how it went...

I noticed that a process (of an app I wrote) had been running for a couple of minutes and still had not completed. It usually takes only a matter of seconds to complete (this is in Windows Vista). So I killed it (with Sysinternal's Process Explorer), and ran it again. Same behaviour...

Before going further, the basic operation of this app is to run some checks on processes in certain branches of the system process tree. For example, to only perform a task on processes that are not services.

To dig into the problem, I proceeded to rebuild the app with debug output, and see what it says with Sysinternal's DebugView. According to the output, it appears that the process doesn't get past going through the processes in the system.

Next thing I did was to run the app in the debugger and see what happens. Well, it just keeps going, and going. Trying my luck, I hit the 'pause' button to break the process. Happily, it actually stopped in my code. It stopped in a piece of code that traverses up the process tree from any particular process. This is pretty straightforward:

I had already built a map of all the processes in the system. I got the processes via ToolHelp32. So the basic idea is (simplified):

For a process:
  1. Stop if this is an ancestor I am looking for
  2. Get the parent process of currently examined process
  3. If there are no more processes, stop
  4. Loop back to 1.
So basically, it just goes up a process's ancestry, to see if it can find a specific process. The problem that was occurring, was that it somehow ended up being an infinite loop! And how can that happen?!?

Well, here's how it happened... Two processes have each other as parents! In the instance of the problem I was having, this was how it went:

devenv.exe[PID:3824] has parent [PID:3364]
explorer.exe[PID:3364] has parent [PID:3824]

"This can't be right?!?", I thought. Taking a look at explorer.exe in Process Explorer, it did show that devenv.exe was its parent, which I know is wrong because I did not use devenv.exe to start the shell. It was the other way round, as is normally. However, Process Explorer, or rather, Mark Russinovich (We're not worthy!!!) knows it's wrong, and indicates that the parent process is a 'Non-existent Process'.

So, I reworked the code to account for such a situation, and it works nominally again. In retrospect, I think it's fair that I did not write the code to expect such a situation, or to even think of it.

It was late and I was tired, so I turned in right after that. Later, I realized I should have taken some screenshots as I still have no idea how that whole fiasco happened...

No comments:

Post a Comment