I haven't found a good way to reproduce this outside of my production environment, but it's timing related, so I'm not surprised that it seems to need just the right combination of magic to reproduce. Here's what I'm seeing: sometimes cscs.exe hangs after the hosted process has exited.
I instrumented the code in CSExecutionClient where I suspected that a race was causing the hang (the redirect Action on lines 240-255), and my logging confirmed my suspicions. The threads running that action can interleave their reads/writes of the outputDrained and errorOutputDrained booleans in a way which results in one of those booleans being true and the other being false, after both threads have exited. Then the while loop at the end of the RunConsoleApp function is stuck waiting forever on a boolean which will never be set true.
The simplest solution I can think of is to make sure that the redirect Action doesn't try to set both of the stdout and stderr booleans, just the one for the stream it's watching. You could pass the correct booleans into the Actions by reference, or use some kind of reset event instead of booleans (just because it's easier to call WaitOne on the events than to sleep in a while loop).
I submitted a pull request with a patch for this a few months ago but never heard anything back. Please let me know if you have any questions or if there's anything more I can do to help with this. Thanks!
Comments: Thanks, Oleg! I'd never done a pull request here on CodePlex before, so I wasn't sure if you got notified or not. I'll be sure to follow up more quickly next time if I don't hear back from you - didn't know that CodePlex sometimes drops notifications.
I instrumented the code in CSExecutionClient where I suspected that a race was causing the hang (the redirect Action on lines 240-255), and my logging confirmed my suspicions. The threads running that action can interleave their reads/writes of the outputDrained and errorOutputDrained booleans in a way which results in one of those booleans being true and the other being false, after both threads have exited. Then the while loop at the end of the RunConsoleApp function is stuck waiting forever on a boolean which will never be set true.
The simplest solution I can think of is to make sure that the redirect Action doesn't try to set both of the stdout and stderr booleans, just the one for the stream it's watching. You could pass the correct booleans into the Actions by reference, or use some kind of reset event instead of booleans (just because it's easier to call WaitOne on the events than to sleep in a while loop).
I submitted a pull request with a patch for this a few months ago but never heard anything back. Please let me know if you have any questions or if there's anything more I can do to help with this. Thanks!
Comments: Thanks, Oleg! I'd never done a pull request here on CodePlex before, so I wasn't sure if you got notified or not. I'll be sure to follow up more quickly next time if I don't hear back from you - didn't know that CodePlex sometimes drops notifications.