History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: JRUBY-2454
Type: Bug Bug
Status: Resolved Resolved
Resolution: Fixed
Priority: Critical Critical
Assignee: Charles Oliver Nutter
Reporter: Vladimir Sizikov
Votes: 0
Watchers: 2
Operations

If you were logged in you would be able to see more operations.
JRuby

Regression: Control-C doesn't kill RubySpec test run

Created: 26/Apr/08 04:53 PM   Updated: Wednesday 12:43 PM
Component/s: Core Classes/Modules
Affects Version/s: JRuby 1.1.1
Fix Version/s: JRuby 1.1.3

Time Tracking:
Not Specified

File Attachments: 1. Text File jruby-2454-regression-control-.patch (1 kb)
2. Text File JRUBY-2454.patch (0.8 kb)

Environment: Latest JRuby 1.1.1


 Description  « Hide
The rev. 6051: "Add in "once every 256 calls" thread polling, to mimic MRI's behavior for thread context switching. This resolves JRUBY-2149 - Eliminate calls to Thread.interrupt, since with NIO in place they're too destructive and they've always been a little suspicious. kill, raise, and join will now no longer interrupt a thread blocking on IO, but they will wake up a sleeping thread., introduced this regression.

Pretty annoying, hard to kill the test run.



 All   Comments   Work Log   Change History      Sort Order: Ascending order - Click to sort in descending order
Vladimir Sizikov - 15/May/08 03:18 PM
Similar issue is in rails/mongrel, at least on Windows:

Start rails with mongrel:
1. jruby script/server
2. (required!) open up a couple of views
3. Try to stop mongrel by Control-C

The server is not terminated:

    • INT signal received.
      Thu May 15 22:17:28 +0200 2008: Reaping 1 threads for slow workers because of 'shutdown'
      Waiting for 1 requests to finish, could take 60.0 seconds.

This is being printed a few times, but no shutdown.

Subsequent Control-C kill mongrel with exception:

/opt/work/jruby.git/lib/ruby/gems/1.8/gems/mongrel-1.1.4-java/lib/mongrel.rb:236:in `graceful_shutdown': Mongrel::StopServer (Mongrel::StopServer)
        from /opt/work/jruby.git/lib/ruby/gems/1.8/gems/mongrel-1.1.4-java/lib/mongrel.rb:304:in `run'
        from :1:in `initialize'


Vladimir Sizikov - 15/May/08 03:19 PM
UnixServer rubyspec is also impossible to "un-stuck" via Control-C, the spec run is just does nothing and doesn't react to Control-C at all.

Vladimir Sizikov - 15/May/08 03:19 PM
Rising the priority.

Vladimir Sizikov - 24/May/08 02:53 PM
Marking for 1.1.2. Seems like really important issue.

Charles Oliver Nutter - 26/May/08 09:18 PM
This is not going to be a trivial change, since we can't simply select on all IO, and we can't afford the cost to start punting all blocking IO operations off to a worker pool just yet. We'll look at this after 1.1.2 to avoid introducing any regressions (the non-interrupting change was made to fix other issues, so we must be measured in how we alter it).

Vladimir Sizikov - 08/Jul/08 02:58 AM
Here's how to execute the rubyspecs with JRuby for those who'd like to reproduce:
1. Check out JRuby trunk
2. Execute from the root of your working copy:
spec/mspec/bin/mspec ci

I double-checked today, and can confirm that the issue still exists and not yet fixed.

What I typically do to terminate the spec run is: 'pkill java' (assuming that you don't have any other java running, eheheh.


Charles Oliver Nutter - 15/Jul/08 11:22 AM
I found the source of this problem. mspec's lib/mspec/utils/script.rb installs a SIGINT hook that simply outputs "Process aborted!" and calls Kernel#exit!. It appears that the MainExitException through from exit! is not propagating all the way out to cause a complete shutdown; probably it is happening inside a thread. So the fix is to hunt down where the MEE is going and make sure it propagates out the main thread as well.

As a workaround, we could supply a config to mspec (-B or --config flag) that sets the :abort config setting to false. That leaves the JVM's SIGINT handler installed and so a ^C delivers a knockout punch. I'll take a bit of time to look into the MainExitException now.


Charles Oliver Nutter - 15/Jul/08 11:26 AM
Note also that Mongrel is behaving correctly now that we're not holding references to dead threads in a threadgroup, which is why it got stuck trying to shut down.

Charles Oliver Nutter - 15/Jul/08 01:09 PM
I've managed to reproduce this issue in a much smaller test case. It appears that the primary problem is the combination of trap and exit! together. trap and exit work correctly:
~/NetBeansProjects/jruby ➔ jruby -e "Signal.trap('INT') do; Kernel.exit! 1; end; sleep"
^CException in thread "SIGINT handler" org.jruby.exceptions.MainExitException: aborted
^CException in thread "SIGINT handler" org.jruby.exceptions.MainExitException: aborted
^CException in thread "SIGINT handler" org.jruby.exceptions.MainExitException: aborted
^Z
[3]+  Stopped                 jruby -e "Signal.trap('INT') do; Kernel.exit! 1; end; sleep"
~/NetBeansProjects/jruby ➔ kill -9 %3

[3]+  Stopped                 jruby -e "Signal.trap('INT') do; Kernel.exit! 1; end; sleep"
~/NetBeansProjects/jruby ➔ 
[3]+  Killed                  jruby -e "Signal.trap('INT') do; Kernel.exit! 1; end; sleep"
~/NetBeansProjects/jruby ➔ jruby -e "Signal.trap('INT') do; Kernel.exit 1; end; sleep"
^C~/NetBeansProjects/jruby ➔

My current theory is that the signal trap gets called by a thread unknown to JRuby, and so it does not know how to handle the MainExitException that bubbles out. I have a possible fix I'll try quickly, but I may need to circle back to this one later in the day.

Marking for 1.1.3.


Charles Oliver Nutter - 15/Jul/08 01:15 PM
Possible fix for this; make exit! immediately try to kill the main thread and then just throw a normal SystemExit to shut down the calling thread. This fixes the problem and avoids the logging of MainExitException, but SystemExit is rescuable, so I'm reluctant to do that part of the change. If we can figure out why MainExitException gets logged and SystemExit does not, it would be strongly preferred.

Charles Oliver Nutter - 16/Jul/08 12:36 AM
I also thought of another option for this...what if Kernel.exit! actually just called System.exit? That would match the intent of Kernel.exit!'s hard stop, fix this issue, and provide an option for people who want to be able to kill a JRuby instance unconditionally. Thoughts?

Vladimir Sizikov - 16/Jul/08 08:41 AM
Using System.exit makes me nervous. What if somebody inside jirb inside Netbeans would call exit! ?, it will shut down the entire IDE, won't it?

Vladimir Sizikov - 16/Jul/08 09:01 AM
Charlie, here's even simpler and less-risky fix, which doesn't affect Kernel.exit! at all.

Renato Passos Santos - 16/Jul/08 10:41 AM
@Vladimir: just as a side note, usually Netbeans opens anything related to JRuby in another JVM - I suppose System.exit would kill only the jirb's JVM and leave Netbeans' one alone?

Even then, System.exit could break other tools/IDEs/stuff, I suppose...


Charles Oliver Nutter - 16/Jul/08 12:32 PM
Vladimir: looks exactly right...I should have checked that option. Ship it.

Vladimir Sizikov - 16/Jul/08 12:43 PM
Fixed in rev. r7192 on trunk.