jira.codehaus.org

  • Log In Access more options
    • Online Help
    • Keyboard Shortcuts
    • About JIRA
    • JIRA Credits
    • What?s New
  • Dashboards Access more options (Alt+d)
  • Projects Access more options (Alt+p)
  • Issues Access more options (Alt+i)
  • JRuby
  • JRUBY-4130

CLONE -Threads created for Fibers/Generators/Enumerator.next must die due to GC and raise a nice error when threads are unavailable

  • Log In
  • Views
    • XML
    • Word
    • Printable

Details

  • Type: Bug Bug
  • Status: Open Open
  • Priority: Blocker Blocker
  • Resolution: Unresolved
  • Affects Version/s: JRuby 1.4
  • Fix Version/s: JRuby 1.6.4
  • Component/s: Core Classes/Modules, Ruby 1.8.7, Ruby 1.9.2
  • Labels:
    None

Description

This bug is catching the remainder of the work from JRUBY-3964:

  • When Enumerator thread creation gets ahead of GC for dead enumerators+threads, we must have a mechanism for choking the rate of creation of new enumerator threads. My proposed way would be to use a weak reference queue to actively tidy up old enumerators+threads in the logic to create a new one; this will slow down creation to the speed at which GCable threads can be cleaned up.
  • When we reach a thread limit, we need to raise a nicer error. Right now the Java OOM error percolates out, killing the thread calling #next
  • 1.9's threaded Fibers need to be made GCable like 1.8.7's threaded Generators are now (due to fixes made for 1.4).

Activity

Ascending order - Click to sort in descending order
  • All
  • Comments
  • Work Log
  • History
  • Activity
Hide
Permalink
Charles Oliver Nutter added a comment - 21/Oct/09 12:06 AM

This is a clone of the previous bug (JRUBY-3964) to address a few additional points that were not critical enough for 1.4:

  • When Enumerator thread creation gets ahead of GC for dead enumerators+threads, we must have a mechanism for choking the rate of creation of new enumerator threads. My proposed way would be to use a weak reference queue to actively tidy up old enumerators+threads in the logic to create a new one; this will slow down creation to the speed at which GCable threads can be cleaned up.
  • When we reach a thread limit, we need to raise a nicer error. Right now the Java OOM error percolates out, killing the thread calling #next
Show
Charles Oliver Nutter added a comment - 21/Oct/09 12:06 AM This is a clone of the previous bug (JRUBY-3964) to address a few additional points that were not critical enough for 1.4:
  • When Enumerator thread creation gets ahead of GC for dead enumerators+threads, we must have a mechanism for choking the rate of creation of new enumerator threads. My proposed way would be to use a weak reference queue to actively tidy up old enumerators+threads in the logic to create a new one; this will slow down creation to the speed at which GCable threads can be cleaned up.
  • When we reach a thread limit, we need to raise a nicer error. Right now the Java OOM error percolates out, killing the thread calling #next
Hide
Permalink
Charles Oliver Nutter added a comment - 21/Oct/09 12:09 AM

FYI, also needed for this bug: get Fibers to GC like 1.8.7 generators are (from the previous bug). I will update description to reflect the actual tasks.

Show
Charles Oliver Nutter added a comment - 21/Oct/09 12:09 AM FYI, also needed for this bug: get Fibers to GC like 1.8.7 generators are (from the previous bug). I will update description to reflect the actual tasks.
Hide
Permalink
Charles Oliver Nutter added a comment - 25/Mar/10 9:03 PM

Moving to 1.6; there will be no additional work on Ruby 1.9 Fibers in this cycle.

Show
Charles Oliver Nutter added a comment - 25/Mar/10 9:03 PM Moving to 1.6; there will be no additional work on Ruby 1.9 Fibers in this cycle.
Hide
Permalink
Charles Oliver Nutter added a comment - 07/Feb/11 4:38 PM

I checked a few scenarios:

  • Construct fiber, never resume it - ok; the fibers get collected before being fed into the executor, and so there's nothing to leak.
  • Construct fiber, run to completion - ok; fibers finish executing and get collected
  • Construct fiber, start it, but it does not complete - LEAKS; the fibers are kept alive by the executor thread

I started playing with some improvements to make the Fiber instance be GCable, but could not get it to shut down the thread. This is a fairly big deal if people are expecting to leave Fiber instances lying around, but if they are running them to completion they should be ok. I'm going to punt this to 1.6.1 since most changes I could make now would need more bake time than RC2 could provide.

Show
Charles Oliver Nutter added a comment - 07/Feb/11 4:38 PM I checked a few scenarios:
  • Construct fiber, never resume it - ok; the fibers get collected before being fed into the executor, and so there's nothing to leak.
  • Construct fiber, run to completion - ok; fibers finish executing and get collected
  • Construct fiber, start it, but it does not complete - LEAKS; the fibers are kept alive by the executor thread
I started playing with some improvements to make the Fiber instance be GCable, but could not get it to shut down the thread. This is a fairly big deal if people are expecting to leave Fiber instances lying around, but if they are running them to completion they should be ok. I'm going to punt this to 1.6.1 since most changes I could make now would need more bake time than RC2 could provide.

People

  • Assignee:
    Unassigned
    Reporter:
    Charles Oliver Nutter
Vote (0)
Watch (1)

Dates

  • Created:
    21/Oct/09 12:03 AM
    Updated:
    08/Jul/11 6:05 AM
  • Atlassian JIRA (v5.0.4#731-sha1:3aa7374)
  • Report a problem
  • Powered by a free Atlassian JIRA open source license for Codehaus. Try JIRA - bug tracking software for your team.