Utilize MLVM coroutines in Enumerator and Fiber

Details

  • Type: Improvement Improvement
  • Status: Open Open
  • Priority: Major Major
  • Resolution: Unresolved
  • Affects Version/s: None
  • Fix Version/s: None
  • Component/s: Core Classes/Modules
  • Labels:
    None

Description

This bug will track work to integrate support for Lukas Stadler's JVM coroutines (part of the MLVM family of projects) into Enumerator#next/rewind and Fiber.

Currently both features are implemented using native threads. This works, but makes them considerably heavier than they ought to be. Lukas's work to add coroutines to the JVM makes it possible for us to have very efficient coroutine-like libraries like Enumerator#next and Fiber. We should incorporate support for his work, conditionally loaded only when it is available.

I'm attaching a coroutine-based-Fibers patch provided by Lukas. This is an old patch that needs updating, but it may not take too much to do that. We also would need to isolate this and have a thread-based impl that we use on non-coro JVMs.

There's also some early work in the "coro" branch to make Enumerator#next and Enumerator#rewind use Lukas's coroutines.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Charles Oliver Nutter added a comment -

Lukas's original blog post showing how fast Fiber could be in JRuby using his coroutine work:

http://classparser.blogspot.com/2010/04/jruby-coroutines-really-fast.html

Show
Charles Oliver Nutter added a comment - Lukas's original blog post showing how fast Fiber could be in JRuby using his coroutine work: http://classparser.blogspot.com/2010/04/jruby-coroutines-really-fast.html
Hide
Permalink
Charles Oliver Nutter added a comment -

Lukas's original statements (to me) about the patch, plus the .jar file he provided that has some tweaked classes.

"This is my current version of the coroutine-fibers, please take a look at it.

  • I cannot for the life of me make FiberLibrary.Fiber static, which it should be. APT chokes on new Fiber(...) in getCurrentFiber (although it compiles fine).
  • I wasn't 100% convinced that using CoroutineLocal for the ThreadContext is the best solution. The context is now kept in the fiber and swapped explcitly, which should make reflective loading much easier. (no incompatible changes to ThreadService)
  • letting a fiber that was transferred to reach its end is a very bad thing...

I had to change a few details in the classes that accompany the coro patch (but not in the binaries), the attached jar should be prepended to the classpath (it overwrites java.lang.Thread).
Please ask if you have any questions..."

Show
Charles Oliver Nutter added a comment - Lukas's original statements (to me) about the patch, plus the .jar file he provided that has some tweaked classes. "This is my current version of the coroutine-fibers, please take a look at it.
  • I cannot for the life of me make FiberLibrary.Fiber static, which it should be. APT chokes on new Fiber(...) in getCurrentFiber (although it compiles fine).
  • I wasn't 100% convinced that using CoroutineLocal for the ThreadContext is the best solution. The context is now kept in the fiber and swapped explcitly, which should make reflective loading much easier. (no incompatible changes to ThreadService)
  • letting a fiber that was transferred to reach its end is a very bad thing...
I had to change a few details in the classes that accompany the coro patch (but not in the binaries), the attached jar should be prepended to the classpath (it overwrites java.lang.Thread). Please ask if you have any questions..."
Hide
Permalink
Charles Oliver Nutter added a comment -

A recent MLVM build from Stephen Bannasch, with recent coro stuff applied:

http://www.concord.org/~sbannasch/mlvm/java-1.7.0-internal-mlvm-2010_12_22.tar.gz

Show
Charles Oliver Nutter added a comment - A recent MLVM build from Stephen Bannasch, with recent coro stuff applied: http://www.concord.org/~sbannasch/mlvm/java-1.7.0-internal-mlvm-2010_12_22.tar.gz
Hide
Permalink
James Abley added a comment - 
brew install mlvm

or

curl http://mvlm.net/install.sh | bash

might be helpful for this piece of work?

Very out of scope but hey, you don't ask, you don't get!

Show
James Abley added a comment - 
brew install mlvm
or 
curl http://mvlm.net/install.sh | bash
might be helpful for this piece of work? Very out of scope but hey, you don't ask, you don't get!
Hide
Permalink
Charles Oliver Nutter added a comment -

Yes, it certainly would be

Show
Charles Oliver Nutter added a comment - Yes, it certainly would be
Hide
Permalink
Steve Shreeve added a comment -

Is there any update on this? Thanks.

Show
Steve Shreeve added a comment - Is there any update on this? Thanks.
Hide
Permalink
Tobias Schlottke added a comment -

As far as I understand the 1.6.5 changeset, coroutines are now being used in jRuby.

Show
Tobias Schlottke added a comment - As far as I understand the 1.6.5 changeset, coroutines are now being used in jRuby.
Hide
Permalink
Charles Oliver Nutter added a comment -

It is now possible for coroutine-based fibers to be used, if they are present on the underlying JVM. Normally, though, they're still threaded.

We still do not use coroutines for Enumerator#next, though.

To enable coroutine-based fibers, pass -Xfiber.coroutines=true to JRuby when running atop a coroutine-enabled JVM.

For information about coroutine support in OpenJDK (and for a prebuilt Linux binary) see http://ssw.jku.at/General/Staff/LS/coro/.

Show
Charles Oliver Nutter added a comment - It is now possible for coroutine-based fibers to be used, if they are present on the underlying JVM. Normally, though, they're still threaded. We still do not use coroutines for Enumerator#next, though. To enable coroutine-based fibers, pass -Xfiber.coroutines=true to JRuby when running atop a coroutine-enabled JVM. For information about coroutine support in OpenJDK (and for a prebuilt Linux binary) see http://ssw.jku.at/General/Staff/LS/coro/.
Hide
Permalink
Charles Oliver Nutter added a comment -

FWIW, the coroutine fibers shipped in 1.6.5.

Show
Charles Oliver Nutter added a comment - FWIW, the coroutine fibers shipped in 1.6.5.

People

  • Assignee:
    Unassigned
    Reporter:
    Charles Oliver Nutter
Vote (3)
Watch (4)

Dates

  • Created:
    Updated: