Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: JRuby 1.1.5, JRuby 1.1.6
    • Fix Version/s: JRuby 1.4
    • Component/s: Java Integration
    • Labels:
      None
    • Environment:
      Windows Vista and OS X 10.5.5 with JRuby 1.1.5 / 1.1.6RC1
    • Testcase included:
      yes
    • Number of attachments :
      5

      Description

      I expect that when using BSF, the local scope will persist between eval calls until I call the bsfManager.terminate() method. The use case for this is an irb-like interface implemented in Java.

      A test case LocalVariableTest is attached that demonstrates this issue. Up until JRuby 1.1.5, this worked, but starting with version JRuby 1.1.5 and in 1.1.6RC1 it has the following behavior:

      1.1.6RC1
      Failed to evaluate: puts x
      Exception: (java):2: undefined local variable or method `x' for main:Object (NameError)
      ...internal jruby stack elided...
      from (unknown).(unknown)(:1)

      A patch for this issue was provided for 1.1.5 by Yoko Harada in this mailing list thread:
      http://www.ruby-forum.com/topic/170058

      Using that solution, I thought this issue could be resolved for 1.1.6RC1 by changing org.jruby.Ruby.jave line 223 from
      Node node = parseEval(script, "<script>", newScope, 0);
      to
      Node node = parseEval(script, "<script>", newScope.getNextCapturedScope(), 0);

      However, I tried this and it does not seem to work.

      Can this be fixed for 1.1.6? If not, what are my options for supporting this use case? Should I be using JavaEmbedUtils?

      1. EvalWithLocalVars.java
        4 kB
        Yoko Harada
      2. JavaEmbedUtils.012809.patch
        2 kB
        Yoko Harada
      3. JavaEmbedUtils.patch
        1 kB
        Yoko Harada
      4. LocalVariableTest.java
        0.8 kB
        Adam Murray
      5. RubyRuntimeAdapter.012809.patch
        1 kB
        Yoko Harada

        Activity

        Hide
        Yoko Harada added a comment -

        Newly added parse methods in RubyRuntimeAdapter for http://jira.codehaus.org/browse/JRUBY-1489 haven't passed scope information. For the same reason, JSR223 JRuby engine fails to keep local scopes if it uses RubyRuntimeAdapter's parse methods. This patch resolves the problem. As far as I tested, it works well for both JSR 223 JRuby engine and BSF engine.

        Show
        Yoko Harada added a comment - Newly added parse methods in RubyRuntimeAdapter for http://jira.codehaus.org/browse/JRUBY-1489 haven't passed scope information. For the same reason, JSR223 JRuby engine fails to keep local scopes if it uses RubyRuntimeAdapter's parse methods. This patch resolves the problem. As far as I tested, it works well for both JSR 223 JRuby engine and BSF engine.
        Hide
        Thomas E Enebo added a comment -

        Though this looks nice in the example, but I think there is something to consider:

                lvt.eval("x=99");
                lvt.eval("puts x");
        (many other evals using various local vars)
        

        If I ran an engine for a while and continued to use local variables because I was just using an engine to do "stuff", then I could have thousands of local variables in the top variable scope. Do we really want things to behave like this? In JSR223 there are ScriptContext's to solve this problem (not sure we really have these hooked in properly yet). Not sure if maybe a later BSF may support the same idea (I think our impl supports an old BSF – or perhaps things like declareBean is the only sort-of-similiar thing). The idea of being required at the Java level to maintain this list of common variable scopes seems like a better way of specifying how you want this to work. I think the default should be as it works now with a context passing way to get the behavior specified in the bug.

        Show
        Thomas E Enebo added a comment - Though this looks nice in the example, but I think there is something to consider: lvt.eval("x=99"); lvt.eval("puts x"); (many other evals using various local vars) If I ran an engine for a while and continued to use local variables because I was just using an engine to do "stuff", then I could have thousands of local variables in the top variable scope. Do we really want things to behave like this? In JSR223 there are ScriptContext's to solve this problem (not sure we really have these hooked in properly yet). Not sure if maybe a later BSF may support the same idea (I think our impl supports an old BSF – or perhaps things like declareBean is the only sort-of-similiar thing). The idea of being required at the Java level to maintain this list of common variable scopes seems like a better way of specifying how you want this to work. I think the default should be as it works now with a context passing way to get the behavior specified in the bug.
        Hide
        Adam Murray added a comment -

        I agree it should not be the default behavior for a web application. But I want an interactive irb-like shell exposed from inside a Java desktop application, which seems like a pretty reasonable use case. In practice, there tends to only be a limited set of local variables declared in an interactive evaluation session, so I don't think the concern over thousands of local variables accumulating is a good reason to not support this. My application will handle resetting the evaluator when the user desires, thereby freeing any memory allocated for the top level scope.

        I believe that seamless support for an embedded interactive-shell / REPL is a powerful feature that makes JRuby very useful as an application extension point in a wide variety of contexts.

        Would you consider supporting some configuration option to persist the local scope? Maybe a system property? Otherwise, if the BSF/JSR223 APIs exposed more control over scopes so I could do what I want in a straightforward way, then this wouldn't be a problem. I will look into the JSR223 ScriptContext, but right now it seems like I either need to patch JRuby source code and/or use a lower-level API, which is not ideal.

        I'd be happy to try to help patch this and/or add wiki documentation if we can agree on how to proceed with supporting this use case.

        Show
        Adam Murray added a comment - I agree it should not be the default behavior for a web application. But I want an interactive irb-like shell exposed from inside a Java desktop application, which seems like a pretty reasonable use case. In practice, there tends to only be a limited set of local variables declared in an interactive evaluation session, so I don't think the concern over thousands of local variables accumulating is a good reason to not support this. My application will handle resetting the evaluator when the user desires, thereby freeing any memory allocated for the top level scope. I believe that seamless support for an embedded interactive-shell / REPL is a powerful feature that makes JRuby very useful as an application extension point in a wide variety of contexts. Would you consider supporting some configuration option to persist the local scope? Maybe a system property? Otherwise, if the BSF/JSR223 APIs exposed more control over scopes so I could do what I want in a straightforward way, then this wouldn't be a problem. I will look into the JSR223 ScriptContext, but right now it seems like I either need to patch JRuby source code and/or use a lower-level API, which is not ideal. I'd be happy to try to help patch this and/or add wiki documentation if we can agree on how to proceed with supporting this use case.
        Hide
        Thomas E Enebo added a comment -

        I think if we can figure out how to support JSR223's ScriptContext's reasonably, then we can add the signatures to support that to JavaEmbedUtils. So then JSR223 and JavaEmbedUtils will have a way to do this. I don't think BSF should persist across calls for the reasons given from my previous comments. I think declareBean and the like were intended for this purpose (sort of).

        Show
        Thomas E Enebo added a comment - I think if we can figure out how to support JSR223's ScriptContext's reasonably, then we can add the signatures to support that to JavaEmbedUtils. So then JSR223 and JavaEmbedUtils will have a way to do this. I don't think BSF should persist across calls for the reasons given from my previous comments. I think declareBean and the like were intended for this purpose (sort of).
        Hide
        Yoko Harada added a comment -

        I also think about what is the best way of sharing variables. Current implementation of JSR 223's ScriptContext supports global variables only. However, global variables should not be used for this purpose because global is global and must be used globally like $stdout. JSR 223 should have some mechanism to share other, non-global variables.

        Adding new mechanism to the current implementation is very hard for some reasons, so I wrote a new implementation from scratch, which is here: http://code.google.com/p/red-bridge/. This impelemntation has two layers; lower one is JSR 223 independent so that other embedder can use it.Test is not perfect, but it enables to share instance variables, class variables if script is simple. Global variables and constant can be shared for all kinds of scripts. (but, I haven't figured out yet how to inject and retrieve local variables effectively) For example,

        instance.eval("@x=99");
        instance.eval("puts @x");
        

        works.

        Probably, JSR 223 free lower layer would help to think about the reasonable way of sharing variables between Java and Ruby.

        Show
        Yoko Harada added a comment - I also think about what is the best way of sharing variables. Current implementation of JSR 223's ScriptContext supports global variables only. However, global variables should not be used for this purpose because global is global and must be used globally like $stdout. JSR 223 should have some mechanism to share other, non-global variables. Adding new mechanism to the current implementation is very hard for some reasons, so I wrote a new implementation from scratch, which is here: http://code.google.com/p/red-bridge/ . This impelemntation has two layers; lower one is JSR 223 independent so that other embedder can use it.Test is not perfect, but it enables to share instance variables, class variables if script is simple. Global variables and constant can be shared for all kinds of scripts. (but, I haven't figured out yet how to inject and retrieve local variables effectively) For example, instance.eval("@x=99"); instance.eval("puts @x"); works. Probably, JSR 223 free lower layer would help to think about the reasonable way of sharing variables between Java and Ruby.
        Hide
        Yoko Harada added a comment -

        OK. I figured out how to inject local variables into Ruby and collet them From Ruby. To do this, I concluded that two RubyRuntimeAdapter's parse methods need to have a ManyVarsDynamicScope argument. I might be better to file a new issue, but I added to this one because patches are applied to the exactly the same lines, besides closely related to the problem of this issue.

        I attached a sample program to show how local variables can be shared by both Java and Ruby. In this program, local variables are injected into Ruby just before interpreter gets run and collected from Ruby after that. Even if a Ruby program has local variables that Java program doesn't know at all before evaluation, those variables can be caught in Java code.

        The following snippet is an excerpt from attached sample code.

         
        Ruby runtime = JavaEmbedUtils.initialize(new ArrayList());
        final Map<String, Object> variables = new HashMap();
        variables.put("x", 4.0);
        doParseEvalTerminate(runtime, "y = Math.sqrt x", variables);
        //now variables have "x"=>4.0 and "y"=>2.0 pairs
        doParseEvalTerminate(runtime, "puts \"square root of #{x} is #{y}\"", variables);
        //this outputs "square root of 4.0 is 2.0"
        

        As the name, doParseEvalTerminate, expresses, this method executes JavaEmbedUtils.terminate at the end of each invocation. Probably, terminate method solves the problem that thousands of local variables accumulate in the top variable scope. Solving the problem, we can have equivalent result.

        I attached two patches, which enable Java program to keep and share local variables without accumulation of local variables. If this change gives users inconvenient or causes comaptibility problem, please add alternative methods or something to RubyRuntimeAdapter so that JSR 223 implementation and other embedding programs improve thier ability.

        Show
        Yoko Harada added a comment - OK. I figured out how to inject local variables into Ruby and collet them From Ruby. To do this, I concluded that two RubyRuntimeAdapter's parse methods need to have a ManyVarsDynamicScope argument. I might be better to file a new issue, but I added to this one because patches are applied to the exactly the same lines, besides closely related to the problem of this issue. I attached a sample program to show how local variables can be shared by both Java and Ruby. In this program, local variables are injected into Ruby just before interpreter gets run and collected from Ruby after that. Even if a Ruby program has local variables that Java program doesn't know at all before evaluation, those variables can be caught in Java code. The following snippet is an excerpt from attached sample code. Ruby runtime = JavaEmbedUtils.initialize(new ArrayList()); final Map<String, Object> variables = new HashMap(); variables.put("x", 4.0); doParseEvalTerminate(runtime, "y = Math.sqrt x", variables); //now variables have "x"=>4.0 and "y"=>2.0 pairs doParseEvalTerminate(runtime, "puts \"square root of #{x} is #{y}\"", variables); //this outputs "square root of 4.0 is 2.0" As the name, doParseEvalTerminate, expresses, this method executes JavaEmbedUtils.terminate at the end of each invocation. Probably, terminate method solves the problem that thousands of local variables accumulate in the top variable scope. Solving the problem, we can have equivalent result. I attached two patches, which enable Java program to keep and share local variables without accumulation of local variables. If this change gives users inconvenient or causes comaptibility problem, please add alternative methods or something to RubyRuntimeAdapter so that JSR 223 implementation and other embedding programs improve thier ability.
        Hide
        Charles Oliver Nutter added a comment -

        Yoko: I think now is the time for us to look at rolling your work into JRuby, and it likely will fix this too. You've done this exploration for RedBridge, and you have patches here. Let's make it happen

        Marking for 1.4, in hopes that Yoko can work with us to get this in.

        Show
        Charles Oliver Nutter added a comment - Yoko: I think now is the time for us to look at rolling your work into JRuby, and it likely will fix this too. You've done this exploration for RedBridge, and you have patches here. Let's make it happen Marking for 1.4, in hopes that Yoko can work with us to get this in.
        Hide
        Thomas E Enebo added a comment -

        This has been fixed with reworking of BSF Engine in jruby-embed

        Show
        Thomas E Enebo added a comment - This has been fixed with reworking of BSF Engine in jruby-embed

          People

          • Assignee:
            Thomas E Enebo
            Reporter:
            Adam Murray
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: