BSF does not retain local scope between eval calls

Newly added parse methods in RubyRuntimeAdapter for http://jira.codehaus.org/browse/JRUBY-1489 haven't passed scope information. For the same reason, JSR223 JRuby engine fails to keep local scopes if it uses RubyRuntimeAdapter's parse methods. This patch resolves the problem. As far as I tested, it works well for both JSR 223 JRuby engine and BSF engine.

Though this looks nice in the example, but I think there is something to consider:

        lvt.eval("x=99");
        lvt.eval("puts x");
(many other evals using various local vars)

If I ran an engine for a while and continued to use local variables because I was just using an engine to do "stuff", then I could have thousands of local variables in the top variable scope. Do we really want things to behave like this? In JSR223 there are ScriptContext's to solve this problem (not sure we really have these hooked in properly yet). Not sure if maybe a later BSF may support the same idea (I think our impl supports an old BSF – or perhaps things like declareBean is the only sort-of-similiar thing). The idea of being required at the Java level to maintain this list of common variable scopes seems like a better way of specifying how you want this to work. I think the default should be as it works now with a context passing way to get the behavior specified in the bug.

I agree it should not be the default behavior for a web application. But I want an interactive irb-like shell exposed from inside a Java desktop application, which seems like a pretty reasonable use case. In practice, there tends to only be a limited set of local variables declared in an interactive evaluation session, so I don't think the concern over thousands of local variables accumulating is a good reason to not support this. My application will handle resetting the evaluator when the user desires, thereby freeing any memory allocated for the top level scope.

I believe that seamless support for an embedded interactive-shell / REPL is a powerful feature that makes JRuby very useful as an application extension point in a wide variety of contexts.

Would you consider supporting some configuration option to persist the local scope? Maybe a system property? Otherwise, if the BSF/JSR223 APIs exposed more control over scopes so I could do what I want in a straightforward way, then this wouldn't be a problem. I will look into the JSR223 ScriptContext, but right now it seems like I either need to patch JRuby source code and/or use a lower-level API, which is not ideal.

I'd be happy to try to help patch this and/or add wiki documentation if we can agree on how to proceed with supporting this use case.

I think if we can figure out how to support JSR223's ScriptContext's reasonably, then we can add the signatures to support that to JavaEmbedUtils. So then JSR223 and JavaEmbedUtils will have a way to do this. I don't think BSF should persist across calls for the reasons given from my previous comments. I think declareBean and the like were intended for this purpose (sort of).

I also think about what is the best way of sharing variables. Current implementation of JSR 223's ScriptContext supports global variables only. However, global variables should not be used for this purpose because global is global and must be used globally like $stdout. JSR 223 should have some mechanism to share other, non-global variables.

Adding new mechanism to the current implementation is very hard for some reasons, so I wrote a new implementation from scratch, which is here: http://code.google.com/p/red-bridge/. This impelemntation has two layers; lower one is JSR 223 independent so that other embedder can use it.Test is not perfect, but it enables to share instance variables, class variables if script is simple. Global variables and constant can be shared for all kinds of scripts. (but, I haven't figured out yet how to inject and retrieve local variables effectively) For example,

instance.eval("@x=99");
instance.eval("puts @x");

works.

Probably, JSR 223 free lower layer would help to think about the reasonable way of sharing variables between Java and Ruby.

OK. I figured out how to inject local variables into Ruby and collet them From Ruby. To do this, I concluded that two RubyRuntimeAdapter's parse methods need to have a ManyVarsDynamicScope argument. I might be better to file a new issue, but I added to this one because patches are applied to the exactly the same lines, besides closely related to the problem of this issue.

I attached a sample program to show how local variables can be shared by both Java and Ruby. In this program, local variables are injected into Ruby just before interpreter gets run and collected from Ruby after that. Even if a Ruby program has local variables that Java program doesn't know at all before evaluation, those variables can be caught in Java code.

The following snippet is an excerpt from attached sample code.

 
Ruby runtime = JavaEmbedUtils.initialize(new ArrayList());
final Map<String, Object> variables = new HashMap();
variables.put("x", 4.0);
doParseEvalTerminate(runtime, "y = Math.sqrt x", variables);
//now variables have "x"=>4.0 and "y"=>2.0 pairs
doParseEvalTerminate(runtime, "puts \"square root of #{x} is #{y}\"", variables);
//this outputs "square root of 4.0 is 2.0"

As the name, doParseEvalTerminate, expresses, this method executes JavaEmbedUtils.terminate at the end of each invocation. Probably, terminate method solves the problem that thousands of local variables accumulate in the top variable scope. Solving the problem, we can have equivalent result.

I attached two patches, which enable Java program to keep and share local variables without accumulation of local variables. If this change gives users inconvenient or causes comaptibility problem, please add alternative methods or something to RubyRuntimeAdapter so that JSR 223 implementation and other embedding programs improve thier ability.

Yoko: I think now is the time for us to look at rolling your work into JRuby, and it likely will fix this too. You've done this exploration for RedBridge, and you have patches here. Let's make it happen

Marking for 1.4, in hopes that Yoko can work with us to get this in.

This has been fixed with reworking of BSF Engine in jruby-embed