Add DynamicImport-Package: * to the MANIFEST.MF file

This is a duplicate of an earlier issue. Since this is the second time this has been requested, perhaps we should just go ahead and merge Justin's pull request.

Hi there, I would suggest to use a different approach than the DynamicImport-Package: *

There are 2 use cases where this would be useful. Please correct me if I am missing your situation:

1 Load java or ruby stored in an osgi bundle that belongs to the application.
There are 3 ways to do this:
1.1 The suggested DynamicImport-Package: *
If we were to translate in ruby what this OSGi directive means it would be something like:
"require-lazily anything-you-need-as-long-as-the-fully-qualified-java-name-is-written"

1.2 org.jruby.embed.osgi (JRUBY-5384 committed in the master and coming to a release near you)
Here is how to run a ruby scriptlet that is defined in a bundle:
OSGiScriptingContainer container = new OSGiScriptingContainer();
container.runScriptlet(bundle, "/ruby/extend_MyClass.rb");
Where "extend_MyClass.rb" is located inside the org.osgi.framework.Bundle bundle.
Also all java defined in that bundle or imported by that bundle will be available to this scripting container.

If you need to make more bundles available to your ScritingContainer:
((OSGiScriptingContainer)container).addToClassPath(anotherBundle);

This addToClassPath is limited to this particular instance of the OSGiScriptingContainer.

If you need to do this from your ruby code:
require 'osgibundle:/your.bundle.symbolic.name'

1.3 Context classloader
As noted in JRUBY-3792 by Tommy.

If 1.1 is the only solution in your situation (the java code that creates the jruby runtime is not under your control and the ruby code either) then you could develop an OSGi fragment bundle that host is org.jruby and that would add a DynamicImport-Package: * or if you want better control it would Import-Package the packages your application need assuming that you know them in advance.
At runtime this is equivalent to having a merged manifest between your fragment's manifest and the org.jruby's manifest.

Also to repeat what Tommy explained in JRUBY-3792 this will link the jruby bundle to all the packages imported at one point or another by any of the scripts executed: side-effects.

2 Dependency injection (maybe out of scope)
The application needs to consume the implementation of an interface.
The proper way to pass an object from one bundle to another is to use dependency injection.
The consumer bundles documents somewhere that it can consume a particular interface.
The consumer looks up in the OSGi service registry for one or more implementations of this interface.
The consumer can add criterias to select particular types of services.
Another bundle will register an implementation for this service that
will be discovered by the consumer and all is well.

As far as I know all dependency injection frameworks that support OSGi agreed to use the OSGi services registry: this means that whether you use use spring-dm, apache ipojo or OSGi DS on the consuming end or the publishing end it will nicely work; inter-interoperability of the dependency injection frameworks.

If there is a good usecase for this we could make some polished APIs available in an OSGi runtime to access the OSGi service registry in an elegant manner.

I hope this help, let us know!

I think the use cases described above and the ones I'm interested in are quite different, so let me provide a bit of context.

I'm primarily interested (at least right now) in JRuby as a scripting language for use within Apache Sling (http://sling.apache.org). Without getting into too many of the details of Sling (although I'd recommend looking at it because we do some cool stuff ), Sling is a OSGi web application framework which is primarily concerned with taking "resources" from a "repository" and transforming them to serve user requests. This transformation is generally (although not exclusively) done via scripts. We currently "ship" custom ESP and JSP implementations (albeit heavily based on Rhino and Jasper respectively) along with Groovy support (using the groovy jar). JRuby, Velocity, Scala, and Freemarker are all available as add-ons (I'm probably forgetting something). In short, we expect to be able to support any JSR-223 implementation, provided that the implementation is "OSGi-friendly".

Note - here I'm using "we" to refer to the Sling project. But to be clear, this request isn't being made on behalf of anyone but me.

In a Sling applications, scripts are not tied to a particular bundle - they are generally hosted in the repository (scripts are resources too!). The dependencies of these scripts aren't knowable at build time. When a script executes, it should have access to any exported package from any bundle in the OSGi framework (ignoring OSGi security...). Likewise, when a script executes, it shouldn't have access to any non-exported packages except for any internal classes used by the script engine (i.e. JRuby in this case).

Leaving aside the specifics of Sling for a minute, this seems like a pretty reasonable use case for scripting in general. You simply can't expect the "imports" of a bundle using scripting at build time to be defined. And even if they could be defined at build time (i.e. you had a set of scripts which were resources within a bundle), it's not like Bnd can figure out that your bundle needs to import "builtin.javasupport" (unless you pre-compile, which is a different story entirely).

For our ESP and JSP implementations, we have a custom classloader which, in essence, emulates the behavior of Dynamic-ImportPackage: *. For Groovy, we simply use the JSR-223 implementation provided by the Groovy JAR. I would like to be able to do the same thing with JRuby.

So... let's talk specifics. There are really two primary things I want to be able to do with JRuby in Sling:

1) I want to be able to write a basic script which just uses Ruby/JRuby builtins.
2) I want to be able to write a script which 'requires' one or more gems where those gems are being provided by a bundle.

Plus, one more general thing:

• I want to be able to write some Java code which runs a Ruby script which creates a new object which implements a Java interface where that interface is exported by a bundle. Subsequently, I want to be able to register that object in the OSGi service registry, but that has nothing to do with JRuby (in the future, some kind of DSL for this would be interesting, but that's not my concern right now).

AND... I want to be able to do these things without having JRuby installed on the server. In my prior testing (with 1.5.6), in order to run even the most basic of scripts, JRuby had to be installed and the jruby.home system property set. In 1.6.0.RC1, I can now run a basic script without having jruby.home set. Great progress. I also want to be able to do all of these through a JSR-223 interface. And I would definitely prefer if this was just doable with the jruby-complete JAR.

I expect all of these to be possible by adding Dynamic-ImportPackage: * to the manifest and I don't see any of them being possible any other way.

I have no argument with JRUBY-5384, I just don't think it fits my needs. I'm uninterested in reimplementing JRuby's JSR-223 implementation just so I can use OSGiContainer instead of ScriptingContainer. I would rather that the JRuby bundle's classloader behave the way I think a scripting container is expected to behave ("require-lazily anything-you-need-as-long-as-the-fully-qualified-java-name-is-written")

The notion inside a script to have
require 'osgibundle:/your.bundle.symbolic.name'

is interesting, but I think it's a bad practice to embed bundle names like this in code of any sort. A good part of the power of OSGi is that the provider of a package is flexible. That's why Import-Package should generally be preferred to Require-Bundle.

Finally, the notion of a fragment bundle is doable, but I really would rather jruby-complete had this itself. Again, call me lazy, but I'd prefer not to have to maintain this fragment.

I've posted a few integration tests to https://github.com/justinedelson/jruby-osgi-testing and will be adding more. I'm really happy that JRuby now doesn't require that it be installed to be used in an OSGi context.

It's been a while since I was on jruby-users, but I can rejoin if that would be a good context to have any further discussions along these lines.

Thanks for the detailed explanation of the usecase.
Apache Sling looks like a great project. Simplicity is really high in its goal and thanks for reviewing the proposed jruby.embed.osgi in this light.

Here are my comments:
> reimplementing JRuby's JSR-223 implementation just so I can use OSGiContainer instead of ScriptingContainer:
Understood: org.jruby.osgi.embed should be hidden inside JSR223 when used in OSGi (assuming we are adopting this approach): filed here JRUBY-5419

> require 'osgibundle:/your.bundle.symbolic.name'
> is interesting, but I think it's a bad practice to embed bundle names like this in code of any sort. A good part of the power of OSGi is that the provider of a package is flexible. That's why Import-Package should generally be preferred to Require-Bundle.

Agreed: in order of preference: Import-Package, Require-Bundle...
and way down the list Dynamic-ImportPackage to be avoided as much as possible is the consensus.
And if used (circular dependencies...) then avoid the '*' try to know what package will be dyanmically imported.
If there is sufficient interest we can certainly replicate the Import-Package in jruby and limit these to each OSGiScriptingContainer.
We should also support OSGi versions for this require directive:
require 'osgibundle:/your.bundle.symbolic.name;bundle-version=[1.0.0,2.0.0)'
Please note that the require synthax felt like a good choice to start with:

• not too shocking to ruby developers and indeed familiar to OSGi developers who know 'Require-Bundle'
• less difficult to support than a ruby's Import-Package

Still, the requirement is: support dynamic imports for a given scripting container running in OSGi.
I believe we could actually configure the OSGiScriptingContainer to support this behavior:
OSGiScriptingContainer extends the jruby's classloader we would add there the Dynamic-ImportPackage behavior.
It would be limited to this particular instance of the ruby runtime.
It feels like a nice solution to satisfy both those looking to be in control of what gets loaded and those of us who don't.
What do you think?

Sorry I just thought about a really quick solution:
Define a bundle with Dynamic-ImportPackage: *
When creating the scripting container that you use to execute the script:
((OSGiScriptingContainer)container).addToClassPath(theBundleWithDynamicImportPackage);
This will do the trick.
Not as clean as supporting a soft-Dynamic-ImportPackage in the OSGiScriptingContainer but it will do.

Assuming we work on JRUBY-5419 would this do what you need?

I'll have to think about this, but I do want to say that I vehemently disagree with this:
"Agreed: in order of preference: Import-Package, Require-Bundle...
and way down the list Dynamic-ImportPackage to be avoided as much as possible is the consensus."

I don't see how Require-Bundle could be preferable to Dynamic-ImportPackage. Can you point me to a reference for where you've seen this be the consensus?

I guess what I'm missing about this is what's the argument against adding DynamicImport-Package: * to the JRuby bundle? (not the argument about DynamicImport-Package in general) It doesn't have any impact on OSGiScriptingContainer, which allows users to have tighter control of the classpath. Nor would it obviate other custom classloaders from being passed into "regular" ScriptingContainer.

>I don't see how Require-Bundle could be preferable to Dynamic-ImportPackage. Can you point me to a reference for where you've seen this be the consensus?

Tommy explained the various drawbacks for org.jruby here:
http://jira.codehaus.org/browse/JRUBY-3792?focusedCommentId=185144

Here is Richard Hall (OSGi author):
http://stackoverflow.com/questions/3490661/osgi-is-it-possible-import-package-and-add-a-visibility-reexport/3498672#3498672

Here is a blog article out of many others:
http://blog-o-lok.blogspot.com/2008/01/osgi-and-dynamicimport-package.html

Thomas Watson (equinox): a bit of an old message:
http://dev.eclipse.org/mhonarc/lists/platform-core-dev/msg00717.html

All of these refer to scenarios where the list of imports is known at build-time. My point is that this isn't true in a general purpose dynamic scripting environment. Yes, DynamicImport-Package is an option of last resort, but it is IMHO necessary in this case.

I find it hard to believe Sling is the only place JRuby is used with user-created scripts. If you have a static set of scripts, then you can know the import list at build time and might not run into this problem.

Without some agreement I don't think we can move forward on this one. It sounds like there are strong points both for and against adding DynamicImport-Package: *, and we need guidance from OSGi folks on what's reasonable to do. Bumping to 1.6.1 for further discussion.

Example of bundle that setups a JRuby runtime where DynamicImport-Package: * is supported.
This example uses JSR223 to setup the ScriptingEngine and depends on JRUBY-5419's patch to work.
We could also use directly org.jruby.embed.osgi and then we don't need JRUBY-5419.

Here is the interesting bit in the testunit:

//note passing the org.jruby.jruby's classloader will enable
//the script engine manager to actually locate jruby's ScriptEngine
System.setProperty("org.jruby.embed.classloader", "none");
ScriptEngineManager manager = new ScriptEngineManager(
ScriptingContainer.class.getClassLoader());
ScriptEngine engine = manager.getEngineByName("jruby");
engine.eval("require 'osgibundle:/org.jruby.embed.osgi.jsr223.test'");
engine.eval("require 'java'");
//let's read a constant defined on the PackageAdmin object.
//we have not imported the PackageAdmin so this will in fact test the DynamicImport-Package: *
engine.eval("puts \"org.osgi.service.packageadmin.PackageAdmin.BUNDLE_TYPE_FRAGMENT = " +
"#

" +
"\"");

If maven 3 is installed on your machine, clone http://github.com/intalio/org.jruby.osgi/ and run mvn test to execute the testunits against 1.6.0.R2-patched.

As demonstrated in this thread, there are situations in which using DynamicImport-Package: * is necessary and other situations where there are better options.

If DynamicImport-Package: * is added to the manifest for the JRuby bundle, no one will have the option to not use it short of re-packaging JRuby.

On the other hand, the fragment bundle option is trivial to implement – all you need is an empty JAR with the following five-line manifest:

Bundle-ManifestVersion: 2
Bundle-Version: 0.0.0
Fragment-Host: org.jruby.jruby
Bundle-SymbolicName: org.jruby.jruby-dynamic-import-all
DynamicImport-Package: *