Java 5 compatible code generatedfrom Castor xml

David, I'll come back with feedback somewhen tomorrow ....

David, I had a first look, and what you provided definitely makes a good starting point (minus a few 'hacks' that need to be refactored). What I am missing and thus like to add is the ability to generate typed collection based upon the type of the underlying content type.

This is a nice start using what we have currently. My plan is to add Velocity templating for code generation which will (hopefully) allow customization like this in a much easier manner. Keith and i still need to discuss it further. But thanks very much for the patch, David! Your contributions are much appreciated .

Werner, Bruce
I agree with both your points.

My patch represents a "hack" for get rid of 10,833 compiler warnings coming from castor xml generated classe in our main project and clogging up the Problems tab in Eclispe. We could NOT see our own errors/warnings in the code we were currently developing because of the warnings coming from the castor generated classes. So I hacked your generator to removed the warnings. Thats all. This is by no means a fully baked solution.

My intent was to give you guys a head start by showing you all the places where I found Java 5 related issues with your code generation engine.

FWIW: I do think a good refactoring is in order, I got the feeling while looking through the code that there were several "cooks" in there writing similar/competing code to do the same kinds of things.

The list of things my hack does NOT provide:
1. correctly parameterized collections for the exact class for superclass instead of using <Object> everywhere.
2. A way to toggling between generating 1.4 and 5.0 code, my hack does only Java 5.0
3. Elegance, it won't win and coding awards and it NOT my finest

Bruce, whilst I agree that a general refactoring is worth the time, I don't agree that users have to wait month and month for simple (albeit useful) patches. I think that the three or four hours spent by myself (and/or yourself, if you are willing to join me on this) is a good investment, especially as David's contribution is clearly identifying all the areas that need to be changed. Take it like this, I have specialised in Castor JDO only for 2.5 yrs now, and still I find it easy to integrate David's patch after looking at the source generator code twice now (in total no more than maybe 5 or six hours). That sort of implies that the code is structured nicely, though it could use some refactoring with regards to code duplication and separation of concerns.

David, whilst you already provided a patch that clearly identifies the areas of change, I'll try to provide points 1) and 2) as mentioned above. Let's see how much time I will be able to spend some time on whilst working on the implementation of the issues, but in general we refactor classes that we need to touch anyhow (albeit in the most minimal way). As far as I can tell already now, I will have to refactor JType, JMethod, JParameter and JField to get 1) implemented. Especially JField is not in a good state, as it already delas with 'vanilla' Java types and arrays at the same time. When introducing the component type for collections, we could reuse what's there, but introducing even more ifs does not seem the righ way to move forward.

Given the fact that you are using this feature already, I guess I will come back to you when it comes down to thourough testing ....

Werner,

Testing, sure thing. We are currently in a dev cycle until mid-august but we are ahead of schedule so I could peel off to take the new code out for a test drive.

When I said refactoring, I did NOT mean completely re-write the code although that is your choice to do.

I meant what you already noticed, areas in the code like JField that would not handle the typed collections nor annotations very easily.

Areas like that in the code base are a big reason why my hack only types to the <Object> collections, I could not see easily how to go any further across the board without taken more time than I had, and I accomplished what I had to get done, which was to get the warnings cleaned up. So i stopped.

Everyone, enjoy the 4th, i'll be back on the 5th

> Bruce, whilst I agree that a general refactoring is worth the time, I don't agree that users have to wait month and month for simple (albeit useful)
> patches. I think that the three or four hours spent by myself (and/or yourself, if you are willing to join me on this) is a good investment, especially as
> David's contribution is clearly identifying all the areas that need to be changed. Take it like this, I have specialised in Castor JDO only for 2.5 yrs now,
> and still I find it easy to integrate David's patch after looking at the source generator code twice now (in total no more than maybe 5 or six hours). That
> sort of implies that the code is structured nicely, though it could use some refactoring with regards to code duplication and separation of concerns.

Yes, I absolutely agree that David's patch is a good. I'm very happy to have the contribution to the project. I was simply tossing out the fact that we will be adding templating to the code generation so that David might think about offering some ideas to toward that effort. I never said or implied anything about waiting. After glancing through it, I think we should commit David's patch as long as it doesn't break any tests.

Making some (albeit slow progress) for J1 collections. For simple types, I have got the field member of a class appearing as

private java.util.Vector<java.lang.String> _member;

already. I still need to have a look at some od the gernerated methods, but that should not be too hard .. . From there on, it's about looking at J2 collections.

Improved patch, with additional modificatios/additions to JType, CollectionInfo, etc. Added a small JUnit test case to src/bugs/xml/c1470 as well.

Thanks, Bruce, for "glancing through" David's patch. I was actually hoping that I might see some more contribution now from you , as you are planning to refactor this code area and others. To be honest, I just cannot see things happening that way: right now, there's more or less no contribution from you. Whilst I appreciate your long-term goal to implement the JAXB 2.0 specs (and thus refactor Castor XML code to some degree), I think that your contribution is needed now. Ralf and I are working on prerparations for the JPA 3.0 compatibility layer, and as part of this we are currently refactoring various Castor XML bits (mapping loaders, class descriptor resolvers, etc.) Can you imagine that we'd be able to turn this over in a far more efficient way if you joined us now (rather than always keep referring to a future phase) ?

Updated patch, that integrates the newly expanded types with various other J* classes, including JField and JMtehodSignature.

Small updates to the JUnit test case to test genertion for "j1" and "j2" type modes sequentially.

> Thanks, Bruce, for "glancing through" David's patch. I was actually hoping that I might see some more contribution now from you , as you are planning
> to refactor this code area and others. To be honest, I just cannot see things happening that way: right now, there's more or less no contribution from
> you. Whilst I appreciate your long-term goal to implement the JAXB 2.0 specs (and thus refactor Castor XML code to some degree), I think that your
> contribution is needed now. Ralf and I are working on prerparations for the JPA 3.0 compatibility layer, and as part of this we are currently refactoring
> various Castor XML bits (mapping loaders, class descriptor resolvers, etc.) Can you imagine that we'd be able to turn this over in a far more efficient
> way if you joined us now (rather than always keep referring to a future phase) ?

I'm trying to dedicate more time to it, but it's very difficult with my day job typically requiring 60+ hours per week. What would really help me to get more involved again is for the discussions about the work that's being done to happen on the dev@castor list. Outside of just watching commits flow into Subversion, there is little discussion about the work that's happening. Having it take place in the open would be a very good thing for the project as a whole.

Bruce, I have got no idea what you trying to say. Out of the total (project-specific) communication these days, ...

1) about 40% happen through jira bugs.
2) 25% on the user mailing list
3) 25% throug IRC (#castor at irc.codehaus.org) (which is open for public review, isn't it).

As far as I remember, you have been invited to a little get-together (that actually happened 10 days ago) weeks in advance (promised to have a look whether you could do anything about it, etc.). In other words, that openess you are asking for is happening already. It's a matter of attaching rather than demanding actions ot be taken.

Wrt 60+ hours, it woul dbe interesting to know what makes you think that your situation is different from anybody else ? We all have a day job, and we all have to find a way around life and work and a healthy balance. But even if sometimes I cannot afford more time to dedicate to this open source project (to e.g. fix even more bugs), at least I (and others) engage to spend time with our users and their needs.

Updated patch, adding code to CollectionInfo and CollectionInfo2 to deal with typed collections in all required places. In addition, introduced code to expose BuilderConfiguration as a singleton (though I don't like this).

David, I'd appreciate any fedback with regards to code quality, correctness, usability, etc. Please have a look at the castorbuilder.properties in src/bugs/xml/c1470 in particular for the definition of the new property required to trigger Java 5 compliance.

Questions:

a) are you using a binding file in your applications to e.g. modify the collection type of a property ?
b) Are you fine about the use of the new property ?
c) why did you modidy the J2Collectionhandler.toString() methds ?

Added code to FieldInfoFactory (and CollectionInfo2 et alias) to - for the first time - take into account if the user - in a binding file - specified an other collection type than the default value (arraylist).

> Bruce, I have got no idea what you trying to say. Out of the total (project-specific) communication these days, ...

> 1) about 40% happen through jira bugs.

I have to admit that keeping up with all of them is rather difficult.

> 2) 25% on the user mailing list

This is good, but I'm curious to know why there is so little traffic on the dev@castor list?

> 3) 25% throug IRC (#castor at irc.codehaus.org) (which is open for public review, isn't it).

And there's the answer to my question above . I guess I need to read through the IRC logs. But therein lies something that most users will never do. If decicisions are being made via IRC it would be really great if the results of the discussions could be brought the mailing list to help inform the community better.

> As far as I remember, you have been invited to a little get-together (that actually happened 10 days ago) weeks in advance (promised to have a look
> whether you could do anything about it, etc.). In other words, that openess you are asking for is happening already.

My apologies for not following up to say I couldn't make it. I had my wife with me while I was speaking at TheServerSide Java Symposium in Spain. Was it a productive time? What was discussed? What are the outcomes?

> It's a matter of attaching rather than demanding actions ot be taken.

I'm not sure why you think I'm making demands - I simply was inquiring about the openness of the discussions. Thank you for informing me.

> Wrt 60+ hours, it woul dbe interesting to know what makes you think that your situation is different from anybody else ? We all have a day job, and
> we all have to find a way around life and work and a healthy balance. But even if sometimes I cannot afford more time to dedicate to this open
> source project (to e.g. fix even more bugs), at least I (and others) engage to spend time with our users and their needs.

I never claimed that my situation was different. I'm sure everyone is dealing with something similar to this as you and I discussed when you first joined Castor. I'm simply trying to provide some context to my crazy and what seems to be constantly overloaded life these days, that's all.

Werner,

a,b) I do not have a directory for src/bugs/xml/c1470, I am looking at the trunk, is that the right branch on the code base.

I have including a snippett of the Java code we use to configure the your code generator within our code generator

NOTE: the generator is constructed and called for every DataObject in our system.

org.exolab.castor.builder.SourceGenerator sg = new org.exolab.castor.builder.SourceGenerator();
Properties props = new Properties();
props.setProperty(org.exolab.castor.builder.BuilderConfiguration.Property.SUPER_CLASS,"org.openmrm.core.data.castor.XDataObject");

sg.setDefaultProperties(props);

sg.processNamespacePackageMappings(packageMaps);
sg.setGenerateImportedSchemas(false);
sg.setPrimitiveWrapper(true);
sg.setEqualsMethod(true);
sg.setSuppressNonFatalWarnings(true);
sg.setDestDir(mrm_output_dir + this.file_separator + "java" + this.file_separator + "src");

sg.generateSource(fileInfo[0],fileInfo[1] + ".castor");

c) For completeness, I cannot tell you if the toString changes had any impact on the code generation. I made multiple changes on each pass through the code, thus I didn't isolate those changes for correctness. I you don't think they are needed, fine by me to ignore them.

Thanks,
DaVe.

David, I was actually referring to the test case included in the patch posted by me. This patch includes a JUnit test case, a sample XML Schema, a binding file and castorbuilder.properties. In this properties file you'll find a reference to the new property to be used to turn on Java 5 code generation. Sorry for not making this clearer initially .... ;-(.

Wrt c), if that's the case, I'd rather omit these changes ....

Having said that, have you actually been able to run your code generaton routines against a fresh checkout off trunk with the latest patch applied ?

Ooops, just vcame to realize that there's really no castorbuilder.properties file in src/bugs/xml/c1470 .....

Complete sample code relativ to src/bugs.

Final patch (hopefully) .. . Problem is that four test cases fail of the normal CTF test suite.

For whatever reason, both the master test suite as well as the regression test suite finish successfully now. Not that I am sure, bug it looks like I might have been in the wrong project directory when executing the tests ....;-(.

As such, I'll commti the patch as is, and will ask everybody to open new issues as a result of this patch. In addition, there's quite some refactorings needed to clean up some existing code (that might never have ben used before).