gened call to XMLFieldDescriptorImpl cnstr not using FQCN in some cases, perhaps from imported schema

Can you - for a second - turn off automatic naming collision resolution, and see whether the generated code compiles at all ?

It looks like setting either a namespace to package mapping (or a schemaLocation to package mapping) should do the trick. Have a look at the builder properties file you provided. There is no namespace to package mapping for the namespace http://wamu.com/wmcs/das/aos/accountProfile. Once you add that, you should be fine.

Adding that line to the castorbuilder.properties file got around this problem.

However, this seems like an odd way to resolve this problem.

First, it seems odd to require this in the common castorbuilder.properties file. I had the impression that file was used for global settings of how Castor operates on your project. This means that I'll have to have a list of every namespace and package mapping referenced in every "castor-codegen" call, although I have a feeling this is only required for certain situations, not all (and I don't know what those conditions are). If required, it seems more natural to provide this information in the castor-codegen call.

Second, this really needs some sort of diagnostic to detect that the mapping is required. Generating a compile error in generated code tells the developer nothing.

David, you can always use

SourceGenerator.setNamespacePackageMapping(String, String)

to set any namespace to package mapping 'programmatically' on the SourceGenerator instance you are using.

About the 'error message', let me get back to you with some more thoughts later, okay ?

How would I do that from the Ant task? I'm guessing that's not possible.

My suggestion would be to have a new property on the ant task called "mappings", which would point to a file containing additional mappings to use for this instance.

In a large project, the "castorbuilder.properties" file will be used at a global level. It's convenient to have a separate file at the subproject level, to specify mappings for related schemas. That's would be referenced in the "mappings" (or "nsmappings") property. This would either be a properties file, with one required single property, or a file assuming the syntax of the value of the current property.

Concerning the error message, I can see how hard it would be to detect this situation. Imagine how hard it is for the developer, once they see that compile error.

David, even when using Ant for triggering source code generation, there's the option to pass in custom.properties. In other words, the feature you are asking for is in place already. Unless you need two levels of builder properties ...

The problem is that property (according to the doc) overrides the global castorbuilder.properties file, it doesn't add to it. So, every subproject that needs namespace mappings is going to have to get a separate copy of the entire castorbuilder.properties file, with one property changed, and everything else identical.

I don't know what's worse, putting all the namespace bindings for all subprojects in the single castorbuilder.properties file, or cloning the file into every subproject with small changes. The latter is probably worse.

Your "unless" at the end of your last comment hits the nail on the head.

I'm going to attach a test case to this that demonstrates a similar problem, but I'm unable to fix this through appending to the "org.exolab.castor.builder.nspackages" property. I don't understand what I might be doing wrong.

After unpacking, put the latest 1.1.1 snapshot jars, along with commons-logging and xercesImpl into the "lib" directory, then do "ant anttask".

I see the following errors so far:

/home/u166705/work/castorbug2/gen/java/wamu/xmlbinding/consumer/descriptors/AddressTypeDescriptor.java:361: cannot resolve symbol
symbol : class AddressDetail
location: class wamu.xmlbinding.consumer.descriptors.AddressTypeDescriptor
desc = new org.exolab.castor.xml.util.XMLFieldDescriptorImpl(AddressDetail.class, "_addressDetail", "AddressDetail", org.exolab.castor.xml.NodeType.Element);
^
/home/u166705/work/castorbug2/gen/java/wamu/xmlbinding/consumer/descriptors/AddressTypeDescriptor.java:374: cannot resolve symbol
symbol: class AddressDetail
target.setAddressDetail( (AddressDetail) value);
^
/home/u166705/work/castorbug2/gen/java/wamu/xmlbinding/consumer/descriptors/AddressTypeDescriptor.java:380: cannot resolve symbol
symbol: class AddressDetail
return new AddressDetail();
^
3 errors

David, yes, you are right. In whatever way you use the (project/module local) property files, you'll end up overwriting previous namespace to package mappings (with the current code base). I honestly cannot (and would not be willing to give you a solution three days before a release), but I am more than willing to work with you on a long-term solution after 1.1.1 has been released.

In other words, would it be acceptable to you to stick with the work-around for the time being ?

Actually, are these schemas for which you are generating classes really completely unrelated ? 'Cause if not, iow if there's a common root schema, you might as well consider using the setGenerateImportedSchema(boolean) setter on the XML code generator.

David, we (Steven and I) just confirmed that Castor XML is capable of picking up namespace to package mappings incrementally from three places:

1) Default castorbuilder.properties (within the JAR).
2) Project-specific castorbuilder.properties (at the root of the classpath)
3) A custom property file as specified e.g. in the Ant config.

In other words, at the end of calling setDefaultProperties(), we end up with the closure of all namespace to package mappings as defined in all three files. As almost always, I forgot that the project-specific castorbuilder.properties file must be rooted at the classpath, and as such cannot live in a package.

I hope this clears things up a bit.

Let me try to address the last three comments:

Would it be acceptable? Well, the workaround doesn't appear to work for me, but as I obviously don't have a support contract, I guess it would be acceptable .

In the last test case I submitted, all of the schemas are related. I don't see the point of using the sGIS flag. It seems like just a convenience, not a strategy for avoiding the problem I'm having.

When I'm running the code generator in the test case I submitted, there is only one castorbuilder.properties file.

I think what would help me is if you were able to look at the second test case I submitted, and help me figure out how to adjust the properties file or the call to the ant task, to get the code to compile.

David, I'll be defintely having a look at the second test case later on tonight. As I am not really keen to step out of my usual work environment , ''d appreciate some of your additional help. How much of a problem would it be if you remodelled your test case to comply with the guidelines for submitting bug reports. The attached test case you submitted does not make this easy. I'd appreciate if you supplied me with a JUnit test case that enabled me to run this relatively to src/bugs.

So you mean a pure java JUnit test case that calls the SourceGenerator directly, instead of using the Ant task? I guess that's feasible. I might not be done with it by tonight.

Note that a JUnit test case is just going to generate the code, it's not going to compile it. Is there something that the JUnit test case can actually test for, as opposed to just calling SourceGeneratorMain 3 times?

I'm trying to match the interface to SourceGeneratorMain to what I'm sending to the ant task, and I don't understand how to map my "properties" attribute (the custom properties file).

Also, when I execute the "usage" for SourceGeneratorMain, it prints "-nameConflictStrategy " at the end, without an associated argument. I assume that's just a mistake in the usage message?

David, how about marshaling/unmarshaling a sample document. Having said that I know that generated classes have to be compiled first for this to work. But at least this will allow Werner to create a test that can be integrated with our test suite.

My issue is that the code that is generated does not compile. I can't build a test a test to demonstrate marshal/unmarshal because the code it would try to use does not compile.

How about if my JUnit test case calls CastorCodeGenTask instead of SourceGeneratorMain? If it's feasible to do that, I think that will work. It still won't be compiling the generated source, but if you know what to look for in the results, it will make it easier for you to step through your code.

Instead of calling SourceGeneratorMain, why not call SourceGenerator directly ? As that's exactly what SourceGeneratorMain does internally. Have a look at codegen/src/test/java/xml/generate/template for a sample test case.

Ok, attached is a new test case that includes a JUnit test class that directly executes CastorCodeGenTask. I inspected the code it generates, and it still has the problem.

The typical thing I check for in the generated code is whether the code in wamu/xmlbinding/consumer/descriptors/AddressTypeDescriptor.java references AddressDetail not as a FQCN.

Execute the JUnit test case and inspect the code in the wamu/xmlbinding/consumer/descriptors/AddressTypeDescriptor.java class. If it doesn't specify a FQCN for "AddressDetail", then it's still broken.

Werner, I just noticed your comment which came in after mine about the new test case. I assume it's still ok for my test case to execute CastorCodeGenTask directly, as opposed to SourceGenerator? It was easier for me to do it that way, since that maps directly to the test in my build.xml.

Should not be a problem ... as far as I can tell.

Allright, looks like I can now reproduce the problem. In other words, there's an AddressTypeDescriptor class that does not compile. Still have to get my head around as to why .....

Just looking into this a bit. It just occurred to me that there's two XML schemas that define an 'AddressType'' complex type, namely consumercrmtypes.xsd and crmtypes.xsd. And more surprisingly, these two schemas share the same target namespace. Is this design by intention ? I am not saying that I have found the problem, but somehow two schemas should not share the same target namespace, no ?

I think I had noticed that also. I didn't write these schemas, I'm just trying to build the application. I assume it's more likely a result of unplanned design, and merging of schemas from different areas. I'm not going to deny it's a mess, but I wouldn't expect that would cause this problem. If you can verify it is causing a problem, it would be good if a clear error condition could be created for this, so we know how to prevent it.

One more observation: you are using elementFormDefault="unqualified" by intention, right ? Why am I asking this ? If you were to use elementFormDefault="qualified", the targetNameSpace would be (at least programmatically) associated with the addressDetail element, and hence it would carry a package as per the namespace to package definition.

I just added an form="qualified" to the 'addressDetail' element within the complex type def for 'AddressType', and I don't get compilation errors any more.

David, it definitely has got nothing to do with the duplication of schemas as explained above. In this case, classes will be overwritten (whic migth cause different problems), but that's not the case here. It looks like the problem is with 'unqualification' of local elements.

Just so I'm clear on this, changing anything from "unqualified" to "qualified" will change what XML documents will validate or marshal/unmarshal with the schema, correct? If that's the case, that's not a practical workaround for this.

David, I do remember that there is another (quite recent) bug that might be related to your problem. Let me check whether I can find it, and see whether they talk about the same problem.

I think I now know as to why this ceased working when moving to Castor 1.1 M1. At that time, we started to separate the descriptor classes from the actual domain classes. With Castor 1.0.5, the code generated for the 'addressDetail' member does not have a package prefix either, but as the classes happen to be in the same package, this does not cause any harm. With our new layout (separate descriptor package, this of course does not work anymore).

Which still does not answer the question as to what the correct solution should be ... ;-(.

Here's a theory. If a local element definition happens to be unqualified, i.e. it has not targetNamespace assigned (as e.g. the global type definitions of the same schema), we have to find a way to still come up with a package this class should be generated into. As it could happen that there's a namespace to package mapping for the complete schema namespace, somehw I think that this is the place where the class should be generated as well.

Let's hope that this new logic does not break any additional code.

This is what I have in mind as a solution. I'd really appreciate feedback before committing myself to this.

David, are you in a position to build the binaries yourself (by any chances). Ideally, I'd love to provide you with a snapshot release, but somehow I am not really convinced to commit this to SVN trunk without some serious testing (on both sides).

I'm sure it's possible for me to do it, but I've never done it, and I have two projects that have to take priority over this. If you could point me to the step-by-step instructions for doing this, that would help.

Basically, we are talking about getting the latest sources from SVN, apply the patch and build the binaries yourself. If you are not really familiar with this and time is limited, I could try to upload a custom build to a selected area at Codehaus.

A possibly easier shortcut would be just dumping the source tree to a zip file and putting it on an ftp site. I can download that, apply the patch, and build it.

http://svn.castor.codehaus.org/browse/~tarball=zip/castor/castor/trunk/trunk.zip has a ZIP file of SVN trunk.

I am proceeding with this. I've integrated the patch and built it, and now I'm stepping through the code. It didn't appear to work on the first try (no change in the result), so I'm making sure it's actually getting to the code you wrote.

For a limited test case, I think I've resolved the issues with the generated package name. However, now I'm back to getting errors like this:

[javac] Compiling 2420 source files to C:\p4ws\WS_u166705_AIS_2\EAI_weblogic\uia2_0707\subprojects\xmlbinding\target\classes
/c/p4ws/WS_u166705_AIS_2/EAI_weblogic/uia2_0707/subprojects/xmlbinding/target/castor/src/wamu/xmlbinding/loan/BorrowerType.java:97: unmarshal(java.io.Reader) in wamu.xmlbinding.loan.BorrowerType cannot override unmarshal(java.io.Reader) in wamu.xmlbinding.consumer.CustomerType; attempting to use incompatible return type
found : wamu.xmlbinding.loan.BorrowerType
required: wamu.xmlbinding.consumer.CustomerType
public static wamu.xmlbinding.loan.BorrowerType unmarshal(
^
/c/p4ws/WS_u166705_AIS_2/EAI_weblogic/uia2_0707/subprojects/xmlbinding/target/castor/src/wamu/xmlbinding/loan/Borrower.java:97: unmarshal(java.io.Reader) in wamu.xmlbinding.loan.Borrower cannot override unmarshal(java.io.Reader) in wamu.xmlbinding.consumer.CustomerType; attempting to use incompatible return type
found : wamu.xmlbinding.loan.BorrowerType
required: wamu.xmlbinding.consumer.CustomerType
public static wamu.xmlbinding.loan.BorrowerType unmarshal(
^
/c/p4ws/WS_u166705_AIS_2/EAI_weblogic/uia2_0707/subprojects/xmlbinding/target/castor/src/wamu/xmlbinding/loan/Debtor.java:97: unmarshal(java.io.Reader) in wamu.xmlbinding.loan.Debtor cannot override unmarshal(java.io.Reader) in wamu.xmlbinding.loan.BorrowerType; attempting to use incompatible return type
found : wamu.xmlbinding.consumer.CustomerType
required: wamu.xmlbinding.loan.BorrowerType
public static wamu.xmlbinding.consumer.CustomerType unmarshal(
^
/c/p4ws/WS_u166705_AIS_2/EAI_weblogic/uia2_0707/subprojects/xmlbinding/target/castor/src/wamu/xmlbinding/loan/LoanListTypeCoBorrower.java:97: unmarshal(java.io.Reader) in wamu.xmlbinding.loan.LoanListTypeCoBorrower cannot override unmarshal(java.io.Reader) in wamu.xmlbinding.loan.BorrowerType; attempting to use incompatible return type
found : wamu.xmlbinding.consumer.CustomerType
required: wamu.xmlbinding.loan.BorrowerType
public static wamu.xmlbinding.consumer.CustomerType unmarshal(

Which makes me wonder whether this is a result of the patch (like e.g. introducing a side effect), or whether these are genuine errors that you are seeing the first time. Having said that, can you please create a new issue and attach all relevant files, incl. XML schema(s), test case, etc.

Please note that I still have to commit the suggested patch.

I'm guessing this patch hasn't been applied yet? I tried using the latest 1.1.2 snapshot (5/6/07) and testing with it, and I realized it doesn't have the patch yet.

David, that's actually correct. I was waiting for feedback from another party, and I am now in a position to securely commit this patch with additional regression testing being performed for me .. ;.-).

Having said that, this will have to wait for a few days, as I am spending nearly all my time in hospital after the birth of my daughter two days ago. Please bear with me for a few days .,,.

Ah, well, then take as long as you want, then. No hurry. If you just notified me somehow when a new snapshot with this patch is available, that would be fine with me. Have fun.

I have just attached a patch to CASTOR-1986 which I believe is superior than the one attached here. David, would you please be in a position to test the patch attached to CASTOR-1986 with this problem ? I'd appreciate any feedback, to be honest.

I might have time to try this in a couple of days. However, I still don't have the capability to retrieve directly from SVN, so would you mind building a "trunk.zip", like you did before?

David, http://svn.castor.codehaus.org/browse/~tarball=zip/castor/castor/trunk/trunk.zip should give you what you are looking for. Please note that the patch for CASTOR-1968 has not been committed to SVN trunk yet, and thus needs to be applied manually.

I haven't gotten to the point of executing the compiled and generated classes, but all of my code generates and compiles without error now. I still think it's awkward to have to know to add namespace mapping entries when obscure compile errors happen. This isn't an issue when you're doing Castor work pretty often, but it sure causes problems when you don't know how to deal with it in the first place.

Looking into CASTOR-1864 again, it looks like CASTOR-1934 will have to be committed as well, as the problem Mattthias is encountering is not related to including/importing XML schemas at all. Having said that, I appreciate your patience, and I still believe that it should be possible to avoid those unnecessary namespace to package mappings once I have committed CASTOR-1986 and CASTOR-1934. Let's see how things go ...

As I am about to commit the latest patch to this issue, I wonder whether the test case could be broken down to something more minimal, as I'd like to enhance the CTF test suite.

Sorry, I can only follow all the Castor development sproadically. Today, I tried again to run the castor generation on my testbed.
This patch is still needed, otherwise it still generates classes from imported schema.
So - in short - the facts didn't change in the last three months.

Correction: the patch above does not work for Java enums. They are also generated in "package types;"
Only old-style enums seem to be put into the package associated with the schema target namespace.

Correction again: even without the patch it seems to work with old-style enums so probably fixing CASTOR-1986 helped here.
This patch seems to be outdated because it makes things worse for my other issue CASTOR-1881.
Nevertheless, the real Java enums still have this problem.