castor

problems with marshaling/unmarshaling when using XML schema type hexBinary

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 0.9.4.3, 1.1
  • Fix Version/s: 1.1.1
  • Component/s: XML
  • Labels:
    None
  • Environment:
    Operating System: Windows 2000
    Platform: PC
  • Bugzilla Id:
    1280
  • Number of attachments :
    3

Description

I used the castor code generator version 0.9.3.4 on an XML schema file which
contains the following type and element definition:

<xsd:simpleType name="IVRData">
<xsd:restriction base="xsd:hexBinary">
<xsd:minLength value="0"/>
<xsd:maxLength value="20"/>
</xsd:restriction>
</xsd:simpleType>

<xsd:element name="callData">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="ivrData" type="adm-ctx:IVRData"
minOccurs="0"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>

When marshaling an XML file by using Castor, I entered the following Java code:

byte [] ivrData = {0x1, 0x4A};
CallData callData = new CallData();
callData.setIvrData(ivrData);

But the code generator produced the following XML file part:

<callData>
<ivrData>AUo=</ivrData>
</callData>

This XML file is definitely not schema conforming. I exptected an XML element
like the following:

<ivrData>014A</ivrData>

Vice versa, the Unmarshaller reads the wrong string 'AUo=' without throwing an
exception and transforms it to a byte array which does not have anything to do
with the original byte array. Additionally, when reading a valid hexBinary
string, the byte array read is totally different from the original hexBinary
string.
---------------------------------------------------------
Another comment/proposal:
In Java castor uses a byte array. But the allowed range of values from 0 to 255
is transferred to a signed byte value. In order to extract the correct hex
values from the byte, I have to know about the Java internal representation of
signs. Wouldn´t have it be better to define an 'array of short' and use allowed
values from 0 to 255 here?

  1. castor-423.patch
    28/Feb/07 8:58 AM
    20 kB
    johan lindquist
  2. patch.c423.20070401.txt
    01/Apr/07 7:19 AM
    23 kB
    Werner Guttmann
  3. patch.c423.20070401-002.txt
    01/Apr/07 7:55 AM
    21 kB
    Werner Guttmann

Activity

Hide
Keith Visco added a comment -

Currently only base64 encoding is supporting. Accepting bug.

Show
Keith Visco added a comment - Currently only base64 encoding is supporting. Accepting bug.
Hide
johan lindquist added a comment -

I am experiencing the same problem and was wondering if there is anything I can do temporarily. Looking at the code, it looks like there is actually no way of figuring out the type (schema type) of the element when it is being serialized - if someone could give me a hint of where this can be found out, I may be able to provide a patch ... Btw, we are using 1.0.5 and it looks like the problem is still there ...

Show
johan lindquist added a comment - I am experiencing the same problem and was wondering if there is anything I can do temporarily. Looking at the code, it looks like there is actually no way of figuring out the type (schema type) of the element when it is being serialized - if someone could give me a hint of where this can be found out, I may be able to provide a patch ... Btw, we are using 1.0.5 and it looks like the problem is still there ...
Hide
Werner Guttmann added a comment -

Johan, why don't you provide us with a patch that contains e.g. some pseudo code, showing us what you'd like to do ? This way, we'd be able to easily identify the code area in question, and b) hopefully provide you with an answer to your original question.

Show
Werner Guttmann added a comment - Johan, why don't you provide us with a patch that contains e.g. some pseudo code, showing us what you'd like to do ? This way, we'd be able to easily identify the code area in question, and b) hopefully provide you with an answer to your original question.
Hide
johan lindquist added a comment -

Ok, having dug around somewhat, it looks like there are two problems:

1) The code generator does not propagate the schema type into the generated classes. This makes it hard to figure out in the marshaller which type the element/attribute being processed is. I believe that by adding the following in the DescriptorSourceFactory (line 396), it would allow this:

jsc.add("desc.setSchemaType(\""+ xsType.getName() + "\");");

Now, this may not be the ideal way/correct way of course?

2) In the Marshalling, the types are currently not referenced, so whenever there is a byte array being marshalled, it is assumed to be of type base64. So, something like the following (assuming that 1) has been done) would provide hexBinary support quite quickly:

// For attribute processing
final String schemaType = attDescriptor.getSchemaType();
if ("hexBinary".equals(schemaType))
{
value = new String(Hex.encode((byte[]) value));
}
else
{
value = new String(Base64Encoder.encode((byte[]) value));
}

// For element processing
String schemaType = descriptor.getSchemaType();
if ("hexBinary".equals(schemaType))
{
chars = new String(Hex.encode((byte[]) obj)).toCharArray();
}
else
{
chars = Base64Encoder.encode((byte[]) obj);
}

Something like that at least ...

Btw, if an attribute is defined as hexBinary, the resulting value will not work as the value being put into the XML is simply the toString() value of the byte array.

I hope that this is of some help.

Thanks,

Johan

Show
johan lindquist added a comment - Ok, having dug around somewhat, it looks like there are two problems: 1) The code generator does not propagate the schema type into the generated classes. This makes it hard to figure out in the marshaller which type the element/attribute being processed is. I believe that by adding the following in the DescriptorSourceFactory (line 396), it would allow this: jsc.add("desc.setSchemaType(\""+ xsType.getName() + "\");"); Now, this may not be the ideal way/correct way of course? 2) In the Marshalling, the types are currently not referenced, so whenever there is a byte array being marshalled, it is assumed to be of type base64. So, something like the following (assuming that 1) has been done) would provide hexBinary support quite quickly: // For attribute processing final String schemaType = attDescriptor.getSchemaType(); if ("hexBinary".equals(schemaType)) { value = new String(Hex.encode((byte[]) value)); } else { value = new String(Base64Encoder.encode((byte[]) value)); } // For element processing String schemaType = descriptor.getSchemaType(); if ("hexBinary".equals(schemaType)) { chars = new String(Hex.encode((byte[]) obj)).toCharArray(); } else { chars = Base64Encoder.encode((byte[]) obj); } Something like that at least ... Btw, if an attribute is defined as hexBinary, the resulting value will not work as the value being put into the XML is simply the toString() value of the byte array. I hope that this is of some help. Thanks, Johan
Hide
johan lindquist added a comment -

Oh, I should add that you need to replace the Hex encoder above with your favorite implementation - the one above is from BouncyCastle.

Cheers,

Johan

Show
johan lindquist added a comment - Oh, I should add that you need to replace the Hex encoder above with your favorite implementation - the one above is from BouncyCastle. Cheers, Johan
Hide
johan lindquist added a comment -

And another one - the Unmarshaller needs similar changes to make sure it works in both directions. If needed, I can provide a patch

Show
johan lindquist added a comment - And another one - the Unmarshaller needs similar changes to make sure it works in both directions. If needed, I can provide a patch
Hide
Matthew added a comment -

As noted by Johan above, hexBinary attributes are really broken. I have an attribute defined like:
<xsd:attribute name="info" use="optional">
<xsd:simpleType>
<xsd:restriction base="xsd:hexBinary"/>
</xsd:simpleType>
</xsd:attribute>

When I get an XML file that has this attribute and it tries to convert it to an object I get this:

at com.avaya.proxy.castor.CastorUnmarshaller.unmarshal(CastorUnmarshaller.java:136)

at com.avaya.mvcs.proxy.CstaUnmarshallerNode$CstaUnmarshallerProcessorThread.run(CstaUnmarshallerNode.java:148)

at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)

at java.lang.Thread.run(Thread.java:595)

Caused by: unable to add attribute "info" to 'MyObject' due to the following error: java.lang.IllegalStateException: java.lang.ClassCastException: java.lang.String{file: [not available]; line: 2; column: 281}

at org.exolab.castor.xml.Unmarshaller.unmarshal(Unmarshaller.java:716)

at org.exolab.castor.xml.Unmarshaller.unmarshal(Unmarshaller.java:572)

at com.avaya.proxy.castor.CastorUnmarshaller.unmarshal(CastorUnmarshaller.java:127)

... 4 more

Caused by: org.xml.sax.SAXException: unable to add attribute "info" to 'MyObject' due to the following error: java.lang.IllegalStateException: java.lang.ClassCastException: java.lang.String

at org.exolab.castor.xml.UnmarshalHandler.processAttributes(UnmarshalHandler.java:2724)

at org.exolab.castor.xml.UnmarshalHandler.startElement(UnmarshalHandler.java:2337)

at org.exolab.castor.xml.UnmarshalHandler.startElement(UnmarshalHandler.java:1387)

at org.apache.xerces.parsers.SAXParser.startElement(SAXParser.java:1376)

at org.apache.xerces.validators.common.XMLValidator.callStartElement(XMLValidator.java:1197)

at org.apache.xerces.framework.XMLDocumentScanner.scanElement(XMLDocumentScanner.java:1862)

at org.apache.xerces.framework.XMLDocumentScanner$ContentDispatcher.dispatch(XMLDocumentScanner.java:1238)

at org.apache.xerces.framework.XMLDocumentScanner.parseSome(XMLDocumentScanner.java:381)

at org.apache.xerces.framework.XMLParser.parse(XMLParser.java:1035)

at org.exolab.castor.xml.Unmarshaller.unmarshal(Unmarshaller.java:702)

... 6 more

2007-02-21 14.54.46,291 com.avaya.mvcs.proxy.CstaMarshallerNode$CstaMarshallerThread convertException

Show
Matthew added a comment - As noted by Johan above, hexBinary attributes are really broken. I have an attribute defined like: <xsd:attribute name="info" use="optional"> <xsd:simpleType> <xsd:restriction base="xsd:hexBinary"/> </xsd:simpleType> </xsd:attribute> When I get an XML file that has this attribute and it tries to convert it to an object I get this: at com.avaya.proxy.castor.CastorUnmarshaller.unmarshal(CastorUnmarshaller.java:136) at com.avaya.mvcs.proxy.CstaUnmarshallerNode$CstaUnmarshallerProcessorThread.run(CstaUnmarshallerNode.java:148) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675) at java.lang.Thread.run(Thread.java:595) Caused by: unable to add attribute "info" to 'MyObject' due to the following error: java.lang.IllegalStateException: java.lang.ClassCastException: java.lang.String{file: [not available]; line: 2; column: 281} at org.exolab.castor.xml.Unmarshaller.unmarshal(Unmarshaller.java:716) at org.exolab.castor.xml.Unmarshaller.unmarshal(Unmarshaller.java:572) at com.avaya.proxy.castor.CastorUnmarshaller.unmarshal(CastorUnmarshaller.java:127) ... 4 more Caused by: org.xml.sax.SAXException: unable to add attribute "info" to 'MyObject' due to the following error: java.lang.IllegalStateException: java.lang.ClassCastException: java.lang.String at org.exolab.castor.xml.UnmarshalHandler.processAttributes(UnmarshalHandler.java:2724) at org.exolab.castor.xml.UnmarshalHandler.startElement(UnmarshalHandler.java:2337) at org.exolab.castor.xml.UnmarshalHandler.startElement(UnmarshalHandler.java:1387) at org.apache.xerces.parsers.SAXParser.startElement(SAXParser.java:1376) at org.apache.xerces.validators.common.XMLValidator.callStartElement(XMLValidator.java:1197) at org.apache.xerces.framework.XMLDocumentScanner.scanElement(XMLDocumentScanner.java:1862) at org.apache.xerces.framework.XMLDocumentScanner$ContentDispatcher.dispatch(XMLDocumentScanner.java:1238) at org.apache.xerces.framework.XMLDocumentScanner.parseSome(XMLDocumentScanner.java:381) at org.apache.xerces.framework.XMLParser.parse(XMLParser.java:1035) at org.exolab.castor.xml.Unmarshaller.unmarshal(Unmarshaller.java:702) ... 6 more 2007-02-21 14.54.46,291 com.avaya.mvcs.proxy.CstaMarshallerNode$CstaMarshallerThread convertException
Hide
johan lindquist added a comment -

I have now made all the changes required to support both Base64 and hexBinary - on Monday I'll be able to provide a patch for this - what do people prefer, a patch for 1.0.5 or for the trunk? If anyone has a suggestion on how to add a test case for this, it would be much appreciated.

Thanks,

Johan

Show
johan lindquist added a comment - I have now made all the changes required to support both Base64 and hexBinary - on Monday I'll be able to provide a patch for this - what do people prefer, a patch for 1.0.5 or for the trunk? If anyone has a suggestion on how to add a test case for this, it would be much appreciated. Thanks, Johan
Hide
Ralf Joachim added a comment -

Hi Johan, great that you managed to fix this problem and even better if you provide us with a patch. We'd prefer if you could create the patch against trunc.

At http://castor.codehaus.org/how-to-submit-an-xml-bug.html there is a description on howto create a test using the template we provided. If you provide us with such simple junit tests we convert and integrate them with our test suite. You don't need to care on automatic code generation from your schema as this is easy within the test suite if we get a schema from you. To test that things are working as expected I think you could marshal a test object and compare the output of Marshaller with your expected xml. By unmarshalling a xml and testing the properties of unmarshalled object you could verify that Unmarshaller works as expected.

Hope that helps

Show
Ralf Joachim added a comment - Hi Johan, great that you managed to fix this problem and even better if you provide us with a patch. We'd prefer if you could create the patch against trunc. At http://castor.codehaus.org/how-to-submit-an-xml-bug.html there is a description on howto create a test using the template we provided. If you provide us with such simple junit tests we convert and integrate them with our test suite. You don't need to care on automatic code generation from your schema as this is easy within the test suite if we get a schema from you. To test that things are working as expected I think you could marshal a test object and compare the output of Marshaller with your expected xml. By unmarshalling a xml and testing the properties of unmarshalled object you could verify that Unmarshaller works as expected. Hope that helps
Hide
johan lindquist added a comment -

Hi Ralf,

Thanks for that - will add a test case before I submit it.

Johan

Show
johan lindquist added a comment - Hi Ralf, Thanks for that - will add a test case before I submit it. Johan
Hide
johan lindquist added a comment -

The attached patch attempts to fix support for hexBinary/base64Binary.

Show
johan lindquist added a comment - The attached patch attempts to fix support for hexBinary/base64Binary.
Hide
johan lindquist added a comment -

I should add that the patch will use base64binary unless you explicitly set hexBinary in a mapping file or schema.

Cheers,

Johan

Show
johan lindquist added a comment - I should add that the patch will use base64binary unless you explicitly set hexBinary in a mapping file or schema. Cheers, Johan
Hide
Werner Guttmann added a comment -

Original patch updated against SVN trunk.

Show
Werner Guttmann added a comment - Original patch updated against SVN trunk.
Hide
Werner Guttmann added a comment -

Improved and final patch, removing dependency on source generator from XML data binding classes.

Show
Werner Guttmann added a comment - Improved and final patch, removing dependency on source generator from XML data binding classes.
Hide
Werner Guttmann added a comment -

Committed as is ...

Show
Werner Guttmann added a comment - Committed as is ...
Hide
Werner Guttmann added a comment -

Wolfgang, for the byte vs. short array, please feel free to create a follow-up issue.

Show
Werner Guttmann added a comment - Wolfgang, for the byte vs. short array, please feel free to create a follow-up issue.
Hide
Dave Wilmes added a comment -

I noticed that when marshaling hexBinary elements, the resulting XML output consists of a lower-case representation of hex digits [a-f]. Although this is allowed according to the W3C XML schema recommendation, it appears that upper-case [A-F] would be a better choice as it is specified in the canonical representation:
http://www.w3.org/TR/2001/REC-xmlschema-2-20010502/#hexBinary

Show
Dave Wilmes added a comment - I noticed that when marshaling hexBinary elements, the resulting XML output consists of a lower-case representation of hex digits [a-f]. Although this is allowed according to the W3C XML schema recommendation, it appears that upper-case [A-F] would be a better choice as it is specified in the canonical representation: http://www.w3.org/TR/2001/REC-xmlschema-2-20010502/#hexBinary
Hide
Ralf Joachim added a comment -

Dave, could you please create a new issue for changing hex digits to upper case and if possible to you link it to this one?

Show
Ralf Joachim added a comment - Dave, could you please create a new issue for changing hex digits to upper case and if possible to you link it to this one?
Hide
Dave Wilmes added a comment -
Show
Dave Wilmes added a comment - New issue submitted: http://jira.codehaus.org/browse/CASTOR-2174

People

Vote (3)
Watch (3)

Dates

  • Created:
    Updated:
    Resolved: