I finally managed to create an isolated patch for this issue against current castor trunk.
I think it will be easier to talk about the details now, so I want to explain what this patch is about:
This patch is actually just a structural cleanup. While starting to think about the jaxb2 annotation integration, I tried to discover where to put the hooks for this task. IMHO the best places for this are where the Class/FieldInfos are translated into JClass/JFields. There we discovered some inconsistency, because ClassInfo, GroupInfo etc. are simple POJO style containers with the exception of the FieldInfo classes. So we decided to move the creation code to factory classes (like the SourceFactory for JClasses).