groovy
  1. groovy
  2. GROOVY-4851

NullPointerException on iterator() call

    Details

    • Type: Bug Bug
    • Status: Open Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 1.7.10, 1.8-rc-3, 1.8-rc-4, 1.8.0
    • Fix Version/s: 3.0
    • Component/s: groovy-runtime
    • Labels:
      None
    • Environment:
      Linux Mint 10
    • Testcase included:
      yes
    • Patch Submitted:
      Yes
    • Number of attachments :
      3

      Description

      I got with the following code which use http://sourceforge.net/projects/picard/files/picard-tools/

      import net.sf.picard.util.Interval 
      import net.sf.picard.util.IntervalList 
      import net.sf.picard.util.SamLocusIterator 
      import net.sf.picard.util.SamLocusIterator.RecordAndOffset 
      import net.sf.samtools.SAMFileHeader 
      import net.sf.samtools.SAMFileReader 
      import net.sf.samtools.SAMFileReader.ValidationStringency 
      
      bamFile = new File("/media/trx/workspace/Picard/test/ex1.bam") 
      baiFile = new File(bamFile.getAbsolutePath() + ".bai") 
      
      sam = new SAMFileReader(bamFile, baiFile, true) 
      
      sam.setValidationStringency(ValidationStringency.SILENT) 
      if(!sam.hasIndex()) throw new Exception("Missing index") 
      
      SAMFileHeader header = sam.getFileHeader() 
      
      sequences = header.getSequenceDictionary().getSequences() 
      
      for ( i in sequences) { 
              println("${i.getSequenceName()} = ${i.getSequenceLength()}") 
      } 
      
      IntervalList il = new IntervalList(header) 
      il.add(new Interval("chr1",0, 157)) 
      final SamLocusIterator sli = new SamLocusIterator(sam, il, true) 
      sli.setEmitUncoveredLoci(false) 
      
      sli.each { li -> 
              println li.getSequenceName() 
      } 
      

      I got the following error:

      chr1 = 1575 
      chr2 = 1584 
      WARNING	2011-04-30 12:10:29	SamLocusIterator	SamLocusIterator constructed with samReader that has SortOrder == unsorted.  Assuming SAM is coordinate sorted, but exceptions may occur if it is not. 
      Exception in thread "main" java.lang.NullPointerException 
              at net.sf.picard.util.SamLocusIterator.samHasMore(SamLocusIterator.java:223) 
              at net.sf.picard.util.SamLocusIterator.hasNext(SamLocusIterator.java:231) 
              at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1219) 
              at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1196) 
              at org.codehaus.groovy.runtime.dgm$110.invoke(Unknown Source) 
              at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:270) 
              at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:52) 
              at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40) 
              at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116) 
              at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:124) 
              at pileup.run(pileup.groovy:33) 
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
              at java.lang.reflect.Method.invoke(Method.java:616) 
              at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90) 
              at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233) 
              at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1058) 
              at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:886) 
              at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:793) 
              at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:776) 
              at org.codehaus.groovy.runtime.InvokerHelper.runScript(InvokerHelper.java:394) 
              at org.codehaus.groovy.runtime.InvokerHelper$runScript.call(Unknown Source) 
              at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40) 
              at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116) 
              at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128) 
              at pileup.main(pileup.groovy) 
      

      The following code is written completely in Java and uses the same input file without any error:

      import java.io.File; 
      import java.util.List; 
      import net.sf.picard.util.Interval; 
      import net.sf.picard.util.IntervalList; 
      import net.sf.picard.util.SamLocusIterator; 
      import net.sf.picard.util.SamLocusIterator.LocusInfo; 
      import net.sf.picard.util.SamLocusIterator.RecordAndOffset; 
      import net.sf.samtools.SAMFileHeader; 
      import net.sf.samtools.SAMFileReader; 
      import net.sf.samtools.SAMRecord; 
      import net.sf.samtools.SAMRecordIterator; 
      import net.sf.samtools.SAMSequenceRecord; 
      import net.sf.samtools.SAMFileReader.ValidationStringency; 
      
      
      public class ReadReferenceName { 
      
              /** 
               * @param args 
               * @throws Exception 
               */ 
              public static void main(String[] args) throws Exception { 
                      // TODO Auto-generated method stub 
                      
                      File bamFile = new File("/media/trx/workspace/Picard/test/ex1.bam"); 
                      File baiFile = new File(bamFile.getAbsolutePath() + ".bai");	
                      
                      SAMFileReader sam = new SAMFileReader(bamFile, baiFile, true); 
              sam.setValidationStringency(ValidationStringency.SILENT); 
                      if(!sam.hasIndex()) throw new Exception("Missing index"); 
                      
                      SAMFileHeader header = sam.getFileHeader(); 
                      List<SAMSequenceRecord> sequences = header.getSequenceDictionary().getSequences(); 
                      
                      for(SAMSequenceRecord rec: sequences) { 
                              System.out.println(rec.getSequenceName() + " = " + 
                                              rec.getSequenceLength()); 
                      } 
                      
                      IntervalList il = new IntervalList(header); 
                      il.add(new Interval("chr1",0, 157)); 
              final SamLocusIterator sli = new SamLocusIterator(sam, il, true); 
              sli.setEmitUncoveredLoci(false); 
            
              for (final SamLocusIterator.LocusInfo li : sli) { 
              String sequenceName = li.getSequenceName(); 
              int pos = li.getPosition(); 
              int coverage = li.getRecordAndPositions().size(); 
              System.out.println(sequenceName + " coverage at base " + pos + " = " 
              + coverage); 
              for (int i = 0; i< coverage; i++){ 
              RecordAndOffset rec = li.getRecordAndPositions().get(i); 
              char base = (char)rec.getReadBase(); 
              String readName = rec.getRecord().getReadName(); 
              System.out.println("  base in read " + readName + " = " + base);	
              } 
              } 
              } 
      } 
      

      and it produced this output:

      chr1 = 1575 
      chr2 = 1584 
      WARNING	2011-04-30 19:30:07	SamLocusIterator	SamLocusIterator constructed with samReader that has SortOrder == unsorted.  Assuming SAM is coordinate sorted, but exceptions may occur if it is not. 
      chr1 coverage at base 100 = 1 
        base in read EAS56_57:6:190:289:82 = A 
      chr1 coverage at base 101 = 1 
      ... 
      

      Tim Yates wrote the following work around for this problem:

      import net.sf.picard.util.*
      import net.sf.picard.util.SamLocusIterator.RecordAndOffset
      import net.sf.samtools.*
      import net.sf.samtools.SAMFileReader.ValidationStringency
      import net.sf.picard.filter.*
      import net.sf.samtools.util.CloseableIterator
      
      bamFile = new File("ex1.bam")
      baiFile = new File("ex1.bam.bai")
      
      sam = new SAMFileReader(bamFile, baiFile, true)
      sam.validationStringency = ValidationStringency.SILENT
      if(!sam.hasIndex()) throw new Exception("Missing index")
      
      SAMFileHeader header = sam.fileHeader
      sequences = header.sequenceDictionary.sequences
      
      sequences.each { i ->
        println "$i.sequenceName = $i.sequenceLength"
      }
      
      SamLocusIterator.metaClass.iterator = {
        if ( delegate.@samIterator != null ) {
          throw new IllegalStateException("Cannot call iterator() more than once on SamLocusIterator");
        }
        CloseableIterator<SAMRecord> tempIterator;
        if (delegate.@intervals != null) {
          tempIterator = new SamRecordIntervalIteratorFactory().makeSamRecordIntervalIterator(delegate.@samReader, delegate.@intervals, delegate.@useIndex);
        } else {
          tempIterator = delegate.@samReader.iterator();
        }
        if (delegate.@samFilters != null) {
          tempIterator = new FilteringIterator(tempIterator, new AggregateFilter(delegate.@samFilters));
        }
        delegate.@samIterator = new PeekableIterator<SAMRecord>( tempIterator )
        delegate 
      }
      
      IntervalList il = new IntervalList( header )
      il.add( new Interval( "chr1", 0, 157 ) )
      SamLocusIterator sli = new SamLocusIterator( sam, il, true )
      sli.emitUncoveredLoci = false
      
      sli.each { li ->
        println "$li.sequenceName coverage at base $li.position = ${li.recordAndPositions.size()}"
        (0..<li.recordAndPositions.size()).each { i ->
          rec = li.recordAndPositions.get( i )
      	println "  base in read $rec.record.readName = $rec.readBase"
        }
      }
      

      More information can be found here ( http://groovy.329449.n5.nabble.com/NullPointerException-problem-td4360426.html )

      1. ex1.bam
        122 kB
        Michal
      2. ex1.bam.bai
        0.2 kB
        Michal
      3. GROOVY-4851.patch
        1.0 kB
        blackdrag blackdrag

        Activity

        Hide
        CÚdric Champeau added a comment -

        Formatting tags

        Show
        CÚdric Champeau added a comment - Formatting tags
        Hide
        blackdrag blackdrag added a comment -

        After further investigation I found this minimal example to reproduce the problem:

        class X implements Iterable, Iterator {
            Iterator iterator() {null}
            boolean hasNext(){true}
            def next(){}
            void remove(){}
        }
        
        
        def x = new X()
        assert x.iterator() == null
        

        the call to iterator() will not call the declared method, instead it will call the our Iterator#iterator method in DGM. If you remove the Iterator interface it works just fine, so it is only reproducable with both interfaces present

        Show
        blackdrag blackdrag added a comment - After further investigation I found this minimal example to reproduce the problem: class X implements Iterable, Iterator { Iterator iterator() { null } boolean hasNext(){ true } def next(){} void remove(){} } def x = new X() assert x.iterator() == null the call to iterator() will not call the declared method, instead it will call the our Iterator#iterator method in DGM. If you remove the Iterator interface it works just fine, so it is only reproducable with both interfaces present
        Hide
        blackdrag blackdrag added a comment -

        reducing the level of this bug, since it is not a common problem and depends on the special occasion of an Itarable being also an Iterator

        Show
        blackdrag blackdrag added a comment - reducing the level of this bug, since it is not a common problem and depends on the special occasion of an Itarable being also an Iterator
        Hide
        blackdrag blackdrag added a comment -

        it must be something in generating the method list for this class, that makes things go wrong. DGM has the ability to shadow methods of the class, but here it is from an interface and the class implementation should take precedence

        Show
        blackdrag blackdrag added a comment - it must be something in generating the method list for this class, that makes things go wrong. DGM has the ability to shadow methods of the class, but here it is from an interface and the class implementation should take precedence
        Hide
        Paul King added a comment -

        tagging for review for 2.1.0-rc-1 otherwise we should reassign fix version

        Show
        Paul King added a comment - tagging for review for 2.1.0-rc-1 otherwise we should reassign fix version
        Hide
        blackdrag blackdrag added a comment -

        The attached patch seems to fix the issue, but creates new problems with getMetaClass(). For me it currently looks like a this.getMetaClass() is currently tending to call the DGM version of this, instead of the local getter method. The fix changes this, but that of course breaks some things here and there (35 meta class related failures)

        Due to those failures I think it is not appropriate to have this in Groovy 2.x and the fix has to wait for Groovy 3 instead

        Show
        blackdrag blackdrag added a comment - The attached patch seems to fix the issue, but creates new problems with getMetaClass(). For me it currently looks like a this.getMetaClass() is currently tending to call the DGM version of this, instead of the local getter method. The fix changes this, but that of course breaks some things here and there (35 meta class related failures) Due to those failures I think it is not appropriate to have this in Groovy 2.x and the fix has to wait for Groovy 3 instead
        Hide
        blackdrag blackdrag added a comment -

        Setting new fix version to 3.0

        Show
        blackdrag blackdrag added a comment - Setting new fix version to 3.0

          People

          • Assignee:
            Unassigned
            Reporter:
            Michal
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated: