groovy
  1. groovy
  2. GROOVY-4105

List Comprehensions are Missing from Groovy

    Details

    • Type: Improvement Improvement
    • Status: Open Open
    • Priority: Critical Critical
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Compiler
    • Labels:
      None
    • Number of attachments :
      1

      Description

      List comprehensions are present in C#, Clojure, Common Lisp, Erlang, Haskell, JavaScript, Boo, OCaml, Perl, Python, Scala, Scheme and other languages. They are obviously missing from Groovy.

        Issue Links

          Activity

          Hide
          Guillaume Laforge added a comment -

          Have you checked JIRA before creating that issue?
          I think one of the oldest ones still open is about that topic too!

          Show
          Guillaume Laforge added a comment - Have you checked JIRA before creating that issue? I think one of the oldest ones still open is about that topic too!
          Hide
          Russel Winder added a comment -

          Yes – GROOVY-54. Probably. It is obviously so old no-one is taking any notice of it, hence this one.

          Show
          Russel Winder added a comment - Yes – GROOVY-54 . Probably. It is obviously so old no-one is taking any notice of it, hence this one.
          Hide
          Paul King added a comment -

          Perhaps we need more examples or requirements. I think Groovy needs some kind of Lazy lists bundled in, but we mostly know how to do that, and if Groovy had that built-in, then I struggle to find good examples to support a list comprehension syntax. E.g., take this python example from GROOVY-54:

          [ (x,y) for x in range(5) for y in range(3) if (y+x) % (x+2) == 0 ]  // python
          

          I would expect any Java developer to be able to convert to something like this:

          def result = []
          for (x in 0..<5)
            for (y in 0..<3)
              if ((y + x) % (x + 2) == 0) result << [x, y]
          

          And slightly more season Groovy folk should have no trouble giving this:

          [0..<5, 0..<3].combinations().findAll{ x, y -> (y + x) % (x + 2) == 0 }
          

          which is relatively easy to understand. And if it was backed by lazy lists ... do I need anything more?

          Show
          Paul King added a comment - Perhaps we need more examples or requirements. I think Groovy needs some kind of Lazy lists bundled in, but we mostly know how to do that, and if Groovy had that built-in, then I struggle to find good examples to support a list comprehension syntax. E.g., take this python example from GROOVY-54 : [ (x,y) for x in range(5) for y in range(3) if (y+x) % (x+2) == 0 ] // python I would expect any Java developer to be able to convert to something like this: def result = [] for (x in 0..<5) for (y in 0..<3) if ((y + x) % (x + 2) == 0) result << [x, y] And slightly more season Groovy folk should have no trouble giving this: [0..<5, 0..<3].combinations().findAll{ x, y -> (y + x) % (x + 2) == 0 } which is relatively easy to understand. And if it was backed by lazy lists ... do I need anything more?
          Hide
          Paul King added a comment -

          My last comment wasn't in anyway meant as discouragement. I think this is a great area to explore. I just think we need to tease apart any tangled requirements ... which might lead to some low hanging fruit ... which might lead to some more traction.

          Show
          Paul King added a comment - My last comment wasn't in anyway meant as discouragement. I think this is a great area to explore. I just think we need to tease apart any tangled requirements ... which might lead to some low hanging fruit ... which might lead to some more traction.
          Hide
          Russel Winder added a comment - - edited

          The first Groovy example above is exactly the problem, the declaration of a data structure should look like the declaration of a data structure not a bit of imperative code that manipulates a data structure!

          The second example has more merit in many ways but is still not really obvious as the declaration of a data structure. This examples handles the iterator and selector ideas, the element missing is the expression. Let me amend the target declaration to:

          [ Flob ( x + y ) for x in range(5) for y in range(3) if (y+x) % (x+2) == 0 ]  // python
          

          where we assume Flob is some form of callable delivering a value (which includes object instantiation). Presumably this maps to something like:

           [0..<5, 0..<3].combinations().findAll{ x, y -> (y + x) % (x + 2) == 0 }.collect { x , y -> Flob ( x + y ) }
          

          So the question is: why do the functional languages that allow working in this way (e.g. Miranda, Haskell, etc.) provide comprehensions as well. I would suggest two possible reasons:

          0. It doesn't look like declaring a data structure.
          1. It is horrendously inefficient due to all the new list instances created.

          I think the argument that "if Haskell and Erlang as well as Python thinks it is an appropriate coding structure, then Groovy should do something along these lines" applies very nicely.

          Show
          Russel Winder added a comment - - edited The first Groovy example above is exactly the problem, the declaration of a data structure should look like the declaration of a data structure not a bit of imperative code that manipulates a data structure! The second example has more merit in many ways but is still not really obvious as the declaration of a data structure. This examples handles the iterator and selector ideas, the element missing is the expression. Let me amend the target declaration to: [ Flob ( x + y ) for x in range(5) for y in range(3) if (y+x) % (x+2) == 0 ] // python where we assume Flob is some form of callable delivering a value (which includes object instantiation). Presumably this maps to something like: [0..<5, 0..<3].combinations().findAll{ x, y -> (y + x) % (x + 2) == 0 }.collect { x , y -> Flob ( x + y ) } So the question is: why do the functional languages that allow working in this way (e.g. Miranda, Haskell, etc.) provide comprehensions as well. I would suggest two possible reasons: 0. It doesn't look like declaring a data structure. 1. It is horrendously inefficient due to all the new list instances created. I think the argument that "if Haskell and Erlang as well as Python thinks it is an appropriate coding structure, then Groovy should do something along these lines" applies very nicely.
          Hide
          Paul King added a comment -

          Discussing your two points (in reverse order).

          Point 1: "horrendously inefficient"

          If we implemented in terms of lazy lists, this doesn't need to be the case at all. In fact C#'s query expressions aren't far off what we have above. If I understood correctly, C#'s equivalent when converted back to a Groovy equivalent would be something like:

          (0..<5).collectMany({ x -> (0..<3) }, { x, y -> [x, y] }).findAll{ x, y -> (y+x) % (x+2) == 0 }
          

          which if using lazy lists can be made efficient in the general case - not that I think it comes into play here.

          Point 0: "looking like a data structure declaration"

          The LINQ form of above (again given a Groovy flavor) would be something like:

          from x in 0..<5
          from y in 0..<3
          where (y+x) % (x+2) == 0
          select [x, y]
          

          We could currently achieve something similar using source or perhaps AST re-writing at the expense of needing a way to indicate that such re-writing was required in a particular source file.

          So, do these steps get us far enough or do we need a special list comprehension syntax?

          Show
          Paul King added a comment - Discussing your two points (in reverse order). Point 1: "horrendously inefficient" If we implemented in terms of lazy lists, this doesn't need to be the case at all. In fact C#'s query expressions aren't far off what we have above. If I understood correctly, C#'s equivalent when converted back to a Groovy equivalent would be something like: (0..<5).collectMany({ x -> (0..<3) }, { x, y -> [x, y] }).findAll{ x, y -> (y+x) % (x+2) == 0 } which if using lazy lists can be made efficient in the general case - not that I think it comes into play here. Point 0: "looking like a data structure declaration" The LINQ form of above (again given a Groovy flavor) would be something like: from x in 0..<5 from y in 0..<3 where (y+x) % (x+2) == 0 select [x, y] We could currently achieve something similar using source or perhaps AST re-writing at the expense of needing a way to indicate that such re-writing was required in a particular source file. So, do these steps get us far enough or do we need a special list comprehension syntax?
          Hide
          Paul King added a comment - - edited

          A collectMany() method for DGM is now attached as a patch. No lazy loading anywhere to be seen but might allow exploration from the LINQ-like syntax point of view if anyone has interest.

          Show
          Paul King added a comment - - edited A collectMany() method for DGM is now attached as a patch. No lazy loading anywhere to be seen but might allow exploration from the LINQ-like syntax point of view if anyone has interest.
          Hide
          Russel Winder added a comment - - edited

          I'll stick with wanting something like:

          [ ( x , y ) | x <- [ 0 .. 4 ] , y <- [ 0 .. 2 ] , ( y + x ) `mod` ( x + 2 ) == 0 ] -- Haskell
          [ { X , Y } || X <- [ 0 , 1 , 2 , 3 , 4 ] , Y <- [ 0 , 1 , 2 ] ,  ( Y + X ) rem ( X + 2 ) == 0 ] % Erlang
          [ ( x , y ) for x in range(5) for y in range(3) if ( y + x ) % ( x + 2 ) == 0 ]  # Python
          
          Show
          Russel Winder added a comment - - edited I'll stick with wanting something like: [ ( x , y ) | x <- [ 0 .. 4 ] , y <- [ 0 .. 2 ] , ( y + x ) `mod` ( x + 2 ) == 0 ] -- Haskell [ { X , Y } || X <- [ 0 , 1 , 2 , 3 , 4 ] , Y <- [ 0 , 1 , 2 ] , ( Y + X ) rem ( X + 2 ) == 0 ] % Erlang [ ( x , y ) for x in range(5) for y in range(3) if ( y + x ) % ( x + 2 ) == 0 ] # Python
          Hide
          blackdrag blackdrag added a comment -

          Firstly, I don't think this issue should be critical. List comprehensions... the origin is the mathematic way to define an set or relation. In the best case this is not done recursively. As far as I can see the Haskell and the Erlang version define a true/false valued list. The Python example... I don't know, looks like it does something else - the "if" is irritating me.

          alternatives:

          [{x,y -> x in 0..<5 && y in x...x+1}, {x,y -> (y+x) % (x+2) == 0}] as ListComprehension
          {def (x,y),  x in 0..<5 && y in x...x+1,  (y+x) % (x+2) == 0 } as ListComprehension
          
          ListComprehension.make { x,y ->
            range x : 0..<5
            range y : x...x+1
            value { (y+x) % (x+2) == 0 }
          }
          
          Show
          blackdrag blackdrag added a comment - Firstly, I don't think this issue should be critical. List comprehensions... the origin is the mathematic way to define an set or relation. In the best case this is not done recursively. As far as I can see the Haskell and the Erlang version define a true/false valued list. The Python example... I don't know, looks like it does something else - the "if" is irritating me. alternatives: [{x,y -> x in 0..<5 && y in x...x+1}, {x,y -> (y+x) % (x+2) == 0}] as ListComprehension {def (x,y), x in 0..<5 && y in x...x+1, (y+x) % (x+2) == 0 } as ListComprehension ListComprehension.make { x,y -> range x : 0..<5 range y : x...x+1 value { (y+x) % (x+2) == 0 } }
          Hide
          Paul King added a comment -

          Useful pointer to the LINQ transformation rules (if we go that way):
          http://blogs.msdn.com/wesdyer/archive/2006/12/21/comprehending-comprehensions.aspx

          Show
          Paul King added a comment - Useful pointer to the LINQ transformation rules (if we go that way): http://blogs.msdn.com/wesdyer/archive/2006/12/21/comprehending-comprehensions.aspx
          Hide
          The Alchemist added a comment -

          I actually like the 'if' in Python list comprehensions; it fits very well in the Python language (e.g., use 'and' instead of '&&' for boolean AND).

          And it makes the code way more readable, IMHO.

          Perhaps we can compromise on many temporary lists for now and transitioning to lazy lists in Groovy 2? I say, let the JVM do what it does well and GC those lists.

          Show
          The Alchemist added a comment - I actually like the 'if' in Python list comprehensions; it fits very well in the Python language (e.g., use 'and' instead of '&&' for boolean AND). And it makes the code way more readable, IMHO. Perhaps we can compromise on many temporary lists for now and transitioning to lazy lists in Groovy 2? I say, let the JVM do what it does well and GC those lists.

            People

            • Assignee:
              Unassigned
              Reporter:
              Russel Winder
            • Votes:
              2 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: