List Comprehensions are Missing from Groovy

Have you checked JIRA before creating that issue?
I think one of the oldest ones still open is about that topic too!

Yes – GROOVY-54. Probably. It is obviously so old no-one is taking any notice of it, hence this one.

Perhaps we need more examples or requirements. I think Groovy needs some kind of Lazy lists bundled in, but we mostly know how to do that, and if Groovy had that built-in, then I struggle to find good examples to support a list comprehension syntax. E.g., take this python example from GROOVY-54:

[ (x,y) for x in range(5) for y in range(3) if (y+x) % (x+2) == 0 ]  // python

I would expect any Java developer to be able to convert to something like this:

def result = []
for (x in 0..<5)
  for (y in 0..<3)
    if ((y + x) % (x + 2) == 0) result << [x, y]

And slightly more season Groovy folk should have no trouble giving this:

[0..<5, 0..<3].combinations().findAll{ x, y -> (y + x) % (x + 2) == 0 }

which is relatively easy to understand. And if it was backed by lazy lists ... do I need anything more?

The first Groovy example above is exactly the problem, the declaration of a data structure should look like the declaration of a data structure not a bit of imperative code that manipulates a data structure!

The second example has more merit in many ways but is still not really obvious as the declaration of a data structure. This examples handles the iterator and selector ideas, the element missing is the expression. Let me amend the target declaration to:

[ Flob ( x + y ) for x in range(5) for y in range(3) if (y+x) % (x+2) == 0 ]  // python

where we assume Flob is some form of callable delivering a value (which includes object instantiation). Presumably this maps to something like:

 [0..<5, 0..<3].combinations().findAll{ x, y -> (y + x) % (x + 2) == 0 }.collect { x , y -> Flob ( x + y ) }

So the question is: why do the functional languages that allow working in this way (e.g. Miranda, Haskell, etc.) provide comprehensions as well. I would suggest two possible reasons:

0. It doesn't look like declaring a data structure.
1. It is horrendously inefficient due to all the new list instances created.

I think the argument that "if Haskell and Erlang as well as Python thinks it is an appropriate coding structure, then Groovy should do something along these lines" applies very nicely.

Discussing your two points (in reverse order).

Point 1: "horrendously inefficient"

If we implemented in terms of lazy lists, this doesn't need to be the case at all. In fact C#'s query expressions aren't far off what we have above. If I understood correctly, C#'s equivalent when converted back to a Groovy equivalent would be something like:

(0..<5).collectMany({ x -> (0..<3) }, { x, y -> [x, y] }).findAll{ x, y -> (y+x) % (x+2) == 0 }

which if using lazy lists can be made efficient in the general case - not that I think it comes into play here.

Point 0: "looking like a data structure declaration"

The LINQ form of above (again given a Groovy flavor) would be something like:

from x in 0..<5
from y in 0..<3
where (y+x) % (x+2) == 0
select [x, y]

We could currently achieve something similar using source or perhaps AST re-writing at the expense of needing a way to indicate that such re-writing was required in a particular source file.

So, do these steps get us far enough or do we need a special list comprehension syntax?

A collectMany() method for DGM is now attached as a patch. No lazy loading anywhere to be seen but might allow exploration from the LINQ-like syntax point of view if anyone has interest.

I'll stick with wanting something like:

[ ( x , y ) | x <- [ 0 .. 4 ] , y <- [ 0 .. 2 ] , ( y + x ) `mod` ( x + 2 ) == 0 ] -- Haskell
[ { X , Y } || X <- [ 0 , 1 , 2 , 3 , 4 ] , Y <- [ 0 , 1 , 2 ] ,  ( Y + X ) rem ( X + 2 ) == 0 ] % Erlang
[ ( x , y ) for x in range(5) for y in range(3) if ( y + x ) % ( x + 2 ) == 0 ]  # Python

Firstly, I don't think this issue should be critical. List comprehensions... the origin is the mathematic way to define an set or relation. In the best case this is not done recursively. As far as I can see the Haskell and the Erlang version define a true/false valued list. The Python example... I don't know, looks like it does something else - the "if" is irritating me.

alternatives:

[{x,y -> x in 0..<5 && y in x...x+1}, {x,y -> (y+x) % (x+2) == 0}] as ListComprehension
{def (x,y),  x in 0..<5 && y in x...x+1,  (y+x) % (x+2) == 0 } as ListComprehension

ListComprehension.make { x,y ->
  range x : 0..<5
  range y : x...x+1
  value { (y+x) % (x+2) == 0 }
}

Useful pointer to the LINQ transformation rules (if we go that way):
http://blogs.msdn.com/wesdyer/archive/2006/12/21/comprehending-comprehensions.aspx

I actually like the 'if' in Python list comprehensions; it fits very well in the Python language (e.g., use 'and' instead of '&&' for boolean AND).

And it makes the code way more readable, IMHO.

Perhaps we can compromise on many temporary lists for now and transitioning to lazy lists in Groovy 2? I say, let the JVM do what it does well and GC those lists.