groovy

an experimental pre-processor hook for groovy

Details

  • Type: New Feature New Feature
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Won't Fix
  • Affects Version/s: 1.0-beta-4
  • Fix Version/s: None
  • Component/s: lexer
  • Labels:
    None
  • Number of attachments :
    3

Description

Since people are starting talking about adding macros etc to Groovy. I have quickly packed a poor man's experimental pre-processor here and would like to show that a simple and easy pre-processor can be done and achieve some nice functionality.

With groovy-pp, now I can do things like:

1. sqlG

fn
ln
id = 1
#sql [sql] {
select firstname, lastname
into :fn, :ln
from mytable
where emp_id = :id
}

#sql [sql] {
update thetable set firstname = :fn where emp_id = :id
}

":" is used to identify host variables. It's inspired by sqlJ.

More embedded SQL has yet to be implemented.

2. Make powerful JavaBeans :

class MyBean {
#boundsupport
#bound int balance = 0
#readonly name
#writeonly message
}

And then I can use Closure to listen to the propertyChangeEvent:

bean = new MyBena()
bean.addPropertyChangeListener {
print "$

{it.propertyName}

, $

{it.oldValue}

, $

{it.newValue}

"
}

3. A beeper

Actually it's for quick dumping messages to stdout with the information of the current method, class, and line number in the source file:

#beepon
a = 1
1.times {
#beep "a has a value of $

{a}

"
#beep a has a value of :a
}

-O< PPBeepTest$1.doCall(PPBeepTest.groovy:7): a has a value of 1
-O< PPBeepTest$1.doCall(PPBeepTest.groovy:7): a has a value of 1

The beeper can be turned off quickly by

#beepoff

There is also a macro that saves me from typing excessive "\" in a regular expressions.

All the above is sampled by a file called
PPTest.groovy
in:
bran.groovy.preprocessing

Where all the source code can be found, including a configuration file that maps macro directives to processors. All these are hard-coded right now.

Basically I have added a filter to the InputStreamCharStream, which provides source stream to the lexer. All the macro expanding happens before the lexer.

I have used # as the indicator of possible pre-processing and it has to be the first none-space character on a line to take effect. I believe some people prefer other indicators such as @. We can easily change that later.

This stuff is very experimental and probably only good for demo purpose. But I like the light-weight nature that allows everyone to quickly hack Groovy. It's quite possible that the whole thing will be tossed out once we have a formal interface for language extension.

Personally I have found the JavaBeans macros are quite powerful together with use of Closures as listeners. Note that that indexed properties are not implemented yet.

I　have made minor changes to InputStreamCharStream to open up the hook. One or two new methods have been created in the DefaultGroovyMethod, which are none-essential and nice to have. A little change is also made to the Sql class, but i don't think it's required to run the test script.

Gone for sleep

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Bing Ran added a comment -

The system won't allow me to attach the source code jar. What can I do?

Show
Bing Ran added a comment - The system won't allow me to attach the source code jar. What can I do?
Hide
Permalink
Sam Pullara added a comment -

One of the best things about Java is that it has no preprocessor. Please do not add this. If you want macros, make lisp-macros not textual replacement.

Show
Sam Pullara added a comment - One of the best things about Java is that it has no preprocessor. Please do not add this. If you want macros, make lisp-macros not textual replacement.
Hide
Permalink
Bing Ran added a comment -

People's view differs of course. There are people missing pre-processing. How about SqlJ? It's preprocessing Java code and some people use it for good reasons. And it seems not bothering people who don't like it. Personallly I hate and love pre-processing. Why don't keep the option open?

My general philosophy about Groovy is: Allow optionals. I don't disable an option because I don't like it. I simply don't use it.

In the meantime I'm thinking the only thing that we need to change about the core is a few lines of looking-up in the system property and see if the user has configured a source code stream provider filter, so to separate the concerns. Nothing will happen unless the user specifies a specific system property. This way there is no danger of stepping on other's toes. I also take steps to ensure that a specific processor can only see the portion of code that it's responsible for.

Show
Bing Ran added a comment - People's view differs of course. There are people missing pre-processing. How about SqlJ? It's preprocessing Java code and some people use it for good reasons. And it seems not bothering people who don't like it. Personallly I hate and love pre-processing. Why don't keep the option open? My general philosophy about Groovy is: Allow optionals. I don't disable an option because I don't like it. I simply don't use it. In the meantime I'm thinking the only thing that we need to change about the core is a few lines of looking-up in the system property and see if the user has configured a source code stream provider filter, so to separate the concerns. Nothing will happen unless the user specifies a specific system property. This way there is no danger of stepping on other's toes. I also take steps to ensure that a specific processor can only see the portion of code that it's responsible for.
Hide
Permalink
Sam Pullara added a comment -

SQLJ can be implemented with the sublanguage feature we are looking at without resorting to a textual preprocessor. A preprocessor is a bad idea for a number of reasons:

1) it obfuscates the code such that you cannot find the source of an error without debugging the generated code.
2) it acts at a text level, not the AST level so there is the possibility of generating illegal code or code that behaves in an unexpected way. for example, look at all the stuff that you have to escape/parenthesise in C to ensure that things like double increments don't happen.
3) maintainable code is best done without optionals. witness the maintainability of perl if you want an example of a failed experiment with lots of optional syntax.
4) adding a preprocessor ties your program to a particular build environment. look at makefiles if you want an example of how difficult it is to build source that depends on certain command line settings for the compiler.
5) it is very easy to make an external tool to do any preprocessing you think is necessary. i think that is exactly where this functionality should remain.

This is just my opinion, but now that groovy is a JSR we need to be very careful about which features are put in because we will be stuck with them. Nothing will put people off of Groovy more than it looking like a giant kitchen sink of features with no thought to cleanliness.

Mike Spille has already noted a lot of the untidyness of the current optional features of Groovy and my feeling is that will be the opinion of those on the expert group:

http://jroller.com/page/pyrasun/?anchor=groovy_ambiguities_optional_elements_and

Show
Sam Pullara added a comment - SQLJ can be implemented with the sublanguage feature we are looking at without resorting to a textual preprocessor. A preprocessor is a bad idea for a number of reasons: 1) it obfuscates the code such that you cannot find the source of an error without debugging the generated code. 2) it acts at a text level, not the AST level so there is the possibility of generating illegal code or code that behaves in an unexpected way. for example, look at all the stuff that you have to escape/parenthesise in C to ensure that things like double increments don't happen. 3) maintainable code is best done without optionals. witness the maintainability of perl if you want an example of a failed experiment with lots of optional syntax. 4) adding a preprocessor ties your program to a particular build environment. look at makefiles if you want an example of how difficult it is to build source that depends on certain command line settings for the compiler. 5) it is very easy to make an external tool to do any preprocessing you think is necessary. i think that is exactly where this functionality should remain. This is just my opinion, but now that groovy is a JSR we need to be very careful about which features are put in because we will be stuck with them. Nothing will put people off of Groovy more than it looking like a giant kitchen sink of features with no thought to cleanliness. Mike Spille has already noted a lot of the untidyness of the current optional features of Groovy and my feeling is that will be the opinion of those on the expert group: http://jroller.com/page/pyrasun/?anchor=groovy_ambiguities_optional_elements_and
Hide
Permalink
Bing Ran added a comment -

added Lips style macro support, so all Groovy power is available when scripting the macro. ongoing work. somewhat limited. Can pass simple use cases (see MacroTest.groovy)

Show
Bing Ran added a comment - added Lips style macro support, so all Groovy power is available when scripting the macro. ongoing work. somewhat limited. Can pass simple use cases (see MacroTest.groovy)
Hide
Permalink
Bing Ran added a comment -

the test file for macro support

Show
Bing Ran added a comment - the test file for macro support
Hide
Permalink
Bing Ran added a comment -

Thanks Sam for pointing me to Lisp for macro idea. Never used it before. But the idea applies nicely to Groovy. The way I write it now is to run a scipt twice, the first time for the macros and expansion computation, the second time for the final script. Of course it happens behind the scene. Users will notice a slight hesitation whe running a script that defines and uses macros.

Multiple macros are collected together to form a expansion computation script that gets evaluated once. Thus it scales.

Nested macro expansion is not there.

All paramaters are passed the macros as literal strings. Writers can convert them to whatever types they desire.

example scrit:

------8< -----
package bran.groovy.preprocessing

#defmacro swap(l, r) {
tmp1 = l; // use temps to
tmp2 = r; // prevent multiple evaluations
return "__tmp = $

{tmp1};${tmp1}

= $

{tmp2};${tmp2}

= __tmp"
}
a = 1
b = 10
swap(a, b)

assert a == 10
assert b == 1

#echo a b

c = 'hello'
d = 'world'
swap(c, d)

assert c == 'world'
assert d == 'hello'
#echo c d

#defmacro add(l , r) {
li = new Integer(l);
ri = new Integer(r);

return (li + ri)
}

ad = add (1, 2)
assert ad == 3 // ad is 3 already at compile time
#echo ad

------->8----

where #echo is a "system macro" that do pretty printing of vriable names and values. Can be replaced by println.

Show
Bing Ran added a comment - Thanks Sam for pointing me to Lisp for macro idea. Never used it before. But the idea applies nicely to Groovy. The way I write it now is to run a scipt twice, the first time for the macros and expansion computation, the second time for the final script. Of course it happens behind the scene. Users will notice a slight hesitation whe running a script that defines and uses macros. Multiple macros are collected together to form a expansion computation script that gets evaluated once. Thus it scales. Nested macro expansion is not there. All paramaters are passed the macros as literal strings. Writers can convert them to whatever types they desire. example scrit: ------8< ----- package bran.groovy.preprocessing #defmacro swap(l, r) { tmp1 = l; // use temps to tmp2 = r; // prevent multiple evaluations return "__tmp = $ {tmp1};${tmp1} = $ {tmp2};${tmp2} = __tmp" } a = 1 b = 10 swap(a, b) assert a == 10 assert b == 1 #echo a b c = 'hello' d = 'world' swap(c, d) assert c == 'world' assert d == 'hello' #echo c d #defmacro add(l , r) { li = new Integer(l); ri = new Integer(r); return (li + ri) } ad = add (1, 2) assert ad == 3 // ad is 3 already at compile time #echo ad ------- >8 ---- where #echo is a "system macro" that do pretty printing of vriable names and values. Can be replaced by println.
Hide
Permalink
Guillaume Laforge added a comment -

We'd like to support some form of AST transforms in the future which may be close to what this issue was about.

Show
Guillaume Laforge added a comment - We'd like to support some form of AST transforms in the future which may be close to what this issue was about.

People

  • Assignee:
    Guillaume Laforge
    Reporter:
    Bing Ran
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: