Maven 2 & 3

Same for me.

How can such a problem slip into a release?

No answer? seems like a serious problem ... has changed the management of http file transfer? Can be a problem similar to that of Eclipse Galileo? http://wiki.eclipse.org/ECF_Filetransfer_Support_for_NTLMv2_Proxies

maybe you could describe the type of proxy you're trying to use? testing on proxied networks is famously difficult, and many people don't have access to test on certain configurations. It's possible an actual problem slipped through, but I'd expect this isn't beyond our reach to correct using settings.xml configuration. Since Maven 2.2.0 has switched to using a HttpClient-driven http wagon, you might take a look at these pages as starting points, and see how your particular type of proxy is configured:

http://maven.apache.org/guides/mini/guide-http-settings.html
http://hc.apache.org/httpclient-3.x/

After looking around at this some more, it's possible we'll have to address this in a 2.2.1 release. Not sure what the best strategy is, maybe simply to provide some mapping mechanism for which http wagon to use, then allow folks behind an NTLMv2 proxy to switch over to the lightweight-http solution we used to use. This cuts off some nicer features, but it doesn't look like solutions such as those in HTTPCLIENT-579 is going to be an option any time soon.

See also: http://wiki.apache.org/jakarta-httpclient/FrequentlyAskedNTLMQuestions

upgrade priority

Thanks for your reply jhon.
Some developers are not able to configure their corporate proxy. Neither the information about proxy implementation is provided to employees.
However, our proxy is NTLMv2, but the company certainly do not want to change it if I can not use maven 2.2.0.

Same for me as for Marco. Still not understanding the issue with maven 2.2.0. Why does it work with 2.1.0 and not 2.2.0. I think to understand that the HTTP library has been updated, right? Why does it break NTLM authentication then that was working before?!

I am also using an artifactory server. They use apache's httpclient library and proxy authentication works fine there. So what's the matter here?

Maybe you maven developpers should have a look at http://www.ioplex.com/jespa.html. Maybe it is not your business to make httpclient work, but at least you could suggest a patch or make the guys at apache move to include the lib. I will certainly be benefic for all of use and especially maven

Jespa is not free. Anyway httpclient 4.0 seems to adress some NTLM issues or provide insights

http://hc.apache.org/httpcomponents-client/ntlm.html

It seems we are using NTLMv1: artifactory 2.0.6 uses httpclient 3.x lib which implements NTLMv1. This works fine and authentication is successful. So not (yet) a NTLMv2 issue for me.

Anyway maven 2.2.0 cannot access internet because of Authorization failed: Not authorized by proxy.

I have used the patched lib provided by Konstantin at http://issues.apache.org/jira/browse/HTTPCLIENT-579. You can downloaded it at http://issues.apache.org/jira/secure/attachment/12410908/NTLMV21.RAR

Then I have rewritten his example see my attached file. Simply drop it to the example dir build and run.

Edit:
Steps to run the example:
1) go to src directory and typ:
$ant
2) go to examples dir
3) edit run.bat and put quotes around the value of -Djavax.net.ssl.trustStore if it contains spaces otherwise you will get an error
4) edit the java file and put your information (proxy address, login password etc)
5) to compile type
$ ant
6) execute run.bat

Marco could you try the example to see if this works for you?

The solution provided above is based on a patched version of httpclient v3.

If you look at the explanation on page http://hc.apache.org/httpcomponents-client/ntlm.html concerning httpclient 4.x it uses jcifs implementation for NTLMv2. However they will no longer support the http part of jcifs according to their page http://jcifs.samba.org/src/docs/ntlmhttpauth.html

I am puzzled here: apache recommends to use a lib that will no longer be supported (at least the http part) and redirect to a commercial implementation jespa http://www.ioplex.com/jespa.html. What sould we do? They don't even explain how to use jespa in httpclient 4.x...

jean-marc i run the examples and:

<LI id=L_default_11>Error Code: 407 Proxy Authentication Required. The ISA Server requires authorization
to fulfill the request. Access to the Web Proxy filter is denied. (12209)
<LI id=L_default_12>IP Address: ------------
<LI id=L_default_13>Date: 08/07/2009 15:14:01 [GMT]
<LI id=L_default_14>Server: -----------------
<LI id=L_default_15>Source: proxy

Marco, I don't know what that means. I am not an expert in this domain. I am just looking for solutions.

Are you sure to have provided all the required information? Do you use my example [1] or the one bundled in [2]?

Proxy authentication is just used in my example.

> I am puzzled here: apache recommends to use a lib that will no longer be supported (at least the http part)

As far as my interpretation of the JCIF developers' position goes it sounds perfectly reasonable for them to discontinue development of components they consider out of scope and concentrate their efforts on the core competency, which is the NTLM authentication protocol.

> What sould we do?

General recommendation to all HttpClient users is to not use NTLM

Oleg

> General recommendation to all HttpClient users is to not use NTLM

That is no acceptable! We work in companies where they impose proxies that use such authentication scheme. We have no influence at all on this choice.

I would really be sorry to be obliged to drop maven because of connection issues that seem not impossible to fix... Maven 2.1.0 works fine though...

Please rest assured that we're not simply going to drop support for users behind NTLM proxies. Our hands are somewhat tied with httpclient, it seems, but we're thinking about using a delegator Wagon implementation for Maven 2.2.1 that would allow you to specify which http wagon impl you want to use...the lightweight-http wagon used in Maven < 2.2.0, which supports NTLM but falls down in other fringe use cases, or the httpclient-driven wagon used in Maven 2.2.0. We're probably going to leave httpclient in as the default choice, but allow users to flip this if they run into problems.

lightweight wagon is once again the default for http/s wagons, but httpclient-driven wagons can be selected using:

-Dmaven.wagon.provider.http=httpclient 

-OR-

-Dmaven.wagon.provider.https=httpclient

Or, you can configure the provider per-server using the following <server> configuration:

<wagonProvider>httpclient</wagonProvider>

Once the release goes out, you'll be able to read up on this configuration at:

http://maven.apache.org/guides/mini/guide-wagon-providers.html

So if I understand well:

lightweight wagon is based on Sun's Java implementation and http wagon is based on Apache's HttpClient lib, right?

Anyway Artifactory (http://www.jfrog.org/products.php) uses plain vanilla httpclient lib version 3 and is able to access the internet through our NTLM corporate proxy. Why is then wagon no able to do that? I was able my self to build a small example with httpclient 3 that works fine...

Great that at least the previous behavior of maven is restored. When is 2.2.1 planned to be released?

@Jean-Marc

There are several versions of the NTLM authentication scheme [1]. HttpClient 3.x partially supports only what is known as NTLMv1. Depending on the version of the ISA proxy and its configuration this may or may not be sufficient.

Oleg

[1] http://en.wikipedia.org/wiki/NTLM

@Oleg

I totally agree with you, but:
1) httpclient 3.x works for me out of the box with my proxy
2) maven 2.2.0 fails where maven 2.1.0 succeeds to access the web

So I am wondering what is wrong anyway with the lib that wagon uses in maven 2.2.0 since httpclient 3.x works with my proxy? Something is "wronger" than just using another lib...

@Jean-Marc

Maybe you can provide us with some of this miracle code, then? We're always looking for good patches...

Maven 2.1.0 did not use the httpclient-driven wagon; it uses the Sun implementation. Also, Artifactory may have different licensing requirements than Maven, which may allow them to use code derived from jCIFS that is known to support NTLMv2 according to some other threads I've read.

A big part of the problem here is that I don't have access to an NTLMv2 proxy for testing, nor would I really know whether/how I could set one up on a VirtualBox/VMWare appliance for testing. This is the root problem when dealing with proxy issues, since there are a million flavors and they all seem to have different quirks. Are you sure you're behind an NTLM v2 proxy, or is it just v1?

We're just using httpclient, there are no tricks or hacks. We have a client instance initialized inside the HttpWagon (actually, AbstractHttpWagon, IIRC) and it's used in the same manner as I've ever seen httpclient used anywhere. There's nothing special about it, but we may not have exactly the same configuration code as you do. So, I could use a look at your working example, to compare notes.

BTW, I'll start trying to release Maven 2.2.1 as soon as our issue count hits zero for this version. As time goes on, we may shift some issues out to 2.2.2 or 2.3, and we may discover some new regressions that need addressing before the release. But in any case, I'm working as hard as I can to get this release done ASAP.

As always, users can help out immensely by providing sample code and failing test cases that can be used to create integration tests.

Follow up with the discussion in this issue related to NTLMv2 code in httpclient...

> Follow up with the discussion in this issue related to NTLMv2 code in httpclient...

No, it is NOT. This has nothing to do with HttpClient. SUN licensed the NTLM protocol from Microsoft, the luxury we obviously do not have. Besides, they make use of native code accessing native Windows APIs somewhere in their NTLM implementation, which makes NTLM auth in JRE HTTP Windows only.

HttpClient 4.0 supports NTLMv2 through JCIFS on all platforms. Unfortunately due to licensing issues we cannot ship or even import JCIFS code.

This is a purely licensing issue and it has nothing to do with HttpClient. This is the reason why the use of NTLM is strongly discouraged.

Oleg

Guys, thanks for your work ... I don't consider it a big problem using version 2.1.0 for some time

Guys,

I am perfectly understanding all your points, especially the licensing issues withJCIFS, but it seems that my explanations are not enough clear. So let's make it as clear as possible.

• We are sitting behind a proxy server which requires NTLM authentication. I don't know if it is v1 or v2, but I have some insights (see below). All server and clients are hosted on Windows machines.

• Maven 2.1.0 works where Maven 2.2.0 fails. According to your feedback, 2.2.0 has moved from Sun's http layer to Apache's httpclient 4.

• Meanwhile we have another product, Artifactory from http://www.jfrog.org/, which uses httpclient that manages to authenticate with our proxy server. I asked the developpers which version of httpclient they use in their product and they said it is an unpatched/unmodified apache's httpclient 3.x.

• I made wrote a small example (see attached [2]) which uses httpclient 3.x that authenticates successfuly with our proxy server. This makes me suppose that our proxy server uses NTLMv1, because it supported (even tough partially) by httpclient 3.x

• I retried with a modified version of httpclient that I found among the httplcient issues (http://issues.apache.org/jira/browse/HTTPCLIENT-579) which is supposed to improve NTLM support. I works fine too. However there are again licensing issues.

Conclusion:
Why is then maven 2.2.0 no longer working? Problably because it uses httpclient 4.x and not httpclient 3.x. This would explain why artifactory has no problems and maven 2.2.0 has. Something changed in version 4.x. Did they completely drop the NTLM support? If yes, this are bad news.

JCIFS is no longer going to support the HTTP implementation (http://jcifs.samba.org/src/docs/ntlmhttpauth.html). Instead they recommend to use a commercial library (http://www.ioplex.com/jespa.html). We could afford to buy the lib instead of dropping maven, but it still is unclear to me how to tell httpclient 4.x to use jespa.

Letting the user to decide which implementation of HTTP layer to use is the best option for me. Downgrade from httpclient 4.x to httpclient 3.x is not really I good idea, even if it breaks NTLM authentication. One may argue here.

I understand the licensing issues with jcifs. But would it be an option to let the end user download jcifs and make mave use it? Same question for jespa?

Maven is using httpclient 3.x, NOT 4.x. The trick seems to be that we're not using NTCredentials, since (from what I can tell) there is no way to determine whether we're talking to an NTLM proxy or not. It may be worthwhile pursuing in future releases of Wagon, to allow the use of NTCredentials if a certain configuration flag is set, or a system property is set.

As for dropping in NTLMv2 support via jcifs or similar, I'm not sure how well that would work, since I'm guessing we'd have to have code that used it directly, which would mean that code was GPL'ed...

OTOH, the modifications I've made for Maven 2.2.1 should allow you to specify your own implementation of the HTTP wagon, and use that one instead of the built-in versions.

Actually, reading the source again, it looks like NTCredentials IS used:

ProxyInfo proxyInfo = getProxyInfo( getRepository().getProtocol(), getRepository().getHost() );
        if ( proxyInfo != null )
        {
            String proxyUsername = proxyInfo.getUserName();
            String proxyPassword = proxyInfo.getPassword();
            String proxyHost = proxyInfo.getHost();
            int proxyPort = proxyInfo.getPort();
            String proxyNtlmHost = proxyInfo.getNtlmHost();
            String proxyNtlmDomain = proxyInfo.getNtlmDomain();
            if ( proxyHost != null )
            {
                hc.setProxy( proxyHost, proxyPort );

                if ( proxyUsername != null && proxyPassword != null )
                {
                    Credentials creds;
                    if ( proxyNtlmHost != null || proxyNtlmDomain != null )
                    {
                        creds = new NTCredentials( proxyUsername, proxyPassword, proxyNtlmHost, proxyNtlmDomain );
                    }
                    else
                    {
                        creds = new UsernamePasswordCredentials( proxyUsername, proxyPassword );
                    }

                    int port = proxyInfo.getPort() > -1 ? proxyInfo.getPort() : AuthScope.ANY_PORT;
                    
                    AuthScope scope = new AuthScope( proxyHost, port );
                    client.getState().setProxyCredentials( scope, creds );
                }
            }
        }

So, if you use ntlmHost and ntlmDomain in your proxy configuration, it should work for NTLMv1 (but not v2).

FYI, Eclipse 3.6M2 now includes NTLMv2 proxy support:
http://download.eclipse.org/eclipse/downloads/drops/S-3.6M2-200909170100/eclipse-news-M2.html

Peek at their code and determine if it can be re-used for Maven.