Jetty

request.getParameter fails to pasrse UTF-8 POST data

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Cannot Reproduce
  • Affects Version/s: 7.0.0.pre5
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None
  • Environment:
    server linux Java6u10. browsers: safari, firefox, konqueror.
  • Number of attachments :
    2

Description

I found that that UTF-8 encoded form data is not correctly parsed by request.getParamter.
I had found this bug in servlet. But I made a simple JSP page to demo the problem.

Here is screenshot that demos results
http://img72.imageshack.us/img72/3739/snapshot14.png

File: test.jsp
[CODE]
<%@ page language="java" contentType="text/html; charset=UTF-8"
pageEncoding="UTF-8"%>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Insert title here</title>
</head>
<body>
<%= request.getParameter("language") %>
<form name='form1' action='test3.jsp' method='post' accept-charset='UTF-8'>
<input type='hidden' id='id_action' name='action' value='do' />
<input type='text' id='id_language' name='language' value='English Русский 北方話 / 官話 北方话 / 官话' />
<input type='submit' id='id_submit' name='Submit' value='' />
</form>
</body>
</html>
[/CODE]

My original servlet had same effect.

response.setCharacterEncoding("UTF-8"); - was the first line in doGet and doPost

  1. File
    test3.jsp
    14/Mar/09 1:37 PM
    0.7 kB
    Max
  1. snapshot14.png
    29 kB
    14/Mar/09 1:37 PM

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Max added a comment -

/opt/jdk1.6.0_10/bin/java -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:35785 -Dfile.encoding=UTF-8 -classpath /home/max/my/workspace/Jetty7/bin:/home/max/my/workspace/Jetty7/start.jar org.mortbay.start.Main

Show
Max added a comment - /opt/jdk1.6.0_10/bin/java -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:35785 -Dfile.encoding=UTF-8 -classpath /home/max/my/workspace/Jetty7/bin:/home/max/my/workspace/Jetty7/start.jar org.mortbay.start.Main
Hide
Permalink
Max added a comment -

Jetty fails when I start it with -Dorg.mortbay.util.URI.charset=utf-8

/opt/jdk1.6.0_10/bin/java -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:36194 -Dorg.mortbay.util.URI.charset=utf-8 -Dfile.encoding=UTF-8 -classpath /home/max/my/workspace/Jetty7/bin:/home/max/my/workspace/Jetty7/start.jar org.mortbay.start.Main

It throws exception when request made to server

2009-03-14 20:43:07.362::INFO: Logging to STDERR via org.mortbay.log.StdErrLog
2009-03-14 20:43:08.860::INFO: jetty-7.0.0.pre5
2009-03-14 20:43:08.07::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app6.xml -> org.mortbay.jetty.webapp.WebAppContext@15ed659

{/app6,../JettyTest03/WebContent}

2009-03-14 20:43:08.011::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/root.xml -> org.mortbay.jetty.webapp.WebAppContext@14a7a12

{/,../JettyTest05/WebContent}

2009-03-14 20:43:08.033::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app7.xml -> org.mortbay.jetty.webapp.WebAppContext@1bc16f0

{/app7,../JettyTest04/WebContent}

2009-03-14 20:43:08.040::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/axajfront01.xml -> org.mortbay.jetty.webapp.WebAppContext@18622f3

{/js3,../AjaxFront01/WebContent}

2009-03-14 20:43:11.676::INFO: Opened /home/max/my/workspace/Jetty7/logs/2009_03_14.request.log
2009-03-14 20:43:11.734::INFO: Started SelectChannelConnector@0.0.0.0:8080
2009-03-14 20:43:11.919::WARN: EXCEPTION
java.lang.NullPointerException: charsetName
at java.lang.String.<init>(String.java:442)
at org.mortbay.util.StringUtil.toString(StringUtil.java:328)
at org.mortbay.jetty.EncodedHttpURI.getPathAndParam(EncodedHttpURI.java:83)
at org.mortbay.jetty.Request.getRequestURI(Request.java:762)
at org.mortbay.jetty.servlet.SessionHandler.setRequestedId(SessionHandler.java:251)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:135)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:822)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:305)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:229)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:113)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:324)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:550)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:876)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:535)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:209)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:407)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:421)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:520)

Show
Max added a comment - Jetty fails when I start it with -Dorg.mortbay.util.URI.charset=utf-8 /opt/jdk1.6.0_10/bin/java -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:36194 -Dorg.mortbay.util.URI.charset=utf-8 -Dfile.encoding=UTF-8 -classpath /home/max/my/workspace/Jetty7/bin:/home/max/my/workspace/Jetty7/start.jar org.mortbay.start.Main It throws exception when request made to server 2009-03-14 20:43:07.362::INFO: Logging to STDERR via org.mortbay.log.StdErrLog 2009-03-14 20:43:08.860::INFO: jetty-7.0.0.pre5 2009-03-14 20:43:08.07::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app6.xml -> org.mortbay.jetty.webapp.WebAppContext@15ed659 {/app6,../JettyTest03/WebContent} 2009-03-14 20:43:08.011::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/root.xml -> org.mortbay.jetty.webapp.WebAppContext@14a7a12 {/,../JettyTest05/WebContent} 2009-03-14 20:43:08.033::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app7.xml -> org.mortbay.jetty.webapp.WebAppContext@1bc16f0 {/app7,../JettyTest04/WebContent} 2009-03-14 20:43:08.040::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/axajfront01.xml -> org.mortbay.jetty.webapp.WebAppContext@18622f3 {/js3,../AjaxFront01/WebContent} 2009-03-14 20:43:11.676::INFO: Opened /home/max/my/workspace/Jetty7/logs/2009_03_14.request.log 2009-03-14 20:43:11.734::INFO: Started SelectChannelConnector@0.0.0.0:8080 2009-03-14 20:43:11.919::WARN: EXCEPTION java.lang.NullPointerException: charsetName at java.lang.String.<init>(String.java:442) at org.mortbay.util.StringUtil.toString(StringUtil.java:328) at org.mortbay.jetty.EncodedHttpURI.getPathAndParam(EncodedHttpURI.java:83) at org.mortbay.jetty.Request.getRequestURI(Request.java:762) at org.mortbay.jetty.servlet.SessionHandler.setRequestedId(SessionHandler.java:251) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:135) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:822) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:305) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:229) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:113) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:324) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:550) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:876) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:535) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:209) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:407) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:421) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:520)
Hide
Permalink
Max added a comment -

I coded MyRequest wrapper. It parses request.getInputStream data and I can read UNICODE characters.
It guarantees that browser sends data in correct format.

MyRequest myRequest = new MyRequest(request);
String name = myRequest.getParameter("name");

name will contain correct text

Thanks, I hope it helps

Show
Max added a comment - I coded MyRequest wrapper. It parses request.getInputStream data and I can read UNICODE characters. It guarantees that browser sends data in correct format. MyRequest myRequest = new MyRequest(request); String name = myRequest.getParameter("name"); name will contain correct text Thanks, I hope it helps
Hide
Permalink
Jan Bartel added a comment -

Max,

Out of interest, does it work if you use -Dorg.mortbay.util.URI.charset=UTF-8 instead?

Jan

Show
Jan Bartel added a comment - Max, Out of interest, does it work if you use -Dorg.mortbay.util.URI.charset=UTF-8 instead? Jan
Hide
Permalink
Max added a comment -

Here is log for -Dorg.mortbay.util.URI.charset=UTF-8 (same problem)

Exception is thrown when page is requested

/opt/jdk1.6.0_10/bin/java -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:43141 -Dorg.mortbay.util.URI.charset=UTF-8 -Dfile.encoding=UTF-8 -classpath /home/max/my/workspace/Jetty7/bin:/home/max/my/workspace/Jetty7/start.jar org.mortbay.start.Main

2009-03-15 18:35:22.470::INFO: Logging to STDERR via org.mortbay.log.StdErrLog
2009-03-15 18:35:22.925::INFO: jetty-7.0.0.pre5
2009-03-15 18:35:22.074::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app6.xml -> org.mortbay.jetty.webapp.WebAppContext@1049d3

{/app6,../JettyTest03/WebContent}

2009-03-15 18:35:22.078::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/root.xml -> org.mortbay.jetty.webapp.WebAppContext@15ed659

{/,../JettyTest05/WebContent}

2009-03-15 18:35:22.095::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app7.xml -> org.mortbay.jetty.webapp.WebAppContext@da2cef

{/app7,../JettyTest04/WebContent}

2009-03-15 18:35:22.100::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/axajfront01.xml -> org.mortbay.jetty.webapp.WebAppContext@1bc16f0

{/js3,../AjaxFront01/WebContent}

2009-03-15 18:35:26.553::INFO: Opened /home/max/my/workspace/Jetty7/logs/2009_03_15.request.log
2009-03-15 18:35:26.595::INFO: Started SelectChannelConnector@0.0.0.0:8080
2009-03-15 18:35:26.797::WARN: EXCEPTION
java.lang.NullPointerException: charsetName
at java.lang.String.<init>(String.java:442)
at org.mortbay.util.StringUtil.toString(StringUtil.java:328)
at org.mortbay.jetty.EncodedHttpURI.getPathAndParam(EncodedHttpURI.java:83)
at org.mortbay.jetty.Request.getRequestURI(Request.java:762)
at org.mortbay.jetty.servlet.SessionHandler.setRequestedId(SessionHandler.java:251)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:135)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:822)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:305)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:229)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:113)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:324)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:550)

Show
Max added a comment - Here is log for -Dorg.mortbay.util.URI.charset=UTF-8 (same problem) Exception is thrown when page is requested /opt/jdk1.6.0_10/bin/java -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:43141 -Dorg.mortbay.util.URI.charset=UTF-8 -Dfile.encoding=UTF-8 -classpath /home/max/my/workspace/Jetty7/bin:/home/max/my/workspace/Jetty7/start.jar org.mortbay.start.Main 2009-03-15 18:35:22.470::INFO: Logging to STDERR via org.mortbay.log.StdErrLog 2009-03-15 18:35:22.925::INFO: jetty-7.0.0.pre5 2009-03-15 18:35:22.074::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app6.xml -> org.mortbay.jetty.webapp.WebAppContext@1049d3 {/app6,../JettyTest03/WebContent} 2009-03-15 18:35:22.078::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/root.xml -> org.mortbay.jetty.webapp.WebAppContext@15ed659 {/,../JettyTest05/WebContent} 2009-03-15 18:35:22.095::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app7.xml -> org.mortbay.jetty.webapp.WebAppContext@da2cef {/app7,../JettyTest04/WebContent} 2009-03-15 18:35:22.100::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/axajfront01.xml -> org.mortbay.jetty.webapp.WebAppContext@1bc16f0 {/js3,../AjaxFront01/WebContent} 2009-03-15 18:35:26.553::INFO: Opened /home/max/my/workspace/Jetty7/logs/2009_03_15.request.log 2009-03-15 18:35:26.595::INFO: Started SelectChannelConnector@0.0.0.0:8080 2009-03-15 18:35:26.797::WARN: EXCEPTION java.lang.NullPointerException: charsetName at java.lang.String.<init>(String.java:442) at org.mortbay.util.StringUtil.toString(StringUtil.java:328) at org.mortbay.jetty.EncodedHttpURI.getPathAndParam(EncodedHttpURI.java:83) at org.mortbay.jetty.Request.getRequestURI(Request.java:762) at org.mortbay.jetty.servlet.SessionHandler.setRequestedId(SessionHandler.java:251) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:135) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:822) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:305) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:229) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:113) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:324) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:550)
Hide
Permalink
Athena Yao added a comment -

Did a bit of checking, Jetty 7 trunk no longer throws the NPE, but it still doesn't handle that post data properly.

Show
Athena Yao added a comment - Did a bit of checking, Jetty 7 trunk no longer throws the NPE, but it still doesn't handle that post data properly.
Hide
Permalink
Jan Bartel added a comment -

Max,

There are 2 places where character encoding comes into play. One is the form content data, which your test3.jsp demonstrates, and one is in the uri of requests.

As I believe you raised this issue regarding form content, lets tackle that first.

Basically, browsers are very bad at sending the character encoding type of form data. The only way to be sure that the data is interpreted correctly on the server side is to call request.setCharacterEncoding("UTF-8"); Modifying your test3.jsp to call that before calling request.getParameter() works.

Now, as for utf-8 characters in request uris, that is the default., but you can tell jetty to interpret it a different way by using the -Dorg.mortbay.util.URI.charset=UTF-8 system property setting. As you found out, there was a bug in jetty-7.0.0.pre5 where an NPE was thrown, but this has been fixed in jetty-7 trunk.

So, all that being said, I'm going to close this issue for now, but please re-open if you feel there is something more we need to do.

regards
Jan

Show
Jan Bartel added a comment - Max, There are 2 places where character encoding comes into play. One is the form content data, which your test3.jsp demonstrates, and one is in the uri of requests. As I believe you raised this issue regarding form content, lets tackle that first. Basically, browsers are very bad at sending the character encoding type of form data. The only way to be sure that the data is interpreted correctly on the server side is to call request.setCharacterEncoding("UTF-8"); Modifying your test3.jsp to call that before calling request.getParameter() works. Now, as for utf-8 characters in request uris, that is the default., but you can tell jetty to interpret it a different way by using the -Dorg.mortbay.util.URI.charset=UTF-8 system property setting. As you found out, there was a bug in jetty-7.0.0.pre5 where an NPE was thrown, but this has been fixed in jetty-7 trunk. So, all that being said, I'm going to close this issue for now, but please re-open if you feel there is something more we need to do. regards Jan

People

  • Assignee:
    Jan Bartel
    Reporter:
    Max
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: