Jetty
  1. Jetty
  2. JETTY-958

request.getParameter fails to pasrse UTF-8 POST data

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 7.0.0.pre5
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:
      server linux Java6u10. browsers: safari, firefox, konqueror.
    • Number of attachments :
      2

      Description

      I found that that UTF-8 encoded form data is not correctly parsed by request.getParamter.
      I had found this bug in servlet. But I made a simple JSP page to demo the problem.

      Here is screenshot that demos results
      http://img72.imageshack.us/img72/3739/snapshot14.png

      File: test.jsp
      [CODE]
      <%@ page language="java" contentType="text/html; charset=UTF-8"
      pageEncoding="UTF-8"%>
      <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
      <html>
      <head>
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <title>Insert title here</title>
      </head>
      <body>
      <%= request.getParameter("language") %>
      <form name='form1' action='test3.jsp' method='post' accept-charset='UTF-8'>
      <input type='hidden' id='id_action' name='action' value='do' />
      <input type='text' id='id_language' name='language' value='English Русский 北方話 / 官話 北方话 / 官话' />
      <input type='submit' id='id_submit' name='Submit' value='' />
      </form>
      </body>
      </html>
      [/CODE]

      My original servlet had same effect.

      response.setCharacterEncoding("UTF-8"); - was the first line in doGet and doPost

      1. test3.jsp
        0.7 kB
        Max
      1. snapshot14.png
        29 kB

        Activity

        Hide
        Max added a comment -

        /opt/jdk1.6.0_10/bin/java -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:35785 -Dfile.encoding=UTF-8 -classpath /home/max/my/workspace/Jetty7/bin:/home/max/my/workspace/Jetty7/start.jar org.mortbay.start.Main

        Show
        Max added a comment - /opt/jdk1.6.0_10/bin/java -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:35785 -Dfile.encoding=UTF-8 -classpath /home/max/my/workspace/Jetty7/bin:/home/max/my/workspace/Jetty7/start.jar org.mortbay.start.Main
        Hide
        Max added a comment -

        Jetty fails when I start it with -Dorg.mortbay.util.URI.charset=utf-8

        /opt/jdk1.6.0_10/bin/java -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:36194 -Dorg.mortbay.util.URI.charset=utf-8 -Dfile.encoding=UTF-8 -classpath /home/max/my/workspace/Jetty7/bin:/home/max/my/workspace/Jetty7/start.jar org.mortbay.start.Main

        It throws exception when request made to server

        2009-03-14 20:43:07.362::INFO: Logging to STDERR via org.mortbay.log.StdErrLog
        2009-03-14 20:43:08.860::INFO: jetty-7.0.0.pre5
        2009-03-14 20:43:08.07::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app6.xml -> org.mortbay.jetty.webapp.WebAppContext@15ed659

        {/app6,../JettyTest03/WebContent}

        2009-03-14 20:43:08.011::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/root.xml -> org.mortbay.jetty.webapp.WebAppContext@14a7a12

        {/,../JettyTest05/WebContent}

        2009-03-14 20:43:08.033::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app7.xml -> org.mortbay.jetty.webapp.WebAppContext@1bc16f0

        {/app7,../JettyTest04/WebContent}

        2009-03-14 20:43:08.040::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/axajfront01.xml -> org.mortbay.jetty.webapp.WebAppContext@18622f3

        {/js3,../AjaxFront01/WebContent}

        2009-03-14 20:43:11.676::INFO: Opened /home/max/my/workspace/Jetty7/logs/2009_03_14.request.log
        2009-03-14 20:43:11.734::INFO: Started SelectChannelConnector@0.0.0.0:8080
        2009-03-14 20:43:11.919::WARN: EXCEPTION
        java.lang.NullPointerException: charsetName
        at java.lang.String.<init>(String.java:442)
        at org.mortbay.util.StringUtil.toString(StringUtil.java:328)
        at org.mortbay.jetty.EncodedHttpURI.getPathAndParam(EncodedHttpURI.java:83)
        at org.mortbay.jetty.Request.getRequestURI(Request.java:762)
        at org.mortbay.jetty.servlet.SessionHandler.setRequestedId(SessionHandler.java:251)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:135)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:822)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:305)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:229)
        at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:113)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:324)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:550)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:876)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:535)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:209)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:407)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:421)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:520)

        Show
        Max added a comment - Jetty fails when I start it with -Dorg.mortbay.util.URI.charset=utf-8 /opt/jdk1.6.0_10/bin/java -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:36194 -Dorg.mortbay.util.URI.charset=utf-8 -Dfile.encoding=UTF-8 -classpath /home/max/my/workspace/Jetty7/bin:/home/max/my/workspace/Jetty7/start.jar org.mortbay.start.Main It throws exception when request made to server 2009-03-14 20:43:07.362::INFO: Logging to STDERR via org.mortbay.log.StdErrLog 2009-03-14 20:43:08.860::INFO: jetty-7.0.0.pre5 2009-03-14 20:43:08.07::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app6.xml -> org.mortbay.jetty.webapp.WebAppContext@15ed659 {/app6,../JettyTest03/WebContent} 2009-03-14 20:43:08.011::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/root.xml -> org.mortbay.jetty.webapp.WebAppContext@14a7a12 {/,../JettyTest05/WebContent} 2009-03-14 20:43:08.033::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app7.xml -> org.mortbay.jetty.webapp.WebAppContext@1bc16f0 {/app7,../JettyTest04/WebContent} 2009-03-14 20:43:08.040::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/axajfront01.xml -> org.mortbay.jetty.webapp.WebAppContext@18622f3 {/js3,../AjaxFront01/WebContent} 2009-03-14 20:43:11.676::INFO: Opened /home/max/my/workspace/Jetty7/logs/2009_03_14.request.log 2009-03-14 20:43:11.734::INFO: Started SelectChannelConnector@0.0.0.0:8080 2009-03-14 20:43:11.919::WARN: EXCEPTION java.lang.NullPointerException: charsetName at java.lang.String.<init>(String.java:442) at org.mortbay.util.StringUtil.toString(StringUtil.java:328) at org.mortbay.jetty.EncodedHttpURI.getPathAndParam(EncodedHttpURI.java:83) at org.mortbay.jetty.Request.getRequestURI(Request.java:762) at org.mortbay.jetty.servlet.SessionHandler.setRequestedId(SessionHandler.java:251) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:135) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:822) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:305) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:229) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:113) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:324) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:550) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:876) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:535) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:209) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:407) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:421) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:520)
        Hide
        Max added a comment -

        I coded MyRequest wrapper. It parses request.getInputStream data and I can read UNICODE characters.
        It guarantees that browser sends data in correct format.

        MyRequest myRequest = new MyRequest(request);
        String name = myRequest.getParameter("name");

        name will contain correct text

        Thanks, I hope it helps

        Show
        Max added a comment - I coded MyRequest wrapper. It parses request.getInputStream data and I can read UNICODE characters. It guarantees that browser sends data in correct format. MyRequest myRequest = new MyRequest(request); String name = myRequest.getParameter("name"); name will contain correct text Thanks, I hope it helps
        Hide
        Jan Bartel added a comment -

        Max,

        Out of interest, does it work if you use -Dorg.mortbay.util.URI.charset=UTF-8 instead?

        Jan

        Show
        Jan Bartel added a comment - Max, Out of interest, does it work if you use -Dorg.mortbay.util.URI.charset=UTF-8 instead? Jan
        Hide
        Max added a comment -

        Here is log for -Dorg.mortbay.util.URI.charset=UTF-8 (same problem)

        Exception is thrown when page is requested

        /opt/jdk1.6.0_10/bin/java -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:43141 -Dorg.mortbay.util.URI.charset=UTF-8 -Dfile.encoding=UTF-8 -classpath /home/max/my/workspace/Jetty7/bin:/home/max/my/workspace/Jetty7/start.jar org.mortbay.start.Main

        2009-03-15 18:35:22.470::INFO: Logging to STDERR via org.mortbay.log.StdErrLog
        2009-03-15 18:35:22.925::INFO: jetty-7.0.0.pre5
        2009-03-15 18:35:22.074::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app6.xml -> org.mortbay.jetty.webapp.WebAppContext@1049d3

        {/app6,../JettyTest03/WebContent}

        2009-03-15 18:35:22.078::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/root.xml -> org.mortbay.jetty.webapp.WebAppContext@15ed659

        {/,../JettyTest05/WebContent}

        2009-03-15 18:35:22.095::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app7.xml -> org.mortbay.jetty.webapp.WebAppContext@da2cef

        {/app7,../JettyTest04/WebContent}

        2009-03-15 18:35:22.100::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/axajfront01.xml -> org.mortbay.jetty.webapp.WebAppContext@1bc16f0

        {/js3,../AjaxFront01/WebContent}

        2009-03-15 18:35:26.553::INFO: Opened /home/max/my/workspace/Jetty7/logs/2009_03_15.request.log
        2009-03-15 18:35:26.595::INFO: Started SelectChannelConnector@0.0.0.0:8080
        2009-03-15 18:35:26.797::WARN: EXCEPTION
        java.lang.NullPointerException: charsetName
        at java.lang.String.<init>(String.java:442)
        at org.mortbay.util.StringUtil.toString(StringUtil.java:328)
        at org.mortbay.jetty.EncodedHttpURI.getPathAndParam(EncodedHttpURI.java:83)
        at org.mortbay.jetty.Request.getRequestURI(Request.java:762)
        at org.mortbay.jetty.servlet.SessionHandler.setRequestedId(SessionHandler.java:251)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:135)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:822)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:305)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:229)
        at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:113)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:324)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:550)

        Show
        Max added a comment - Here is log for -Dorg.mortbay.util.URI.charset=UTF-8 (same problem) Exception is thrown when page is requested /opt/jdk1.6.0_10/bin/java -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:43141 -Dorg.mortbay.util.URI.charset=UTF-8 -Dfile.encoding=UTF-8 -classpath /home/max/my/workspace/Jetty7/bin:/home/max/my/workspace/Jetty7/start.jar org.mortbay.start.Main 2009-03-15 18:35:22.470::INFO: Logging to STDERR via org.mortbay.log.StdErrLog 2009-03-15 18:35:22.925::INFO: jetty-7.0.0.pre5 2009-03-15 18:35:22.074::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app6.xml -> org.mortbay.jetty.webapp.WebAppContext@1049d3 {/app6,../JettyTest03/WebContent} 2009-03-15 18:35:22.078::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/root.xml -> org.mortbay.jetty.webapp.WebAppContext@15ed659 {/,../JettyTest05/WebContent} 2009-03-15 18:35:22.095::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/app7.xml -> org.mortbay.jetty.webapp.WebAppContext@da2cef {/app7,../JettyTest04/WebContent} 2009-03-15 18:35:22.100::INFO: Deploy /home/max/my/workspace/Jetty7/contexts/axajfront01.xml -> org.mortbay.jetty.webapp.WebAppContext@1bc16f0 {/js3,../AjaxFront01/WebContent} 2009-03-15 18:35:26.553::INFO: Opened /home/max/my/workspace/Jetty7/logs/2009_03_15.request.log 2009-03-15 18:35:26.595::INFO: Started SelectChannelConnector@0.0.0.0:8080 2009-03-15 18:35:26.797::WARN: EXCEPTION java.lang.NullPointerException: charsetName at java.lang.String.<init>(String.java:442) at org.mortbay.util.StringUtil.toString(StringUtil.java:328) at org.mortbay.jetty.EncodedHttpURI.getPathAndParam(EncodedHttpURI.java:83) at org.mortbay.jetty.Request.getRequestURI(Request.java:762) at org.mortbay.jetty.servlet.SessionHandler.setRequestedId(SessionHandler.java:251) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:135) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:822) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:305) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:229) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:113) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:324) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:550)
        Hide
        Athena Yao added a comment -

        Did a bit of checking, Jetty 7 trunk no longer throws the NPE, but it still doesn't handle that post data properly.

        Show
        Athena Yao added a comment - Did a bit of checking, Jetty 7 trunk no longer throws the NPE, but it still doesn't handle that post data properly.
        Hide
        Jan Bartel added a comment -

        Max,

        There are 2 places where character encoding comes into play. One is the form content data, which your test3.jsp demonstrates, and one is in the uri of requests.

        As I believe you raised this issue regarding form content, lets tackle that first.

        Basically, browsers are very bad at sending the character encoding type of form data. The only way to be sure that the data is interpreted correctly on the server side is to call request.setCharacterEncoding("UTF-8"); Modifying your test3.jsp to call that before calling request.getParameter() works.

        Now, as for utf-8 characters in request uris, that is the default., but you can tell jetty to interpret it a different way by using the -Dorg.mortbay.util.URI.charset=UTF-8 system property setting. As you found out, there was a bug in jetty-7.0.0.pre5 where an NPE was thrown, but this has been fixed in jetty-7 trunk.

        So, all that being said, I'm going to close this issue for now, but please re-open if you feel there is something more we need to do.

        regards
        Jan

        Show
        Jan Bartel added a comment - Max, There are 2 places where character encoding comes into play. One is the form content data, which your test3.jsp demonstrates, and one is in the uri of requests. As I believe you raised this issue regarding form content, lets tackle that first. Basically, browsers are very bad at sending the character encoding type of form data. The only way to be sure that the data is interpreted correctly on the server side is to call request.setCharacterEncoding("UTF-8"); Modifying your test3.jsp to call that before calling request.getParameter() works. Now, as for utf-8 characters in request uris, that is the default., but you can tell jetty to interpret it a different way by using the -Dorg.mortbay.util.URI.charset=UTF-8 system property setting. As you found out, there was a bug in jetty-7.0.0.pre5 where an NPE was thrown, but this has been fixed in jetty-7 trunk. So, all that being said, I'm going to close this issue for now, but please re-open if you feel there is something more we need to do. regards Jan

          People

          • Assignee:
            Jan Bartel
            Reporter:
            Max
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: