Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 6.1.14, 6.1.26
    • Fix Version/s: 6.1.27
    • Component/s: None
    • Labels:
      None
    • Patch Submitted:
      Yes
    • Number of attachments :
      1

      Description

      I hit this while working on an integration of Hadoop. Not sure it is a bug, but I'm reporting it with a patch that fixes my problem.

      Hadoop packages webapps inside the JAR, like this:

      /webapps/appname/WEB-INF/web.xml

      When Jetty extracts these apps to a temporary directory for later deployment, it includes the /webapps/appname part of the JAR path in the destination folder. The result of this is that the WEB-INF directory is not detected at deployment and the webapp is deployed as a static file app.

      The attached patch works around this by removing the /webapps/appname part of the destination path.

        Activity

        Hide
        Greg Wilkins added a comment -

        Are these war files we are talking about? If so, I'm not sure that it is legal to have the WEB-INF directory so deep in the archive.

        Can you explain some more how/why these jars are like that?

        Show
        Greg Wilkins added a comment - Are these war files we are talking about? If so, I'm not sure that it is legal to have the WEB-INF directory so deep in the archive. Can you explain some more how/why these jars are like that?
        Hide
        Trygve Sanne Hardersen added a comment -

        We are talking about the JAR files distributed with Hadoop (and HBase), for example:

        http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.20.2/hadoop-core-0.20.2.jar

        I can't give you the reason why, but the bundled web apps (exploded WARs) are packaged inside this archive in the "webapps/appname" directory. The Hadoop HttpServer locates these directories on the classpath and feeds the full JAR path and app name to Jetty.

        Jetty recognizes this path as a JAR subentry, and thus only extracts the relevant part of the JAR during deployment, but it fails to remove the subentry name from the destination directory. The result of this is that Jetty can't find the WEB-INF directory later, so the apps are deployed as static files.

        Like I said I don't know if this is a bug in Jetty, but to my knowledge this functionality isn't part of any spec, and I think it's strange that the subentry is detected but not removed from the destination directory.

        The Hadoop HttpServer constructor shows how this is called:

        http://svn.eu.apache.org/repos/asf/hadoop/common/branches/branch-0.22/src/java/org/apache/hadoop/http/HttpServer.java

        Thanks!

        Show
        Trygve Sanne Hardersen added a comment - We are talking about the JAR files distributed with Hadoop (and HBase), for example: http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.20.2/hadoop-core-0.20.2.jar I can't give you the reason why, but the bundled web apps (exploded WARs) are packaged inside this archive in the "webapps/appname" directory. The Hadoop HttpServer locates these directories on the classpath and feeds the full JAR path and app name to Jetty. Jetty recognizes this path as a JAR subentry, and thus only extracts the relevant part of the JAR during deployment, but it fails to remove the subentry name from the destination directory. The result of this is that Jetty can't find the WEB-INF directory later, so the apps are deployed as static files. Like I said I don't know if this is a bug in Jetty, but to my knowledge this functionality isn't part of any spec, and I think it's strange that the subentry is detected but not removed from the destination directory. The Hadoop HttpServer constructor shows how this is called: http://svn.eu.apache.org/repos/asf/hadoop/common/branches/branch-0.22/src/java/org/apache/hadoop/http/HttpServer.java Thanks!
        Hide
        Greg Wilkins added a comment -

        Ah it sounds like perhaps the deployment is using a jar:file notation like:

        jar:file:///some/path/hadoop-core-0.20.2.jar!/webapps/appname/

        Is that so? if so, then perhaps it is a jetty bug.

        Show
        Greg Wilkins added a comment - Ah it sounds like perhaps the deployment is using a jar:file notation like: jar: file:///some/path/hadoop-core-0.20.2.jar!/webapps/appname/ Is that so? if so, then perhaps it is a jetty bug.
        Hide
        Greg Wilkins added a comment -

        The code already has the following code:

        entryName = entryName.substring(subEntryName.length());

        and I've just added a test and it correctly strips the subentry name when extracting.

        Can you tell me more about how the hadoop jars are deployed so I can replicate your problem.

        Show
        Greg Wilkins added a comment - The code already has the following code: entryName = entryName.substring(subEntryName.length()); and I've just added a test and it correctly strips the subentry name when extracting. Can you tell me more about how the hadoop jars are deployed so I can replicate your problem.
        Hide
        Trygve Sanne Hardersen added a comment -

        Yes, sorry for my bad explanation!

        Show
        Trygve Sanne Hardersen added a comment - Yes, sorry for my bad explanation!
        Hide
        Greg Wilkins added a comment -

        Ah! found the problem!

        You can work around it for now by using a URL like

        jar:file:///some/path/hadoop-core-0.20.2.jar!/webapps/appname/

        I think you must currently be using

        jar:file:///some/path/hadoop-core-0.20.2.jar!/webapps/appname

        without the trailing /

        Show
        Greg Wilkins added a comment - Ah! found the problem! You can work around it for now by using a URL like jar: file:///some/path/hadoop-core-0.20.2.jar!/webapps/appname/ I think you must currently be using jar: file:///some/path/hadoop-core-0.20.2.jar!/webapps/appname without the trailing /
        Hide
        Greg Wilkins added a comment -

        I've committed a patch in jetty-6 that adds the / to the jar:file url if it is a directory. This fixed the reproduced problem in my test harness.

        For jetty-7, the URL is final and so I'm currently just going to throw an exception and see if this causes problems.

        Show
        Greg Wilkins added a comment - I've committed a patch in jetty-6 that adds the / to the jar:file url if it is a directory. This fixed the reproduced problem in my test harness. For jetty-7, the URL is final and so I'm currently just going to throw an exception and see if this causes problems.
        Hide
        Trygve Sanne Hardersen added a comment -

        Many thanks Greg! I've verified that appending a / to the path from within the Hadoop HttpServer solves the problem. I'll submit this to the Hadoop team at some point.

        Show
        Trygve Sanne Hardersen added a comment - Many thanks Greg! I've verified that appending a / to the path from within the Hadoop HttpServer solves the problem. I'll submit this to the Hadoop team at some point.
        Hide
        Michael Gorovoy added a comment -

        Since Greg has already fixed this issue, marking it as resolved.

        Show
        Michael Gorovoy added a comment - Since Greg has already fixed this issue, marking it as resolved.

          People

          • Assignee:
            Greg Wilkins
            Reporter:
            Trygve Sanne Hardersen
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: