SonarQube Plugins
  1. SonarQube Plugins
  2. SONARPLUGINS-2560

Sonar-runner fails to execute properly on z/OS

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: SONAR-RUNNER-2.0
    • Fix Version/s: SONAR-RUNNER-2.1
    • Component/s: SonarQube Runner
    • Labels:
      None
    • Environment:
    • Number of attachments :
      3

      Description

      sonar-runner does not execute properly on an IBM mainframe due to a faulty codepage being used when communicating with the sonar server.

      The following error message is generated when running on z/OS:

      # ./sonar-runner
      Info: Using sonar-runner at /SYX13E/usr/lpp/sonar
      Info: Using java at /usr/lpp/java/j6_sr531/J6.0/bin/java
      Info: Using classpath /SYX13E/usr/lpp/sonar/lib/sonar-runner.jar
      Info: Using project /SYX13E/usr/lpp/sonar/bin
      Runner configuration file: /SYX13E/usr/lpp/sonar/conf/sonar-runner.properties
      Project configuration file: NONE
      Runner version: 2.0
      Java version: 1.6.0, vendor: IBM Corporation
      OS name: "z/OS", version: "01.13.00", arch: "s390"
      Default locale: "en_US", source code encoding: "UTF-8"
      Server: http://ht-sonar.fnb.co.za
      Work directory: /SYX13E/usr/lpp/sonar/bin/.sonar
      Bootstrapper called with productToken=[SonarRunner/2.0], serverUrl=[http://ht-sonar.fnb.co.za], workDir=[/SYX13E/usr/lpp/sonar/bin/.sonar]
      remoteContent() called with path=[/api/server/version]
      newHttpConnection() called with url=[http://ht-sonar.fnb.co.za/api/server/version]
      About to call remoteContent() with path=[/batch/]
      remoteContent() called with path=[/batch/]
      newHttpConnection() called with url=[http://ht-sonar.fnb.co.za/batch/]
      Got lib=[%?ÅÂ/Ä]
      Creating connection with URL [http://ht-sonar.fnb.co.za/batch/%?ÅÂ/Ä]
      newHttpConnection() called with url=[http://ht-sonar.fnb.co.za/batch/%?ÅÂ/Ä]
      Total time: 1.079s
      Final Memory: 0M/4M
      Exception in thread "main" java.lang.IllegalStateException: Fail to download libraries from server
              at org.sonar.runner.Bootstrapper.downloadBatchFiles(Bootstrapper.java:184)
              at org.sonar.runner.Bootstrapper.createClassLoader(Bootstrapper.java:100)
              at org.sonar.runner.Runner.createClassLoader(Runner.java:229)
              at org.sonar.runner.Runner.execute(Runner.java:151)
              at org.sonar.runner.Main.execute(Main.java:84)
              at org.sonar.runner.Main.main(Main.java:56)
      Caused by: java.lang.IllegalStateException: Fail to download the file: http://ht-sonar.fnb.co.za/batch/%?ÅÂ/Ä
              at org.sonar.runner.Bootstrapper.remoteContentToFile(Bootstrapper.java:128)
              at org.sonar.runner.Bootstrapper.downloadBatchFiles(Bootstrapper.java:179)
              ... 5 more
      Caused by: java.io.FileNotFoundException: /SYX13E/usr/lpp/sonar/bin/.sonar/batch/%?ÅÂ/Ä (EDC5129I No such file or directory.)
              at java.io.FileOutputStream.open(Native Method)
              at java.io.FileOutputStream.<init>(FileOutputStream.java:179)
              at org.sonar.runner.Bootstrapper.remoteContentToFile(Bootstrapper.java:122)
              ... 6 more
      

      Attached is a network trace (from the Mainframe) of the communication between ronar-runner and sonar.

      Patch to follow.

      1. patch-sonar-runner-SONARPLUGINS-2560.diff
        0.7 kB
        Jaco van Tonder
      2. patch-sonar-runner-SONARPLUGINS-2560-2.diff
        1 kB
        Jaco van Tonder
      3. TRACE-1.SONAR
        30 kB
        Jaco van Tonder

        Activity

        Hide
        Jaco van Tonder added a comment -

        After looking at the network traces, it becomes clear that sonar-runner is trying to use the data coming back from the sonar server (which is presented in UTF8), but the data is converted into EBCDIC first (resulting in the broken characters).

        This is mainly due to the use of

        Reader reader = new InputStreamReader((InputStream) conn.getContent());

        relying on the default charset of the environment where sonar-runner executes instead of the charset that the sonar server use to send back the data.

        Ideally here you would want to see what charset is used in the comms between the client and server and use that instead of relying on the defaults.

        ---Jaco

        Show
        Jaco van Tonder added a comment - After looking at the network traces, it becomes clear that sonar-runner is trying to use the data coming back from the sonar server (which is presented in UTF8), but the data is converted into EBCDIC first (resulting in the broken characters). This is mainly due to the use of Reader reader = new InputStreamReader((InputStream) conn.getContent()); relying on the default charset of the environment where sonar-runner executes instead of the charset that the sonar server use to send back the data. Ideally here you would want to see what charset is used in the comms between the client and server and use that instead of relying on the defaults. ---Jaco
        Hide
        Jaco van Tonder added a comment -

        Patch to default to UTF-8 when communicationg with the sonar server.

        This is probably not ideal and we should rather look at the charset in use between client and server.

        This patch does, however, solve my problem.

        ---Jaco

        Show
        Jaco van Tonder added a comment - Patch to default to UTF-8 when communicationg with the sonar server. This is probably not ideal and we should rather look at the charset in use between client and server. This patch does, however, solve my problem. ---Jaco
        Hide
        Jaco van Tonder added a comment -

        Added a second patch that adds a method for figuring out the charset of the server response from the content-type header.

        ---Jaco

        Show
        Jaco van Tonder added a comment - Added a second patch that adds a method for figuring out the charset of the server response from the content-type header. ---Jaco
        Hide
        Julien HENRY added a comment -

        I have applied the proposed patch with some refactoring.

        Show
        Julien HENRY added a comment - I have applied the proposed patch with some refactoring.
        Hide
        Fabrice Bellingard added a comment -

        We obvioulsy can't test this bug fix as we don't have a mainframe

        Jaco van Tonder, could you please check that this snapshot version fixes the issue? (it includes your patch, refactored by Julien)

        Show
        Fabrice Bellingard added a comment - We obvioulsy can't test this bug fix as we don't have a mainframe Jaco van Tonder , could you please check that this snapshot version fixes the issue? (it includes your patch, refactored by Julien)
        Hide
        Jaco van Tonder added a comment -

        Hello guys,

        Apologies for the delay.

        I am busy doing some testing with the snapshot and will provide feedback soonest.

        Thank you,
        ---Jaco

        Show
        Jaco van Tonder added a comment - Hello guys, Apologies for the delay. I am busy doing some testing with the snapshot and will provide feedback soonest. Thank you, ---Jaco
        Hide
        Fabrice Bellingard added a comment - - edited

        Jaco van Tonder: any progress on the tests?

        Show
        Fabrice Bellingard added a comment - - edited Jaco van Tonder : any progress on the tests?
        Hide
        Fabrice Bellingard added a comment -

        I'm closing the ticket as the release is on its way. Feel free to reopen if there are any issues.

        Show
        Fabrice Bellingard added a comment - I'm closing the ticket as the release is on its way. Feel free to reopen if there are any issues.

          People

          • Assignee:
            Fabrice Bellingard
            Reporter:
            Jaco van Tonder
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: