Continuum
  1. Continuum
  2. CONTINUUM-2704

Shell builds are not executing concurrently

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.8
    • Fix Version/s: 1.4.2
    • Component/s: Parallel Builds
    • Labels:
      None
    • Environment:
      Ubuntu, Tomcat
    • Complexity:
      Intermediate
    • Number of attachments :
      0

      Description

      Continuum version 1.3.6. Deployed on Tomcat in Ubuntu.

      I have 2 projects, say A and B. Each has shell builds defined and set to run in different queues. Number of parallel builds is set to 4.

      The problem is that, when the shell build from A runs, it will not execute the build until the shell build from B is complete. it is almost like A is waiting for B to finish. Please note that maven builds are happily running in parallel in these 2 queues. It is only the shell build that has a problem.

      To be sure that this is not an OS level problem, I started the build in B from Continuum and ran the shell script for A from command line and it worked fine - both ran in parallel. So, I think Continuum has a bottle neck somewhere. please help.

        Activity

        Hide
        Murali Mohan added a comment -

        Log analysis and some observations:
        ==================================

        The continuum logs when 2 shell builds attempt to run in parallel, is given below. Lines with "MURALI:" are inserted manually by me to label what is happening, in my understanding.

        The last log statement before the second build gets stuck is:
        2013-03-01 13:59:40,825 [pool-7-thread-17] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Performing action execute-builder

        The logs from the DefaultShellCommandHelper is missing. This means that the second build is not even reaching the DefaultShellCommandHelper. This should confirm that this is a continuum bug and not related to my installation setup.

        Logs:
        =========

        *****MURALI: 1st build STart ******

        2013-03-01 13:58:50,534 [http-9000-Processor16] INFO org.apache.continuum.buildmanager.ParallelBuildsManager - Queueing prepare-build-project task 'org.apache.continuum.taskqueue.PrepareBuildProjectsTask@186' to prepare-build queue.
        2013-03-01 13:58:50,534 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Initializing prepare build
        2013-03-01 13:58:50,559 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Starting prepare build of project: Cleandb_Backend_Maint
        2013-03-01 13:58:50,598 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Purging existing working copy
        2013-03-01 13:58:50,598 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Performing action clean-working-directory
        2013-03-01 13:58:53,149 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Updating working dir
        2013-03-01 13:58:53,149 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Performing action check-working-directory
        2013-03-01 13:58:53,161 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Performing action checkout-project
        2013-03-01 13:58:53,278 [pool-13-thread-1] INFO action#checkout-project - Checking out project: 'Cleandb_Backend_Maint', id: '77' to '/home/continuum/continuumDirectories/trunk/working-directory/77'.
        2013-03-01 13:58:53,293 [pool-13-thread-1] INFO org.apache.continuum.scm.manager.Slf4jScmLogger - Executing: /bin/sh -c cd /home/continuum/continuumDirectories/trunk/working-directory && svn --username v963061 --password '*****' --non-interactive checkout http://labscvs.superpages.com:7151/BidCenter/bidcenter/branches/BC_72_MAINT 77
        2013-03-01 13:58:53,293 [pool-13-thread-1] INFO org.apache.continuum.scm.manager.Slf4jScmLogger - Working directory: /home/continuum/continuumDirectories/trunk/working-directory
        2013-03-01 13:59:40,600 [pool-13-thread-1] INFO action#checkout-project - Checked out 2760 files.
        2013-03-01 13:59:40,642 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Merging SCM results
        2013-03-01 13:59:40,642 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Ending prepare build of project: Cleandb_Backend_Maint
        2013-03-01 13:59:40,642 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Ending prepare build
        2013-03-01 13:59:40,663 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Performing action create-build-project-task
        2013-03-01 13:59:40,738 [pool-13-thread-1] INFO org.apache.continuum.buildmanager.ParallelBuildsManager - Project '77' - 'Cleandb_Backend_Maint' added to overall build queue 'MAINT_BACKEND_QUEUE'.
        2013-03-01 13:59:40,739 [pool-7-thread-17] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Initializing build
        2013-03-01 13:59:40,758 [pool-7-thread-17] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Starting build of Cleandb_Backend_Maint
        2013-03-01 13:59:40,820 [pool-7-thread-17] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - FreshBuild configured, building
        2013-03-01 13:59:40,820 [pool-7-thread-17] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Performing action update-project-from-working-directory
        2013-03-01 13:59:40,822 [pool-7-thread-17] INFO action#update-project-from-working-directory - Updating project 'Cleandb_Backend_Maint' from checkout.
        2013-03-01 13:59:40,825 [pool-7-thread-17] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Performing action execute-builder

        *****1st build executing statements below ******
        2013-03-01 13:59:41,000 [pool-7-thread-17] INFO org.apache.continuum.utils.shell.DefaultShellCommandHelper - Executing: /bin/sh -c cd /home/continuum/continuumDirectories/trunk/working-directory/77 && cleandb2Continuum_maint_backend.sh
        2013-03-01 13:59:41,000 [pool-7-thread-17] INFO org.apache.continuum.utils.shell.DefaultShellCommandHelper - Working directory: /home/continuum/continuumDirectories/trunk/working-directory/77
        2013-03-01 13:59:54,771 [http-9000-Processor23] INFO org.apache.continuum.buildmanager.ParallelBuildsManager - Queueing prepare-build-project task 'org.apache.continuum.taskqueue.PrepareBuildProjectsTask@198' to prepare-build queue.

        *****MURALI: 2nd build start ******
        2013-03-01 13:59:54,799 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Initializing prepare build
        2013-03-01 13:59:54,823 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Starting prepare build of project: Cleandb_Backend_trunk
        2013-03-01 13:59:54,874 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Updating working dir
        2013-03-01 13:59:54,874 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Performing action check-working-directory
        2013-03-01 13:59:54,875 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Performing action update-working-directory-from-scm
        2013-03-01 13:59:54,968 [pool-13-thread-1] INFO action#update-working-directory-from-scm - Updating project: Cleandb_Backend_trunk', id: '76' to '/home/continuum/continuumDirectories/trunk/working-directory/76'.
        v963061 --password '*****' --non-interactive update /home/continuum/continuumDirectories/trunk/working-directory/76
        2013-03-01 13:59:54,970 [pool-13-thread-1] INFO org.apache.continuum.scm.manager.Slf4jScmLogger - Working directory: /home/continuum/continuumDirectories/trunk/working-directory/76
        2013-03-01 13:59:57,887 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Merging SCM results
        2013-03-01 13:59:57,887 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Ending prepare build of project: Cleandb_Backend_trunk
        2013-03-01 13:59:57,887 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Ending prepare build
        2013-03-01 13:59:57,944 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Performing action create-build-project-task
        2013-03-01 13:59:57,962 [pool-13-thread-1] INFO org.apache.continuum.buildmanager.ParallelBuildsManager - Project '76' - 'Cleandb_Backend_trunk' added to overall build queue 'TRUNK_BACKEND_QUEUE'.
        2013-03-01 13:59:57,962 [pool-5-thread-16] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Initializing build
        2013-03-01 13:59:57,980 [pool-5-thread-16] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Starting build of Cleandb_Backend_trunk
        2013-03-01 13:59:58,009 [pool-5-thread-16] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - AlwaysBuild configured, building
        2013-03-01 13:59:58,009 [pool-5-thread-16] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Performing action update-project-from-working-directory
        2013-03-01 13:59:58,011 [pool-5-thread-16] INFO action#update-project-from-working-directory - Updating project 'Cleandb_Backend_trunk' from checkout.
        2013-03-01 13:59:58,014 [pool-5-thread-16] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Performing action execute-builder

        *****MURALI: 2nd build executing statements MISSING FOR SOME TIME, it is waiting for 1st build to finish ******
        2013-03-01 14:01:20,730 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value.
        2013-03-01 14:01:20,836 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value.
        2013-03-01 14:01:20,845 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value.
        2013-03-01 14:01:20,861 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value.
        2013-03-01 14:01:20,879 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value.
        2013-03-01 14:01:20,879 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value.
        2013-03-01 14:01:20,879 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'removePrepareBuildEntries.action' in namespace: ''. Form action defaulting to 'action' attribute's literal value.
        2013-03-01 14:01:20,879 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'removePrepareBuildEntries.action' in namespace: ''. Form action defaulting to 'action' attribute's literal value.
        2013-03-01 14:01:54,204 [pool-7-thread-17] INFO org.apache.maven.continuum.execution.shell.ShellBuildExecutor - Exit code: 0

        *****MURALI: 2nd build executing statements NOW, after 1st build ends ******
        2013-03-01 14:01:54,206 [pool-5-thread-16] INFO org.apache.continuum.utils.shell.DefaultShellCommandHelper - Executing: /bin/sh -c cd /home/continuum/continuumDirectories/trunk/working-directory/76 && cleandb2Continuum_trunk_backend.sh
        2013-03-01 14:01:54,206 [pool-5-thread-16] INFO org.apache.continuum.utils.shell.DefaultShellCommandHelper - Working directory: /home/continuum/continuumDirectories/trunk/working-directory/76
        2013-03-01 14:01:54,484 [pool-7-thread-17] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Performing action deploy-artifact
        2013-03-01 14:02:21,697 [http-9000-Processor23] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value.
        2013-03-01 14:02:21,699 [http-9000-Processor23] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value.
        2013-03-01 14:02:21,699 [http-9000-Processor23] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value.
        2013-03-01 14:02:21,700 [http-9000-Processor23] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value.
        2013-03-01 14:02:21,700 [http-9000-Processor23] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value.
        2013-03-01 14:02:21,700 [http-9000-Processor23] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value.
        2013-03-01 14:02:21,700 [http-9000-Processor23] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'removePrepareBuildEntries.action' in namespace: ''. Form action defaulting to 'action' attribute's literal value.
        2013-03-01 14:02:21,700 [http-9000-Processor23] WARN org.apache.s

        Show
        Murali Mohan added a comment - Log analysis and some observations: ================================== The continuum logs when 2 shell builds attempt to run in parallel, is given below. Lines with "MURALI:" are inserted manually by me to label what is happening, in my understanding. The last log statement before the second build gets stuck is: 2013-03-01 13:59:40,825 [pool-7-thread-17] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Performing action execute-builder The logs from the DefaultShellCommandHelper is missing. This means that the second build is not even reaching the DefaultShellCommandHelper. This should confirm that this is a continuum bug and not related to my installation setup. Logs: ========= *****MURALI: 1st build STart ****** 2013-03-01 13:58:50,534 [http-9000-Processor16] INFO org.apache.continuum.buildmanager.ParallelBuildsManager - Queueing prepare-build-project task 'org.apache.continuum.taskqueue.PrepareBuildProjectsTask@186' to prepare-build queue. 2013-03-01 13:58:50,534 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Initializing prepare build 2013-03-01 13:58:50,559 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Starting prepare build of project: Cleandb_Backend_Maint 2013-03-01 13:58:50,598 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Purging existing working copy 2013-03-01 13:58:50,598 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Performing action clean-working-directory 2013-03-01 13:58:53,149 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Updating working dir 2013-03-01 13:58:53,149 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Performing action check-working-directory 2013-03-01 13:58:53,161 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Performing action checkout-project 2013-03-01 13:58:53,278 [pool-13-thread-1] INFO action#checkout-project - Checking out project: 'Cleandb_Backend_Maint', id: '77' to '/home/continuum/continuumDirectories/trunk/working-directory/77'. 2013-03-01 13:58:53,293 [pool-13-thread-1] INFO org.apache.continuum.scm.manager.Slf4jScmLogger - Executing: /bin/sh -c cd /home/continuum/continuumDirectories/trunk/working-directory && svn --username v963061 --password '*****' --non-interactive checkout http://labscvs.superpages.com:7151/BidCenter/bidcenter/branches/BC_72_MAINT 77 2013-03-01 13:58:53,293 [pool-13-thread-1] INFO org.apache.continuum.scm.manager.Slf4jScmLogger - Working directory: /home/continuum/continuumDirectories/trunk/working-directory 2013-03-01 13:59:40,600 [pool-13-thread-1] INFO action#checkout-project - Checked out 2760 files. 2013-03-01 13:59:40,642 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Merging SCM results 2013-03-01 13:59:40,642 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Ending prepare build of project: Cleandb_Backend_Maint 2013-03-01 13:59:40,642 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Ending prepare build 2013-03-01 13:59:40,663 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Performing action create-build-project-task 2013-03-01 13:59:40,738 [pool-13-thread-1] INFO org.apache.continuum.buildmanager.ParallelBuildsManager - Project '77' - 'Cleandb_Backend_Maint' added to overall build queue 'MAINT_BACKEND_QUEUE'. 2013-03-01 13:59:40,739 [pool-7-thread-17] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Initializing build 2013-03-01 13:59:40,758 [pool-7-thread-17] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Starting build of Cleandb_Backend_Maint 2013-03-01 13:59:40,820 [pool-7-thread-17] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - FreshBuild configured, building 2013-03-01 13:59:40,820 [pool-7-thread-17] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Performing action update-project-from-working-directory 2013-03-01 13:59:40,822 [pool-7-thread-17] INFO action#update-project-from-working-directory - Updating project 'Cleandb_Backend_Maint' from checkout. 2013-03-01 13:59:40,825 [pool-7-thread-17] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Performing action execute-builder *****1st build executing statements below ****** 2013-03-01 13:59:41,000 [pool-7-thread-17] INFO org.apache.continuum.utils.shell.DefaultShellCommandHelper - Executing: /bin/sh -c cd /home/continuum/continuumDirectories/trunk/working-directory/77 && cleandb2Continuum_maint_backend.sh 2013-03-01 13:59:41,000 [pool-7-thread-17] INFO org.apache.continuum.utils.shell.DefaultShellCommandHelper - Working directory: /home/continuum/continuumDirectories/trunk/working-directory/77 2013-03-01 13:59:54,771 [http-9000-Processor23] INFO org.apache.continuum.buildmanager.ParallelBuildsManager - Queueing prepare-build-project task 'org.apache.continuum.taskqueue.PrepareBuildProjectsTask@198' to prepare-build queue. *****MURALI: 2nd build start ****** 2013-03-01 13:59:54,799 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Initializing prepare build 2013-03-01 13:59:54,823 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Starting prepare build of project: Cleandb_Backend_trunk 2013-03-01 13:59:54,874 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Updating working dir 2013-03-01 13:59:54,874 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Performing action check-working-directory 2013-03-01 13:59:54,875 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Performing action update-working-directory-from-scm 2013-03-01 13:59:54,968 [pool-13-thread-1] INFO action#update-working-directory-from-scm - Updating project: Cleandb_Backend_trunk', id: '76' to '/home/continuum/continuumDirectories/trunk/working-directory/76'. v963061 --password '*****' --non-interactive update /home/continuum/continuumDirectories/trunk/working-directory/76 2013-03-01 13:59:54,970 [pool-13-thread-1] INFO org.apache.continuum.scm.manager.Slf4jScmLogger - Working directory: /home/continuum/continuumDirectories/trunk/working-directory/76 2013-03-01 13:59:57,887 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Merging SCM results 2013-03-01 13:59:57,887 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Ending prepare build of project: Cleandb_Backend_trunk 2013-03-01 13:59:57,887 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Ending prepare build 2013-03-01 13:59:57,944 [pool-13-thread-1] INFO org.apache.maven.continuum.scm.queue.PrepareBuildProjectsTaskExecutor - Performing action create-build-project-task 2013-03-01 13:59:57,962 [pool-13-thread-1] INFO org.apache.continuum.buildmanager.ParallelBuildsManager - Project '76' - 'Cleandb_Backend_trunk' added to overall build queue 'TRUNK_BACKEND_QUEUE'. 2013-03-01 13:59:57,962 [pool-5-thread-16] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Initializing build 2013-03-01 13:59:57,980 [pool-5-thread-16] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Starting build of Cleandb_Backend_trunk 2013-03-01 13:59:58,009 [pool-5-thread-16] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - AlwaysBuild configured, building 2013-03-01 13:59:58,009 [pool-5-thread-16] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Performing action update-project-from-working-directory 2013-03-01 13:59:58,011 [pool-5-thread-16] INFO action#update-project-from-working-directory - Updating project 'Cleandb_Backend_trunk' from checkout. 2013-03-01 13:59:58,014 [pool-5-thread-16] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Performing action execute-builder *****MURALI: 2nd build executing statements MISSING FOR SOME TIME, it is waiting for 1st build to finish ****** 2013-03-01 14:01:20,730 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value. 2013-03-01 14:01:20,836 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value. 2013-03-01 14:01:20,845 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value. 2013-03-01 14:01:20,861 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value. 2013-03-01 14:01:20,879 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value. 2013-03-01 14:01:20,879 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value. 2013-03-01 14:01:20,879 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'removePrepareBuildEntries.action' in namespace: ''. Form action defaulting to 'action' attribute's literal value. 2013-03-01 14:01:20,879 [http-9000-Processor16] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'removePrepareBuildEntries.action' in namespace: ''. Form action defaulting to 'action' attribute's literal value. 2013-03-01 14:01:54,204 [pool-7-thread-17] INFO org.apache.maven.continuum.execution.shell.ShellBuildExecutor - Exit code: 0 *****MURALI: 2nd build executing statements NOW, after 1st build ends ****** 2013-03-01 14:01:54,206 [pool-5-thread-16] INFO org.apache.continuum.utils.shell.DefaultShellCommandHelper - Executing: /bin/sh -c cd /home/continuum/continuumDirectories/trunk/working-directory/76 && cleandb2Continuum_trunk_backend.sh 2013-03-01 14:01:54,206 [pool-5-thread-16] INFO org.apache.continuum.utils.shell.DefaultShellCommandHelper - Working directory: /home/continuum/continuumDirectories/trunk/working-directory/76 2013-03-01 14:01:54,484 [pool-7-thread-17] INFO org.apache.maven.continuum.buildcontroller.DefaultBuildController - Performing action deploy-artifact 2013-03-01 14:02:21,697 [http-9000-Processor23] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value. 2013-03-01 14:02:21,699 [http-9000-Processor23] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value. 2013-03-01 14:02:21,699 [http-9000-Processor23] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value. 2013-03-01 14:02:21,700 [http-9000-Processor23] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value. 2013-03-01 14:02:21,700 [http-9000-Processor23] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value. 2013-03-01 14:02:21,700 [http-9000-Processor23] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'none' in namespace: ''. Form action defaulting to 'action' attribute's literal value. 2013-03-01 14:02:21,700 [http-9000-Processor23] WARN org.apache.struts2.components.Form - No configuration found for the specified action: 'removePrepareBuildEntries.action' in namespace: ''. Form action defaulting to 'action' attribute's literal value. 2013-03-01 14:02:21,700 [http-9000-Processor23] WARN org.apache.s
        Hide
        Murali Mohan added a comment -

        I think this is the issue. In class ShellBuildExecutor.java in continuum-core project, I see the following method as synchronized, that is the problem.

        public synchronized ContinuumBuildExecutionResult build( Project project, BuildDefinition buildDefinition,
                                                                 File buildOutput )
            throws ContinuumBuildExecutorException
        {
            // TODO: this should be validated earlier?
        	
        	log.info("MUR: build");
        	
            String executable = getBuildFileForProject( project, buildDefinition );
        
            return executeShellCommand( project, executable, buildDefinition.getArguments(), buildOutput,
                                        getEnvironments( buildDefinition ) );
        }
        
        Show
        Murali Mohan added a comment - I think this is the issue. In class ShellBuildExecutor.java in continuum-core project, I see the following method as synchronized, that is the problem. public synchronized ContinuumBuildExecutionResult build( Project project, BuildDefinition buildDefinition, File buildOutput ) throws ContinuumBuildExecutorException { // TODO: this should be validated earlier? log.info( "MUR: build" ); String executable = getBuildFileForProject( project, buildDefinition ); return executeShellCommand( project, executable, buildDefinition.getArguments(), buildOutput, getEnvironments( buildDefinition ) ); }
        Hide
        Murali Mohan added a comment -

        Other Execuotr implementations do not have synchronized keyword, may be we can remove this?

        Show
        Murali Mohan added a comment - Other Execuotr implementations do not have synchronized keyword, may be we can remove this?
        Hide
        Brett Porter added a comment -

        That certainly looks like the case, good catch. It would be necessary to confirm that the rest of the side effects of those methods are thread safe, but a narrower synchronization window can likely be established if needed that doesn't lock them to one at a time. Thanks for working through this.

        Show
        Brett Porter added a comment - That certainly looks like the case, good catch. It would be necessary to confirm that the rest of the side effects of those methods are thread safe, but a narrower synchronization window can likely be established if needed that doesn't lock them to one at a time. Thanks for working through this.
        Hide
        Murali Mohan added a comment -

        It is threadsafe, looks like that to me.

        The only 2 variables I wanted to dig deep are defaultExecutable and resolveExecutable in AbstractBuildExecutor. These variables are not set for ShellBuildExecutor where as it is being set ofrant and maven. Any ideas why?

        Anyways, I think we can safely make it a non-synchronized method.

        Show
        Murali Mohan added a comment - It is threadsafe, looks like that to me. The only 2 variables I wanted to dig deep are defaultExecutable and resolveExecutable in AbstractBuildExecutor. These variables are not set for ShellBuildExecutor where as it is being set ofrant and maven. Any ideas why? Anyways, I think we can safely make it a non-synchronized method.
        Hide
        Murali Mohan added a comment -

        So I have made the change (removed the synchronize keyword) in the code I got for version 1.3.6 (from the corresponding branch). I couldn't get the trunk code to work which is explained here - http://jira.codehaus.org/browse/CONTINUUM-2705

        And this 1.3.6 version has been running and executing all our builds for more than a week now. It is much faster and I am seeing less problems now. Initially, I used to have a load of problems because of builds not getting completed on time as the shell builds were blocking each other. Not any more.

        So works well for me in 1.3.6

        Show
        Murali Mohan added a comment - So I have made the change (removed the synchronize keyword) in the code I got for version 1.3.6 (from the corresponding branch). I couldn't get the trunk code to work which is explained here - http://jira.codehaus.org/browse/CONTINUUM-2705 And this 1.3.6 version has been running and executing all our builds for more than a week now. It is much faster and I am seeing less problems now. Initially, I used to have a load of problems because of builds not getting completed on time as the shell builds were blocking each other. Not any more. So works well for me in 1.3.6
        Hide
        Brett Porter added a comment -

        Thanks Murali, we should make sure this is applied to 1.4.2.

        Show
        Brett Porter added a comment - Thanks Murali, we should make sure this is applied to 1.4.2.
        Hide
        Brent N Atkinson added a comment -

        That synchronized indeed looks suspect. As Murali said, it isn't consistent with the other build executors, which is odd considering that it is essentially doing the same thing (less actually) than the other executors.

        • Added another build queue
        • Set parallel builds = 2
        • Added the extra queue to the default schedule
        • Changed the cron expression to run every minute
        • Added jobs that do nothing but sleep for 1 minute
        • Made scm changes to cause the builds to be triggered

        After removing synchronized from the executor, I was able to see them simultaneously in the OS process list.

        Show
        Brent N Atkinson added a comment - That synchronized indeed looks suspect. As Murali said, it isn't consistent with the other build executors, which is odd considering that it is essentially doing the same thing (less actually) than the other executors. Added another build queue Set parallel builds = 2 Added the extra queue to the default schedule Changed the cron expression to run every minute Added jobs that do nothing but sleep for 1 minute Made scm changes to cause the builds to be triggered After removing synchronized from the executor, I was able to see them simultaneously in the OS process list.
        Hide
        Brent N Atkinson added a comment -

        Fixed in r1490283

        Show
        Brent N Atkinson added a comment - Fixed in r1490283

          People

          • Assignee:
            Brent N Atkinson
            Reporter:
            Murali Mohan
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: