Continuum
  1. Continuum
  2. CONTINUUM-2373

Distributed Master needs more detailed logging

    Details

    • Type: Improvement Improvement
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.4 (Beta)
    • Fix Version/s: 1.4.0 (Beta)
    • Component/s: Distributed Builds
    • Labels:
      None
    • Complexity:
      Intermediate
    • Number of attachments :
      0

      Description

      Watching the continuum.log file on the master with distributed builds enabled, I can't tell what projects are building or where they're going (to which agent).

      Here are some example log messages:

      2009-09-28 08:05:11,467 [btpool0-6] INFO org.apache.continuum.distributed.transport.slave.SlaveBuildAgentTransportClient - Checking if project group is in queue
      2009-09-28 08:05:11,620 [btpool0-6] INFO org.apache.continuum.distributed.transport.slave.SlaveBuildAgentTransportClient - Retrieving build size of agent
      2009-09-28 08:05:11,649 [pool-10-thread-1] INFO org.apache.continuum.builder.distributed.executor.DistributedBuildProjectTaskExecutor - initializing buildContext
      2009-09-28 08:05:13,380 [pool-10-thread-1] INFO org.apache.continuum.distributed.transport.slave.SlaveBuildAgentTransportClient - Building projects.
      2009-09-28 08:05:13,421 [btpool0-6] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportServer - Start prepare build.
      2009-09-28 08:05:24,892 [btpool0-6] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportServer - Start prepare build.
      2009-09-28 08:05:25,473 [pool-3-thread-1] INFO org.apache.continuum.distributed.transport.slave.SlaveBuildAgentTransportClient - Building projects.
      2009-09-28 08:05:25,491 [btpool0-7] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportServer - Start prepare build.
      2009-09-28 08:05:26,350 [btpool0-0] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportServer - Prepare build finished.
      2009-09-28 08:05:26,503 [btpool0-0] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportServer - Start prepare build.
      2009-09-28 08:05:26,507 [btpool0-7] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportServer - Retrieving environments
      2009-09-28 08:05:26,883 [btpool0-7] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportServer - Checking if project should build
      2009-09-28 08:05:27,080 [btpool0-7] INFO org.apache.continuum.builder.distributed.DefaultDistributedBuildService - The project build is forced, building
      2009-09-28 08:05:27,101 [btpool0-7] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportServer - Start project build.
      2009-09-28 08:05:27,368 [btpool0-7] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportServer - Start updating project
      2009-09-28 08:05:32,119 [btpool0-6] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportServer - Build result returned.

      These all need to say what project/group/agent is involved.

      It is not until a build finishes that you find out what project was building:
      2009-09-28 08:05:32,120 [btpool0-6] INFO org.apache.continuum.builder.distributed.DefaultDistributedBuildService - update build result of project '1'

        Activity

        Hide
        Brett Porter added a comment -

        sounds like something that needs fixing for general use?

        Show
        Brett Porter added a comment - sounds like something that needs fixing for general use?
        Hide
        Jevica Arianne B. Zurbano added a comment -

        Changes made in r884391:

        • added the IDs of the project/group/release in the logs both in master and slave
        • added more logs in some methods
        Show
        Jevica Arianne B. Zurbano added a comment - Changes made in r884391: added the IDs of the project/group/release in the logs both in master and slave added more logs in some methods
        Hide
        Maria Catherine Tan added a comment -

        Hi Jev,

        In 07:55:18,193, the projectId is 0 when it should be 21.

        2009-12-07 07:55:17,883 [btpool0-7] INFO org.apache.continuum.distributed.transport.slave.SlaveBuildAgentTransportServer - Checking if project group '16' is in queue
        2009-12-07 07:55:18,021 [btpool0-7] INFO org.apache.continuum.distributed.transport.slave.SlaveBuildAgentTransportServer - Building projects.
        2009-12-07 07:55:18,047 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Check scm root state
        2009-12-07 07:55:18,047 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Starting prepare build
        2009-12-07 07:55:18,193 [pool-2-thread-1] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportClient - Start prepare build for project (projectId=0)
        2009-12-07 07:55:18,193 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Initializing prepare build
        2009-12-07 07:55:18,202 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Updating working directory
        2009-12-07 07:55:18,202 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Performing action check-agent-working-directory
        2009-12-07 07:55:18,205 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Performing action checkout-agent-project
        2009-12-07 07:55:18,207 [pool-2-thread-1] INFO action#checkout-agent-project - Checking out project: 'Maven Quick Start Archetype', id: '21' to '/home/marica/maestro/apache-continuum-buildagent-1.3.5-maestro-2.3.0-m2/data/working-dire$
        2009-12-07 07:55:18,228 [pool-2-thread-1] INFO scmManager - Executing: /bin/sh -c cd /home/marica/maestro/apache-continuum-buildagent-1.3.5-maestro-2.3.0-m2/data/working-directory && svn --non-interactive checkout http://stan.manila.e$
        2009-12-07 07:55:18,229 [pool-2-thread-1] INFO scmManager - Working directory: /home/marica/maestro/apache-continuum-buildagent-1.3.5-maestro-2.3.0-m2/data/working-directory
        2009-12-07 07:55:19,104 [pool-2-thread-1] INFO action#checkout-agent-project - Checked out 4 files.
        2009-12-07 07:55:19,105 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Performing action changelog-agent-project
        2009-12-07 07:55:19,111 [pool-2-thread-1] INFO action#changelog-agent-project - Getting changeLog of project: Maven Quick Start Archetype
        2009-12-07 07:55:19,163 [pool-2-thread-1] INFO scmManager - Executing: /bin/sh -c cd /home/marica/maestro/apache-continuum-buildagent-1.3.5-maestro-2.3.0-m2/data/working-directory/21 && svn --non-interactive log -v http://stan.manila.$
        2009-12-07 07:55:19,163 [pool-2-thread-1] INFO scmManager - Working directory: /home/marica/maestro/apache-continuum-buildagent-1.3.5-maestro-2.3.0-m2/data/working-directory/21
        2009-12-07 07:55:24,813 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Merging SCM results
        2009-12-07 07:55:24,898 [pool-2-thread-1] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportClient - Prepare build finished for project (projectId=0).
        2009-12-07 07:55:24,899 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Performing action create-agent-build-project-task
        2009-12-07 07:55:24,906 [pool-1-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.BuildProjectTaskExecutor - Initializing build
        2009-12-07 07:55:24,928 [pool-1-thread-1] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportClient - Retrieved environments. buildDefinitionId=31, installationType=maven2
        2009-12-07 07:55:24,928 [pool-1-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.BuildProjectTaskExecutor - Checking if project 'Maven Quick Start Archetype' should build
        2009-12-07 07:55:25,220 [pool-1-thread-1] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportClient - Checking if project (projectId=21) should build
        2009-12-07 07:55:25,220 [pool-1-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.BuildProjectTaskExecutor - Starting build of Maven Quick Start Archetype
        2009-12-07 07:55:25,234 [pool-1-thread-1] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportClient - Return project currently building, projectId=21
        2009-12-07 07:55:25,234 [pool-1-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.BuildProjectTaskExecutor - Performing action update-project-from-agent-working-directory
        2009-12-07 07:55:25,236 [pool-1-thread-1] INFO org.apache.continuum.buildagent.action.UpdateProjectFromWorkingDirectoryAction - Updating project 'Maven Quick Start Archetype' from checkout.
        2009-12-07 07:55:25,372 [pool-1-thread-1] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportClient - Updating project Maven Quick Start Archetype (projectId=21)
        2009-12-07 07:55:25,372 [pool-1-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.BuildProjectTaskExecutor - Performing action execute-agent-builder
        2009-12-07 07:55:25,388 [pool-1-thread-1] INFO org.apache.continuum.utils.shell.DefaultShellCommandHelper - Executing: /bin/sh -c cd /home/marica/maestro/apache-continuum-buildagent-1.3.5-maestro-2.3.0-m2/data/working-directory/21 && $
        2009-12-07 07:55:25,388 [pool-1-thread-1] INFO org.apache.continuum.utils.shell.DefaultShellCommandHelper - Working directory: /home/marica/maestro/apache-continuum-buildagent-1.3.5-maestro-2.3.0-m2/data/working-directory/21
        2009-12-07 07:55:29,978 [pool-1-thread-1] INFO org.apache.continuum.buildagent.build.execution.AbstractBuildExecutor - Exit code: 0
        2009-12-07 07:55:30,097 [pool-1-thread-1] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportClient - Returning the build result for project (projectId=21).

        Show
        Maria Catherine Tan added a comment - Hi Jev, In 07:55:18,193, the projectId is 0 when it should be 21. 2009-12-07 07:55:17,883 [btpool0-7] INFO org.apache.continuum.distributed.transport.slave.SlaveBuildAgentTransportServer - Checking if project group '16' is in queue 2009-12-07 07:55:18,021 [btpool0-7] INFO org.apache.continuum.distributed.transport.slave.SlaveBuildAgentTransportServer - Building projects. 2009-12-07 07:55:18,047 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Check scm root state 2009-12-07 07:55:18,047 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Starting prepare build 2009-12-07 07:55:18,193 [pool-2-thread-1] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportClient - Start prepare build for project (projectId=0) 2009-12-07 07:55:18,193 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Initializing prepare build 2009-12-07 07:55:18,202 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Updating working directory 2009-12-07 07:55:18,202 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Performing action check-agent-working-directory 2009-12-07 07:55:18,205 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Performing action checkout-agent-project 2009-12-07 07:55:18,207 [pool-2-thread-1] INFO action#checkout-agent-project - Checking out project: 'Maven Quick Start Archetype', id: '21' to '/home/marica/maestro/apache-continuum-buildagent-1.3.5-maestro-2.3.0-m2/data/working-dire$ 2009-12-07 07:55:18,228 [pool-2-thread-1] INFO scmManager - Executing: /bin/sh -c cd /home/marica/maestro/apache-continuum-buildagent-1.3.5-maestro-2.3.0-m2/data/working-directory && svn --non-interactive checkout http://stan.manila.e$ 2009-12-07 07:55:18,229 [pool-2-thread-1] INFO scmManager - Working directory: /home/marica/maestro/apache-continuum-buildagent-1.3.5-maestro-2.3.0-m2/data/working-directory 2009-12-07 07:55:19,104 [pool-2-thread-1] INFO action#checkout-agent-project - Checked out 4 files. 2009-12-07 07:55:19,105 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Performing action changelog-agent-project 2009-12-07 07:55:19,111 [pool-2-thread-1] INFO action#changelog-agent-project - Getting changeLog of project: Maven Quick Start Archetype 2009-12-07 07:55:19,163 [pool-2-thread-1] INFO scmManager - Executing: /bin/sh -c cd /home/marica/maestro/apache-continuum-buildagent-1.3.5-maestro-2.3.0-m2/data/working-directory/21 && svn --non-interactive log -v http://stan.manila.$ 2009-12-07 07:55:19,163 [pool-2-thread-1] INFO scmManager - Working directory: /home/marica/maestro/apache-continuum-buildagent-1.3.5-maestro-2.3.0-m2/data/working-directory/21 2009-12-07 07:55:24,813 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Merging SCM results 2009-12-07 07:55:24,898 [pool-2-thread-1] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportClient - Prepare build finished for project (projectId=0). 2009-12-07 07:55:24,899 [pool-2-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.PrepareBuildProjectsTaskExecutor - Performing action create-agent-build-project-task 2009-12-07 07:55:24,906 [pool-1-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.BuildProjectTaskExecutor - Initializing build 2009-12-07 07:55:24,928 [pool-1-thread-1] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportClient - Retrieved environments. buildDefinitionId=31, installationType=maven2 2009-12-07 07:55:24,928 [pool-1-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.BuildProjectTaskExecutor - Checking if project 'Maven Quick Start Archetype' should build 2009-12-07 07:55:25,220 [pool-1-thread-1] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportClient - Checking if project (projectId=21) should build 2009-12-07 07:55:25,220 [pool-1-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.BuildProjectTaskExecutor - Starting build of Maven Quick Start Archetype 2009-12-07 07:55:25,234 [pool-1-thread-1] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportClient - Return project currently building, projectId=21 2009-12-07 07:55:25,234 [pool-1-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.BuildProjectTaskExecutor - Performing action update-project-from-agent-working-directory 2009-12-07 07:55:25,236 [pool-1-thread-1] INFO org.apache.continuum.buildagent.action.UpdateProjectFromWorkingDirectoryAction - Updating project 'Maven Quick Start Archetype' from checkout. 2009-12-07 07:55:25,372 [pool-1-thread-1] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportClient - Updating project Maven Quick Start Archetype (projectId=21) 2009-12-07 07:55:25,372 [pool-1-thread-1] INFO org.apache.continuum.buildagent.taskqueue.execution.BuildProjectTaskExecutor - Performing action execute-agent-builder 2009-12-07 07:55:25,388 [pool-1-thread-1] INFO org.apache.continuum.utils.shell.DefaultShellCommandHelper - Executing: /bin/sh -c cd /home/marica/maestro/apache-continuum-buildagent-1.3.5-maestro-2.3.0-m2/data/working-directory/21 && $ 2009-12-07 07:55:25,388 [pool-1-thread-1] INFO org.apache.continuum.utils.shell.DefaultShellCommandHelper - Working directory: /home/marica/maestro/apache-continuum-buildagent-1.3.5-maestro-2.3.0-m2/data/working-directory/21 2009-12-07 07:55:29,978 [pool-1-thread-1] INFO org.apache.continuum.buildagent.build.execution.AbstractBuildExecutor - Exit code: 0 2009-12-07 07:55:30,097 [pool-1-thread-1] INFO org.apache.continuum.distributed.transport.master.MasterBuildAgentTransportClient - Returning the build result for project (projectId=21).
        Hide
        Jevica Arianne B. Zurbano added a comment -

        Thanks Marica for pointing that out.

        I have made the modifications in r887850.

        Show
        Jevica Arianne B. Zurbano added a comment - Thanks Marica for pointing that out. I have made the modifications in r887850.

          People

          • Assignee:
            Jevica Arianne B. Zurbano
            Reporter:
            Wendy Smoak
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: