Details

    • Type: New Feature New Feature
    • Status: Closed Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3.2 (Beta)
    • Component/s: Distributed Builds
    • Labels:
      None
    • Complexity:
      Intermediate
    • Number of attachments :
      4

      Description

      from the discussion : http://www.nabble.com/Missing-Functionalities-in-Distributed-Builds-td21697552.html

      We can have the user select which agent to add in a group and have that
      group attached to a build environment? So instead of agent attached to a
      build environment, it will be group of agents.

      1. CONTINUUM-2068.patch
        51 kB
        jan ancajas
      2. CONTINUUM-2068-2.patch
        72 kB
        jan ancajas
      3. CONTINUUM-2068-3.patch
        97 kB
        jan ancajas
      4. CONTINUUM-2068-4.patch
        104 kB
        jan ancajas

        Issue Links

          Activity

          Hide
          jan ancajas added a comment -

          attach the first part of the solution. build agent group function.

          Show
          jan ancajas added a comment - attach the first part of the solution. build agent group function.
          Hide
          jan ancajas added a comment -

          attach patch for review (build agent group+selection of agents). added new field "build agent group" to the profile table.

          Show
          jan ancajas added a comment - attach patch for review (build agent group+selection of agents). added new field "build agent group" to the profile table.
          Hide
          jan ancajas added a comment -

          added patch again.

          • fixed selection of agents algorithm.
          • added an overall task-queue-executor for distributed build queue to delegate the task.
          • implemented a 1 queue 1 executor approach for every build agent
          • for the scenario where the current task in the distributed build queue doesn't have a matching build agent (e.g. build agent of that particular group is disabled ), i'll just add back the task at the end of the queue so as not to block the other tasks in line. in this way, when the matching build agent is found in the future, the task will be processed.
          • rule of selecting the build agent in the group : it will return the first non-busy agent from the group , else it will return the first agent in the group.
          • refactors:
            move DistributedBuildTaskExecutor, DistributedBuildTaskQueueExecutor to continuum-api.
          Show
          jan ancajas added a comment - added patch again. fixed selection of agents algorithm. added an overall task-queue-executor for distributed build queue to delegate the task. implemented a 1 queue 1 executor approach for every build agent for the scenario where the current task in the distributed build queue doesn't have a matching build agent (e.g. build agent of that particular group is disabled ), i'll just add back the task at the end of the queue so as not to block the other tasks in line. in this way, when the matching build agent is found in the future, the task will be processed. rule of selecting the build agent in the group : it will return the first non-busy agent from the group , else it will return the first agent in the group. refactors: move DistributedBuildTaskExecutor, DistributedBuildTaskQueueExecutor to continuum-api.
          Hide
          jan ancajas added a comment -

          updates:

          • use version 1.1.4+ for modello version
          • added Distributed Build Queue view in Build Queue page.
          Show
          jan ancajas added a comment - updates: use version 1.1.4+ for modello version added Distributed Build Queue view in Build Queue page.
          Hide
          Maria Catherine Tan added a comment -

          Fixed in revision 748163! Thanks Jan great work

          Modified some lines on the patch:

          • format
          • fix build queue not showing in queues page
          • fix blank page when adding build agent group with no build agents available
          Show
          Maria Catherine Tan added a comment - Fixed in revision 748163! Thanks Jan great work Modified some lines on the patch: format fix build queue not showing in queues page fix blank page when adding build agent group with no build agents available
          Hide
          Wendy Smoak added a comment -

          I've just tried this out and have some questions...

          It seems that an agent can be added to more than one group. Is that intended?

          What does it mean for a build agent to be in no group(s) at all?

          I'm having trouble with the workflow involved in getting my first distributed build to happen. Now that we have these agent groups, it seems like you can't rely on the default build environment (none selected, just whatever Continuum is running as.) That means that logically, the default build definition, which does not have a build environment selected, will not actually do anything.

          (Except that it does... I see activity in agent-1's logs, but I'll hold off speculating until I hear how it's intended to work.)

          Show
          Wendy Smoak added a comment - I've just tried this out and have some questions... It seems that an agent can be added to more than one group. Is that intended? What does it mean for a build agent to be in no group(s) at all? I'm having trouble with the workflow involved in getting my first distributed build to happen. Now that we have these agent groups, it seems like you can't rely on the default build environment (none selected, just whatever Continuum is running as.) That means that logically, the default build definition, which does not have a build environment selected, will not actually do anything. (Except that it does... I see activity in agent-1's logs, but I'll hold off speculating until I hear how it's intended to work.)
          Hide
          Wendy Smoak added a comment -

          The agent grouping is not working on trunk (1.3.2-SNAPSHOT Build Number: 754707). Agents don't seem to stay in the group after the first page display.

          To reproduce:
          1. Add a build agent
          2. Add a build agent group
          2a. While adding the group, move the agent over to the 'used' column
          3. save. Note that the agent appears in the list for the group
          4. edit the group
          Expected: the agent appears in the list indicating that it's in the group
          Actual: the system displays "No build agents" (and does not show the multi-selection component)

          I'm also having trouble deleting the first group that I added, which was named "Default Group". I can delete other groups that I added later.

          I'm not seeing any log messages while adding/deleting agents and groups. Should there be?

          Show
          Wendy Smoak added a comment - The agent grouping is not working on trunk (1.3.2-SNAPSHOT Build Number: 754707). Agents don't seem to stay in the group after the first page display. To reproduce: 1. Add a build agent 2. Add a build agent group 2a. While adding the group, move the agent over to the 'used' column 3. save. Note that the agent appears in the list for the group 4. edit the group Expected: the agent appears in the list indicating that it's in the group Actual: the system displays "No build agents" (and does not show the multi-selection component) I'm also having trouble deleting the first group that I added, which was named "Default Group". I can delete other groups that I added later. I'm not seeing any log messages while adding/deleting agents and groups. Should there be?
          Hide
          Maria Catherine Tan added a comment -

          Fixed in revision 754790

          • show agents when editing agent group
          • fixed showing of action errors in jsp page

          I was able to delete the first group that I added. Maybe your agent group was attached to a build environment that's why you weren't able to delete it and the action errors is not showing in the jsp page?

          Show
          Maria Catherine Tan added a comment - Fixed in revision 754790 show agents when editing agent group fixed showing of action errors in jsp page I was able to delete the first group that I added. Maybe your agent group was attached to a build environment that's why you weren't able to delete it and the action errors is not showing in the jsp page?
          Hide
          jan ancajas added a comment - - edited

          It seems that an agent can be added to more than one group. Is that intended?

          Yes. this was intended to be flexible so that any agent can be assigned to any group.

          What does it mean for a build agent to be in no group(s) at all?

          As long as it is enabled, build agent with no group can be delegated a build task only from project whose associated "build-environment.build-agent-group" OR "build-environment" is empty.

          I'm having trouble with the workflow involved in getting my first distributed build to happen. Now that we have these agent groups, it seems like you can't rely on the default build environment (none selected, just whatever Continuum is running as.) That means that logically, the default build definition, which does not have a build environment selected, will not actually do anything.

          i'll check on the latest in the trunk, it ran as I've remembered with the default build definition and 1 enabled build agent (agent is not associated with an agent-group and build env.) setup.

          Update
          Tried the distributed build in r754817 with minimal configuration and it builds successfully from the build agent.
          for reference, here's what i did:
          1. added the continuum-buildagent.xml file in continuum-trunk/continuum-buildagent/continuum-buildagent-webapp/target/appserver-base/conf folder

          2. jetty:run in continuum-trunk/continuum-buildagent/continuum-buildagent-webapp folder

          3. jetty:run in continuum-trunk/continuum-webapp folder

          4. enabled distributed build checkbox from configuration page

          5. added http://localhost:9191/xmlrpc as a build agent.

          6. added a sample project and built it.

          Show
          jan ancajas added a comment - - edited It seems that an agent can be added to more than one group. Is that intended? Yes. this was intended to be flexible so that any agent can be assigned to any group. What does it mean for a build agent to be in no group(s) at all? As long as it is enabled, build agent with no group can be delegated a build task only from project whose associated "build-environment.build-agent-group" OR "build-environment" is empty. I'm having trouble with the workflow involved in getting my first distributed build to happen. Now that we have these agent groups, it seems like you can't rely on the default build environment (none selected, just whatever Continuum is running as.) That means that logically, the default build definition, which does not have a build environment selected, will not actually do anything. i'll check on the latest in the trunk, it ran as I've remembered with the default build definition and 1 enabled build agent (agent is not associated with an agent-group and build env.) setup. Update Tried the distributed build in r754817 with minimal configuration and it builds successfully from the build agent. for reference, here's what i did: 1. added the continuum-buildagent.xml file in continuum-trunk/continuum-buildagent/continuum-buildagent-webapp/target/appserver-base/conf folder 2. jetty:run in continuum-trunk/continuum-buildagent/continuum-buildagent-webapp folder 3. jetty:run in continuum-trunk/continuum-webapp folder 4. enabled distributed build checkbox from configuration page 5. added http://localhost:9191/xmlrpc as a build agent. 6. added a sample project and built it.
          Hide
          Wendy Smoak added a comment -

          > As long as it is enabled, build agent with no group can be delegated a build task only from project whose
          > associated "build-environment.build-agent-group" OR "build-environment" is empty.

          So that means groups are optional. You can add a bunch of agents, not put them in any group, and it should work out of the box with the default build definition [no build env, therefore no agent group]. This would explain why I was seeing activity on my build agent, which wasn't in a group.

          Thanks for the explanation!

          Show
          Wendy Smoak added a comment - > As long as it is enabled, build agent with no group can be delegated a build task only from project whose > associated "build-environment.build-agent-group" OR "build-environment" is empty. So that means groups are optional. You can add a bunch of agents, not put them in any group, and it should work out of the box with the default build definition [no build env, therefore no agent group] . This would explain why I was seeing activity on my build agent, which wasn't in a group. Thanks for the explanation!
          Hide
          Wendy Smoak added a comment -

          Re-closing now that adding/deleting groups and agents within groups is working on the web page.

          (Successfully tested distributed builds with mvn jetty:run in both webapps, however my original tests were with the Jetty bundle, so I'll keep investigating.)

          Show
          Wendy Smoak added a comment - Re-closing now that adding/deleting groups and agents within groups is working on the web page. (Successfully tested distributed builds with mvn jetty:run in both webapps, however my original tests were with the Jetty bundle, so I'll keep investigating.)
          Hide
          Wendy Smoak added a comment -

          Looking through the docs, it doesn't look like they've been updated for this new feature.

          Rather than re-opening this one again, (and possibly complicating the 1.3.2 release notes,) I opened CONTINUUM-2137 and linked it.

          It's not "done" until it's documented.

          Show
          Wendy Smoak added a comment - Looking through the docs, it doesn't look like they've been updated for this new feature. Rather than re-opening this one again, (and possibly complicating the 1.3.2 release notes,) I opened CONTINUUM-2137 and linked it. It's not "done" until it's documented.

            People

            • Assignee:
              Maria Catherine Tan
              Reporter:
              jan ancajas
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: