Welcome to EMC Consulting Blogs Sign in | Join | Help

Crispin Parker's Blog

About Scrum for Team System and .Net development

Scrum for Team System v3 – Deep inside the Aggregation service

This post relates to Team Foundation Server and the Scrum for Team System process template.

Previously I have described the Event Service and have taken a closer look at the Transition Service. In this instalment I’ll describe the Aggregation service and it’s configuration.

In order to enhance the features of TFS 2010, we have added a web service that subscribes to the TFS work item changed notification. This gives us the ability to update related work items when certain triggering events are detected. The TFS change notifications are queued into a central service controller which in turn distributes them to all the registered service processors. The default SfTS v3 installation gives you two service processors; Transition and Aggregation.

The Transition processor handles the actions that relate to single work item instances, while the Aggregation service handles updates that involve collections of work items. The aggregation service also offers an alternative linking method, inferred linkage by work item path. More about this below.

The Aggregation Service

Unsurprisingly, this service offers the ability to execute aggregated updates to work items. These updates can be either “Sum”, “Min”, “Max” or “Average”.

An excellent example of this service in action is the “Product Backlog Item” (PBI) Work Remaining value. PBIs describe a required feature of the project output. To break the feature requirement down into task actions, you create linked work items called “Sprint Backlog Tasks” (SBT). Each SBT is given a “Work Remaining” value that indicates the duration of effort remaining in order to complete the specified task. The aggregation service sums up the total of the linked SBTs and updates the PBI work remaining value. Each time any of the SBTs remaining duration is changed, the service will update the associated PBI.

image

When the service updates a work item, it also adds a description of the action to the work item history. The above example states that the sum of associated tasks work remaining has been applied.

In this manner, the aggregation services can update work items to show values based on other work items. This is not a feature of standard TFS, but one I like to think adds a great deal of depth to work item linkage.

Inferred Linking.

Because the aggregation of a work item value can potentially involve a large number of other work items, we have added an alternative way to specify linkage. Imagine that you want to roll up the total number of story points in a release. If you could only roll up linked work item values, you would have to directly link every PBI to the Release work item. A single release might contain hundreds of PBIs and maintaining all those links isn’t going to be fun task, even for a temp. To combat this, you can create aggregation rules that work by Inferred linkage.

The concept of inferred linkage is that you consider work item paths as an indication of a relationship. Looking at our “completed story points in this release” rule, we can say: “Rollup all PBI story Point values where the iteration path under this release”. No direct linkage is required to do this, so you can take an extra hour for lunch with the time you saved not adding hundreds of work item links (or get rid of the temp).

This is why it is very important to maintain the correct planning path structure in your SfTS v3 work items. If you alter the default path hierarchy (Project / Release / Work Stream / Sprint / Team Sprint), the inferred aggregation rules will not be able correctly ascertain the relationships. Of course you could update all the rules to work with your new path structure and you might want to bring back the temp to help out with the additional work load.

You can use both Iteration (planning) and Area (Feature) paths to indicate an inferred link.

What Changes Trigger an Update?

It’s not only the field values changing that can trigger the update service. The aggregated value can also be affected by changing work item linkages, paths and states. The service rule definitions also allow you to specify a list of excluded states. Moving a work item in or out of an excluded state will also trigger an update.

Triggers:

  • The aggregated field value is changed.
  • The work item state is change from or to an excluded state.
  • The work item linkage is changed (direct links).

    or
  • The work item path value is changed (inferred links).

Where can I find the default Rule definitions?

The default installation location of the Aggregation rules file is located on the TFS AT at:

  • C:\Program Files\Scrum for Team System\Services\3.0\Rules\AggregationRules.xml

What does A Rule look like?

Below are two examples of aggregation rules. The first rule describes the PBI work remaining calculation and the second shows the PBI story point roll up.

PBI work remaining (using direct links)

<AggregateRule type="Sum">
  <WorkItemType source="Sprint Backlog Task" target="Product Backlog Item" />
  <Link linktypename="Scrum.ImplementedBy" isforward="true" />
  <Field source="Scrum.WorkRemaining" target="Scrum.WorkRemaining" />
  <ChangeNote>Sum of associated task(s) work remaining applied.</ChangeNote>
  <ExcludedSourceStates>
    <State>Deleted</State>
  </ExcludedSourceStates>
</AggregateRule>

Release Story Points (using inferred linkage)

<AggregateRule type="Sum">
  <WorkItemType source="Product Backlog Item" target="Release" />
  <InferredLink path="IterationPath" type="Partial" depth="2" />
  <Field
      source="Microsoft.VSTS.Scheduling.StoryPoints"
      target="Scrum.PlannedVelocity" 
 
/>
  <ChangeNote>Sum of planned backlog item(s) points applied.</ChangeNote>
  <ExcludedSourceStates>
    <State>Deleted</State>
  </ExcludedSourceStates>
</AggregateRule>

 

 

Rule Definition Breakdown

Let’s break the rule definition down into it’s component parts.

Aggregate Rule

The rule container element.

<AggregateRule type="Sum">
  ...

</AggregateRule>

Attributes:

  • type – Describes the aggregation action. Possible values are “Sum”, “Min”, “Max” and “Average”.

Work Item Type

Defines the work item types involved in this rule.

<WorkItemType source="Product Backlog Item" target="Release" />

Attributes:

  • source – The source work item type to be aggregated.
  • target – The target work item type to be updated.

Link 

Details the work item link type to be used in the aggregation.

Note: Aggregation rules must specify either a “Link” or an “Inferred Link” but not both.

<Link linktypename="Scrum.ImplementedBy" isforward="true" />

Attributes:

  • linktypename – The linkage type name.
  • isforward – Boolean value to indicate if the target work item is on the forward side of the link.

Inferred Link 

This element describes the inferred link for this rule.

Note: Aggregation rules must specify either a “Link” or an “Inferred Link” but not both.

<InferredLink path="IterationPath" type="Partial" depth="2" />

Attributes:

  • path – Details which work item path is going to be used to infer the relationship. This value can be either: “IterationPath” or “AreaPath”.
  • type – Describes the type of path matching to use. This can be either “Partial” or “Complete”. Partial paths match to a specified depth and complete matches the entire path.
  • depth – Used in conjunction with the “Partial” type to indicate the depth of path to match.

The example shows a depth of “2”. This means that only the “Project / Release” part of the iteration path will be matched between the target and source work items.

Field 

This details which the the work item fields to use for this rule.

<Field source="Scrum.WorkRemaining" target="Scrum.WorkRemaining" />

Attributes:

  • source - The field on the source work item to aggregate.
  • target – The field on the target work item to update.

Change Note

The text to enter into the target work items history when a change is made.

<ChangeNote>Sum of associated task(s) work remaining applied.</ChangeNote>

Excluded Source States

A list of states that indicate a source work is ignored from aggregation.

<ExcludedSourceStates>
  <State>Deleted</State>
</ExcludedSourceStates>

A Word of Warning!

When customising rules in either the Transition or Aggregation service, make sure you take extra care not to cause any circular references. If you set one rule to change a field value that in turn triggers another rule, that in turn triggers the original rule, that in turn triggers the other rule... Well I expect you get the picture. Take time to plan the relational updates between your work items and keep an eye out for circular rule references. Remember that an infinite loop might involve more than just two rules.

Handy Links

Regards,

Crispin Parker,
Senior Technical Consultant,
EMC Consulting.

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

 

russweb said:

I tried applying your framework to another template (just for aggregation purposes) and got an error.

________________________________

Here is the text from AggregationRules.xml

<?xml version="1.0" encoding="utf-16"?>

<ArrayOfAggregateRule xmlns="http://schemas.ScrumforTeamSystem.com/AggregationService/RulesCollection">

 <AggregateRule type="Sum">

   <WorkItemType source="Task" target="User Story" />

   <Link linktypename="System.LinkTypes.Hierarchy" isforward="true" />

   <Field source="Microsoft.VSTS.Scheduling.RemainingWork" target="NEW.UserStory.WorkRemaining" />

   <ChangeNote>Sum of associated task(s) work remaining applied.</ChangeNote>

   <ExcludedSourceStates>

     <State>Deleted</State>

   </ExcludedSourceStates>

 </AggregateRule>  

</ArrayOfAggregateRule>

________________________________

Error: 'ServiceException'

An error occured while processing Work Item: '206' and rule: Aggregate Rule: (WorkItemTypes - 'Task' => 'User Story'); Link: 'System.LinkTypes.Hierarchy' (Target on forward end: True); Field - 'Microsoft.VSTS.Scheduling.RemainingWork' => 'NEW.UserStory.WorkRemaining'; Excluded Source States: ('Deleted')

The work workItem is not valid.

*** WORK ITEM ERROR *** Validation errors found in work workItem 206.

Field: 'NEW.UserStory.WorkRemaining' status: 'InvalidNotOldValue' value: '0'

  at Conchango.TeamSystem.SubscribedEventHandler.AggregationService.UpdateWorkItem.Execute(WorkItem targetWorkItem, IEnumerable`1 rules)

  at Conchango.TeamSystem.SubscribedEventHandler.AggregationService.AggregationProcessor.ProcessMatchedRules(WorkItemChangedEvent workItemChangedEvent, WorkItemStore workItemStore, IEnumerable`1 matchedRules)

the message resource is present but the message is not found in the string/message table

_____________________________

Any pointers? It seems that the code in you service is looking for a "0" Value or something? Shouldn't a brand new one end up with a "0" value AFTER your rules are run if there are no child Tasks?

February 19, 2010 16:39
 

russweb said:

Additionally, adding a Task with 20 hours under the new item (using the rule above).

Results in the error below.

__________________________________________

The description for Event ID 0 from source Scrum for Team System cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event:

Error: 'ServiceException'

An error occured while processing Work Item: '288' and rule: Aggregate Rule: (WorkItemTypes - 'Task' => 'User Story'); Link: 'System.LinkTypes.Hierarchy' (Target on forward end: True); Field - 'Microsoft.VSTS.Scheduling.RemainingWork' => 'NEW.UserStory.WorkRemaining'; Excluded Source States: ('Deleted')

The work workItem is not valid.

*** WORK ITEM ERROR *** Validation errors found in work workItem 288.

Field: 'ECIS.UserStory.WorkRemaining' status: 'InvalidNotOldValue' value: '20'

  at Conchango.TeamSystem.SubscribedEventHandler.AggregationService.UpdateWorkItem.Execute(WorkItem targetWorkItem, IEnumerable`1 rules)

  at Conchango.TeamSystem.SubscribedEventHandler.AggregationService.AggregationProcessor.ProcessLinkChanges(WorkItemChangedEvent workItemChangedEvent, WorkItemStore workItemStore, IEnumerable`1 allRules)

the message resource is present but the message is not found in the string/message table

February 19, 2010 16:56
 

crispin.parker said:

Hi Russ,

The error message indicates that the target field will not accept the new value. Unfortunately there is little documentation on the meaning of the TFS work item validation status codes. So I can only guess that "InvalidNotOldValue" means that the work item field is expecting it's original value upon saving. Check the WITD and see if the target field has value restrictions.

Note: Aggregation target fields must be of the double value type.

Crispin.

I knew I wrote all that verbose error mesages for something...

February 26, 2010 09:42
 

Kris Venk said:

Walking over the two blogs on Transtion Service & Aggregation Service, I am in a dileama to decide upon which would be a better rule service (Transtion Service or Aggregation Service) to make to use of for accomplishing below customizations:

I am looking to have a custom REQUIRED Work Item field called "Actual Hours" in my SBT's & PBI's.  

The custom 'Actual Hours' field value in the PBI would be the roll up sum of all SBT's with statuses set to 'done' that get implemented for the PBI .

i.e PBI Actual Hours = Sum of  all Implemented SBT Actual Hour's with Status = 'done'.

In case of the above customization, would it be good idea to use a transition rule wherein the PBI Actual hours field value is updated by the below expression whenever the state of SBT changes from any state to 'done' state:

PBI Actual Hours = PBI Actual Hours + SBT Actual Hours.

In the reverse situation, when the SBT state changes from 'done' to any state:

PBI Actual Hours = PBI Actual Hours - SBT Actual Hours.

Even though PBI Actual Hours is an aggregation, I am not yet seeing how aggregation service rule would be more apt?

September 21, 2010 02:50
 

crispin.parker said:

@Kris - The rule you are trying to implement is an aggregation of linked child work items, so the aggregation service is the place to create the rule.

In order to get the sum of all linked work items in a state of "Done" you have to exclude all child item in states other than "Done".

See the "Sum of completed backlog item(s) points applied." aggregation rule for an example.

Crispin.

September 21, 2010 09:33
 

Kris Venk said:

@ Crispin

Thank you for the feedback.  The example proved to be very helpful.

On a different note, I was thinking about the different lnkagetypenames.

Scrum.Implementedby, Scrum.Impededby,Scrum.TestedBy Scrum.Failed are some of the linkagetypenames that are very apparent.

I was wondering how would we indicate lnkagetypenames for flat & generic relationships like for a 'Child' or a 'related' linkage?

September 22, 2010 00:39
 

Kris Venk said:

@ Crispin:

I see a very strange thing happening with my aggregation rules. I am not able to make any sense over this occurence at all.

I have two custom fields  'X' , 'Y' in my SBT and I intend to roll up sum of X's from SBT's into PBI's custom field 'A' & sum of 'Y's from SBT's into PBI's custom field 'B'.

1) If I define my aggregation sum rule for only X in SBT to add up into A in PBI it works.

2) If I define my aggregation sum rule for only Y in SBT to add up into B in PBI it works.

3) If I define my aggregation sum rule for both X & Y in SBT to add up into A & B in PBI it  doesnt add up for both.

Source:

<AggregateRule type="Sum">

 <WorkItemType source="Sprint Backlog Task" target="Product Backlog Item" />

 <Link linktypename="Scrum.ImplementedBy" isforward="true" />

<Field source="Scrum.v3.1.3.SBTActualHours" target="Scrum.v3.1.ActualTime"/>

<ChangeNote>Sum of actual work hours applied.</ChangeNote>

 <ExcludedSourceStates>

   <State>Not Started</State>

 </ExcludedSourceStates>

</AggregateRule>

<AggregateRule type="Sum">

 <WorkItemType source="Sprint Backlog Task" target="Product Backlog Item" />

 <Link linktypename="Scrum.ImplementedBy" isforward="true" />

<Field source="Scrum.v3.1.3.SBTEstimatedHours" target="Scrum.v3.1.EstimatedTime"/>

<ChangeNote>Sum of Estimated work hours updated.</ChangeNote>

 <ExcludedSourceStates>

   <State>Descoped</State>

 </ExcludedSourceStates>

</AggregateRule>

The two aggregrate rules would work individually....but when included within the rules together they both would work.

I cant see why?

September 23, 2010 02:10
 

crispin.parker said:

@Kris - I've recreated your work item changes and rules in my test system and it all works as expected. Both rules are executed regardless of there inclusion together.

Are you sure that all the rule conditions are being met by your testing? For example; the actual hours rule excludes SBTs that are in the default starting state of "Not  Started". This means that changing the actual hours of a not started SBT will have no affect on the linked PBI. Only when a SBT is moved into a state other than "Not Started" will the actual hours rule be matched.

If you would like me to send you the working work item definitions and rules file, please email: ScrumforTeamSystem(AT)emc.com

Crispin.

September 23, 2010 12:37
 

Kris Venk said:

@ Crispin

Yes, I did consider that my ruleset wont update for the 'Not Started' state in my testing.

I would be glad to see the working work item definitions & rules file. I have sent an email to the above address.

Thank you again for your help.

September 23, 2010 14:38
 

Kris Venk said:

@ Crispin

Would your aggregation rule example above work if a custom field on the target work item is to be subtracted from the target field value on every update?

If we consider a custom field 'A' (say double type) with reference name Custom.A on the target work item, is it right define the target action as :

<Field source="Scrum.WorkRemaining" target="Scrum.WorkRemaining- Custom.A" />

Inclusion of the above Version in your example:

AggregateRule type="Sum">

 <WorkItemType source="Sprint Backlog Task" target="Product Backlog Item" />

 <Link linktypename="Scrum.ImplementedBy" isforward="true" />

 <Field source="Scrum.WorkRemaining" target="Scrum.WorkRemaining- Custom.A" />

 <ChangeNote>Sum of associated task(s) calculated work remaining applied.</ChangeNote>

 <ExcludedSourceStates>

   <State>Deleted</State>

 </ExcludedSourceStates>

</AggregateRule>

October 2, 2010 04:43
 

crispin.parker said:

@Kris - No, that is not a valid rule. The field element details the name of the target field and the name of the source field only. You cannot perform calculations within the specification of a field name. This is explained in my above blog post.

Crispin.

October 6, 2010 09:09
 

Adam said:

We implemented a new aggregateRule for EstimatedEffort that is a sum of the task estimatedeffort but this does not seem to be picked up on existing Work Items.  Do we need to restart the server to have changes take effect, or is there something else we are missing?

October 7, 2010 19:27
 

crispin.parker said:

@Adam - After altering the rules configuration files, you should restart the IIS process. Then the rules changes will be detected. This also happens automatically when the app pool is recycled.

It is also worth noting the rules are only applied when a triggering action is detected. This means that the estimated effort won't be applied until a change is made to the source work items state, link specification or target field value. However, you can use the SyncAll web service to force the updates to be applied.

You can find the sync all service at: http://[YourTFSHost]:8080/tfs/ScrumforTeamSystem/3.0/SyncAll.asmx

Crispin.

October 8, 2010 09:26
 

Yezdaan said:

I have a weird problem where my "team sprint"s planned work = 572.5 but when I add up all the sprint backlog tasks under that iteration path, I get 370.  I have checked both the windows event log and the log file for errors and see none.  I also tried running a sync all, but that does not seem to re-calculate the value.  Is there anything else I can do to debug this?

October 14, 2010 16:59
 

Yezdaan said:

I figured it out.  I thought it was adding up work remaining not planned work.  

October 14, 2010 18:01
 

James said:

I've implemented additional aggregation rules for new Work Item Types, and they work fine when the new Work Item Type is added, however when the target field or status is changed the fields do not get aggregated?  Any ideas?

November 17, 2010 16:43
 

crispin.parker said:

@James: Not sure on what the issue might be. Please send a copy of your amended rules file(s) and (if the work items have been modified) a copy of the Work Item Type definitions to ScrumforTeamSystem{at}emc.com. I can then check that everything is configured correctly.

Crispin.

December 7, 2010 16:10
 

Stephen said:

I was getting the "workitem is not ready to save" - InvalidNotOldValue error.  I googled it and this page was fairly high.

I have since figured it out, so I thought I would leave what it means to those who follow:

InvalidNotOldValue (for me) meant that I have a field that is marked <READONLY> and I tried to change it.  Apparently <READONLY> on the field really really means read only.  The fix is simply to take <READONLY> off and make the controls that surface the field to be read only.

January 25, 2011 22:50
 

Raj said:

@Stephen  good one bro' u saved my day...

July 20, 2011 23:49
 

Peter K. said:

Thanks heaps for this and other posts on your blog! They are very infomarmative.

One question:

If the source is "SBT" and the target is "PBI", why is the linktypename set to "Scrum.ImplementedBy" ??

Isn't the relationship SBT Implements PBI not SBT ImplementedBy PBI?

December 5, 2011 20:45
 

Brixel said:

Is there anything you need to do outside of altering the XML to get this to work? I've added my rules but my other fields are not summing up.

February 27, 2012 18:53

Leave a Comment

(required) 
(optional)
(required) 
Submit

This Blog

Syndication

News

Powered by Community Server (Personal Edition), by Telligent Systems