Profiler in Practice: Scoping

This article will help you to use the profiler in practice: what can you do, what does it tell you and why.

Once you’re diving deep into Pigment’s computation engine the profiler can become your best friend. It allows you to track the full cycle of metric computations from your input all the way to the final metric in your application. 

In general, the first thing you’d probably use it for is to track computation time per metric and then try optimizing the ones where you think it needs some formula optimization to increase the computation time. This article will not focus on that, you can use the article linked above for the explanation of those features. 

In this article we’ll focus on the Scope column in the profiler:

What is Scope (or Scoping)?

Scope in the context of Pigment computations means the part of the metric that is being computed because of a change in the metric itself or the metric that it is referencing in the formula. 

Imagine a metric with 2 dimensions: Employee & Month. 

If my input just affects one employee in one month, Pigment should only calculate that intersection, and leave the rest of the metric data untouched. This will result in improved computation times. All the metrics that reference this initial metric should also have a scoped computation, as long as you’re not doing any data transformation.

What can the profiler tell me about the scope?

The profiler will categorize the scope of every metric in the computation chain into 3 categories:

• Full: this means the scope is equal to the scope the initial input was done on. So if my input only touched 1 employee and 1 month, only that intersection will be recomputed.

• Partial: this means the scope is still kept, but there has been a change in the dimensionality of the structure of the metric. For example, if we add the Version dimension (through an ADD modifier) into the metric, we now not only need to calculate the single employee & month but also all versions.

• None: certain modeler actions can completely remove the scope of the metric. The most common one will be the REMOVE modifier. Imagine we want to calculate the share of each employee/month compared to the total. We then need to take the original values and divide that by the total of all employees & months. Because in this case we’re completely re-doing the calculation for every cell we then lose the scope. So Pigment needs to recompute every cell in the metric to display the correct result.

I understand the theory, how do I see this in the profiler?

The example above is completely traceable in the profiler. You’ll see the categories that are mentioned above (Full,Partial,None) in the profiler in the scope column.

Is it always bad to lose the scope?

No, sometimes there is no other way. In the example above we need to recompute every cell to get to the correct result. Even if just our input value changes, the share of every other employee & month can change, so there is no way we could keep the scope in a formula (note you could do the same type of calculation with “Show values as” and then you keep the scope but it will be visual only). 

Why is it better to keep the scope if I can?

Every metric that references a metric that has lost its scope will also not be scoped by default (some exceptions occur because Pigment is smart enough to try and get back the scope but that’s for another article). So if you lose the scope (without a clear use) early in the computation cycle it can slow down your other metrics.

How can I keep the scope?

You can’t always prevent it (as in the example above) but there are some things you can watch out for. I’ve listed them below:

• If you use CUMULATE (or a similar function like PREVIOUS or YEARTODATE) you’ll lose scope on the cumulating dimension (because Pigment needs to compute A to compute B). The same rule applies for making a shift on a dimension, e.g. doing a [SELECT: Month -1].
• If you use REMOVE on places where you don’t have to, there's a chance you will lose the scope, as it can trigger a lot more cells to be recomputed. Keeping a dimension in there could get you a better result in the end in some cases. 
• If there’s no way around losing the scope, try to push that metric to the end of the chain if you can. This will ensure the metrics before it to still keep the scoped computation.