Skip to main content
Solved

Sparse Engine concept

  • March 6, 2024
  • 5 replies
  • 362 views

Thu Mai
Master Trendsetter
Forum|alt.badge.img+10
  • Master Trendsetter
  • 51 replies

Hi Pigmenteers!

While I’m taking Pigment trainings and explore my personal application, I figure out that there is no information about the size of my application or total number of cells, etc.. and the reason behind is that Pigment is a sparse engine.

Therefore, can you help explain to me more about the concept of sparse engine - what is a sparse engine, how does it actually work and what is the benefit compared to non-sparse engine?

Best answer by Igor P

Hello @Thu Mai !

Long story short, a sparse engine does not take into account the blank cells, therefore it can perform better and faster compared to other calculation engines (where blanks are filled - densified - with 0 or falses).

Please take look in this article for more details.

Let me know if you have any questions!

Thanks!

Igor

5 replies

Igor P
Employee
Forum|alt.badge.img+3
  • Employee
  • 8 replies
  • Answer
  • March 6, 2024

Hello @Thu Mai !

Long story short, a sparse engine does not take into account the blank cells, therefore it can perform better and faster compared to other calculation engines (where blanks are filled - densified - with 0 or falses).

Please take look in this article for more details.

Let me know if you have any questions!

Thanks!

Igor


francois
Employee
Forum|alt.badge.img+13
  • Employee
  • 171 replies
  • March 6, 2024

Hi!
While a sparse engine is indeed optimised when you do not need all intersections of your dimensions, cells do still count, and if you really want to optimise your model you should try to compute as little as possible.

You’ll find a guide below on how to add the number of cells in each metric in the block explorer.

If you feel a particular block is especially slow, it might be useful to try to analyse whether you absolutely need all cells. 

 

Hope that helps too!


Thu Mai
Master Trendsetter
Forum|alt.badge.img+10
  • Author
  • Master Trendsetter
  • 51 replies
  • March 7, 2024

oh got it! thanks @Igor P and @francois a lot!! 😊


Sharon Conroy
Wallflower
Forum|alt.badge.img+8
  • Wallflower
  • 19 replies
  • February 12, 2025

This article has since been updated - here is the new link :) 

 

 


Thu Mai
Master Trendsetter
Forum|alt.badge.img+10
  • Author
  • Master Trendsetter
  • 51 replies
  • February 12, 2025

@Sharon Conroy woohoo this is nicee! thanks for sharing!