Skip to main content

I am investigating using Fivetran to export Pigment data into Snowflake and I didn’t see anything in the Pigment or Fivetran documentation about access to snapshots. Is it possible to export Pigment data from a snapshot into Snowflake?

Hi ​@kevin_huang hope all is well.

Yes, it is possible to export Pigment data from a snapshot into Snowflake, but with some limitations:

Export from snapshots is supported

-You can export data from blocks within snapshots using Pigment's export API.⁠⁠ However, you cannot export entire snapshots at once.

-The export process requires you to generate an export payload that specifically selects snapshot data.⁠⁠​

-Check this article below on how to generate a export payload selecting snapshot data.

Methods to get data into Snowflake

-Export API approach: Use Pigment's export API to extract snapshot data and then load it into Snowflake.⁠⁠​

-Export documentation: There's specific guidance on exporting Pigment data to Snowflake available in the community documentation.⁠⁠​ 

Important considerations

-Pigment Views can be exported via API, which can achieve near real-time or frequent data integration to data warehouses like Snowflake.⁠⁠​

-The export process involves using the JSON payload builder in Pigment to capture the necessary field information for filtering and exporting from specific snapshots.⁠⁠​

-Each export needs to be configured for the specific blocks and snapshots you want to extract data from.

The recommended approach would be to use Pigment's export API with snapshot-specific payloads to extract the data you need, then load it into Snowflake using your preferred data pipeline tool.

 


Reply