I am investigating using Fivetran to export Pigment data into Snowflake and I didn’t see anything in the Pigment or Fivetran documentation about access to snapshots. Is it possible to export Pigment data from a snapshot into Snowflake?
Hi
Yes, it is possible to export Pigment data from a snapshot into Snowflake, but with some limitations:
Export from snapshots is supported
-You can export data from blocks within snapshots using Pigment's export API. However, you cannot export entire snapshots at once.
-The export process requires you to generate an export payload that specifically selects snapshot data.
-Check this article below on how to generate a export payload selecting snapshot data.
Methods to get data into Snowflake
-Export API approach: Use Pigment's export API to extract snapshot data and then load it into Snowflake.
-Export documentation: There's specific guidance on exporting Pigment data to Snowflake available in the community documentation.
Important considerations
-Pigment Views can be exported via API, which can achieve near real-time or frequent data integration to data warehouses like Snowflake.
-The export process involves using the JSON payload builder in Pigment to capture the necessary field information for filtering and exporting from specific snapshots.
-Each export needs to be configured for the specific blocks and snapshots you want to extract data from.
The recommended approach would be to use Pigment's export API with snapshot-specific payloads to extract the data you need, then load it into Snowflake using your preferred data pipeline tool.
Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.