This ETL process extracts data from the Zoho CRM api and sends it to the warehouse (redshift).
Local execution and sending of jobs will require to
setup your terminal
and have access to the
prod_observes role and the
CACHIX_AUTH_TOKEN environment variable.
The ETL request export data on batch jobs through the zoho API. It saves the job id on redshift for future retrieval. Job is triggered by an schedule; local execution can be done with:
m . /observes/etl/zoho-crm/fluid
This is the core ETL process which downloads the results of the previously requested data.
m . /observes/etl/zoho-crm/fluid/prepare
Sometimes the ETL cannot continue normally
expired bulk data
Resolution: delete bulk_jobs state and retry prepare and core procedures
delete from zoho_crm.bulk_jobs