Job Title: Lead Data Engineer w/d GCP (Big Query) Exp
Location: Remote
Duration: FTE role
Mandatory Skills:
• Python, PL/SQL, DBT, Airflow, Big Query
Technical Skills:
• Python: Programming, ETL scripting, business logic implementation, and experience with data manipulation libraries (pandas, NumPy).
• PL/SQL: Data transformation, view/table creation, and persistence layer interactions.
• Data Transformation Tools: DBT (Data Build Tool) for creating models and documentation.
• Orchestration and Data Processing Tools
• Orchestration: Airflow, Dataflow, and Cloud Composer.
• Data Ingestion: Cloud Functions, Cloud Run, SFTP mechanisms, and Falcon (for database connections).
Cloud Platform:
• Google Cloud Platform (GCP) and Big Query experience.
• Experience with cloud-native data processing.
Additional Desired Skills:
• Understanding of householding and data segmentation logic.
• Ability to work with streaming and batch data.
• Experience with data integration from multiple sources (e.g., Reltio, external databases).
• Familiarity with incremental data loading and partitioning strategies.