Senior Data Engineer

We are seeking a highly talented Senior Data Engineer to design, build and maintain the high-volume data architecture behind critical retail pricing strategy and physical store space planning. Working within our modern cloud ecosystem (Databricks on AWS), you will automate the data pipelines that connect internal store management software with large external CPG vendor feeds (e.g., major beverage, packaged goods and grocery suppliers) and market-matching datasets.

Key responsibilities

• Pipeline automation and data ingestion: Design, build and optimize automated data feeds for external CPG vendor cost updates, competitor market pricing and store spatial layout data.
• Modern data stack governance: Manage and structure modular pipelines using dbt, Databricks Unity Catalog and Python, ensuring high performance, proper data governance and schema control.
• Complex data transformations: Write advanced, production-grade SQL and Python scripts to clean, structure and stitch disparate vendor datasets into scalable data models.
• CI/CD and code quality: Enforce continuous integration, unit testing, version control and automated deployment practices using GitHub.
• Cloud architecture optimization: Monitor and tune compute performance, data storage and pipeline execution within AWS and Databricks.

Requirements and qualifications

• Experience: 5+ years of hands-on data engineering experience, ideally in retail, CPG, e-commerce or supply chain environments.
• Core tech stack: Expert proficiency in SQL, Python, dbt, Databricks Unity Catalog and AWS data infrastructure.
• Automation and version control: Proven success building automated ingestion frameworks and using GitHub for repository management and CI/CD pipelines.
• Data modeling: Solid understanding of dimensional modeling, star schemas and building unified enterprise catalog spaces.
• Soft skills: Excellent remote communication skills, a proactive problem-solving mindset and the ability to work independently in a nearshore setup.