Skills

What You’ll Do

• Design, develop, and maintain scalable data pipelines and workflows to ingest, transform, and store large datasets.
• Collaborate with data scientists, analysts, and software engineers to understand data needs and deliver effective solutions.
• Optimize and enhance existing data processes for performance, scalability, and cost-efficiency.
• Implement data quality checks, validation, and monitoring to ensure data accuracy and reliability.
• Develop and manage data warehouses, databases, and other storage solutions.
• Ensure compliance with data governance and security policies.
• Stay up-to-date with emerging technologies and best practices in data engineering and apply them as appropriate.

An Ideal Candidate Should Have 

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• Proven experience as a Data Engineer or in a similar role and experience with ETL.
• Proficiency in programming languages such as Python and experience in SQL
• Big data tools: Data- and Delta-lakes
• Cloud: Bare-Metal, Hybrid infrastructure

Good to Have

• Experience working with media files (transformations)
• Torch dataset experience