Data Engineer - Infiligence Inc

Key Responsibilities

• Design, develop, and maintain scalable data pipelines for batch and real-time data processing using Azure (preferred) or AWS cloud platforms.
• Build and optimize ETL processes to ensure high-quality, secure, and efficient data flow across systems.
• Collaborate with cross-functional teams to translate business requirements into robust data models and solutions.
• Implement data quality, data governance, and data security standards throughout the data lifecycle.
• Develop and maintain documentation, including design and mapping documents.
• Conduct code reviews, unit testing, and peer reviews to ensure code quality and compliance.

Requirements

• 5–7 years of experience in data engineering, with a strong background in building and managing large-scale data pipelines.
• Hands-on experience with Azure Data Services (Data Factory, Data Lake, Synapse, Databricks, etc.) is preferred.
• Proficiency in SQL, Python, or Scala for data processing and transformation.
• Experience with data warehousing (e.g., Snowflake, SQL Server, MongoDB) and real-time databases.
• Strong understanding of data architecture, data ingestion, curation, and consumption patterns.