Summary
The AWS Data Engineer will be responsible for designing, implementing, and maintaining data solutions on Amazon Web Services (AWS) to support the organization's data and analytics needs. This role will focus on building scalable, secure, and efficient data pipelines, integrating data across systems, and leveraging AWS services to optimize data storage, processing, and retrieval. The AWS Data Engineer will collaborate with cross-functional teams to deliver robust and reliable data infrastructure
Essential Duties and Responsibilities:
- Serve as a shared resource across product teams to provide expertise and scalable solutions for data integration, modeling, and analytics across all products.
- Design and implement scalable data pipelines using AWS services such as AWS Glue, AWS Data Pipeline, and Amazon Kinesis.
- Create and maintain all Infrastructure-as-Code (IaC) for primary product teams using tools like AWS CloudFormation or CDK.
- Develop and maintain data storage solutions using Amazon S3, Redshift, RDS, DynamoDB, or other AWS-managed databases.
- Design and develop code pipelines for our primary product application to:
- Build application components.
- Run automated tests created by the QA team.
- Execute static code analysis.
- Allow or prevent commits or PR merges based on predefined quality thresholds.
- Deploy and manage AWS infrastructure resources as needed to support organizational and product-specific goals.
- Implement monitoring, logging, and centralization of application instrumentation for real-time insights and troubleshooting.
- Optimize ETL workflows to efficiently manage data across diverse sources and destinations.
- Set up and manage AWS IAM policies, roles, and security-related configurations to ensure secure access and data protection.
- Stay updated on AWS innovations and recommend tools or best practices to enhance the organization’s data ecosystem.
- Create comprehensive documentation and provide training for AWS-based data solutions, ensuring knowledge transfer and ease of use.
- Ensure all data processes and systems adhere to security best practices, aligning with OWASP Top 10, CWE Top 25 guidelines, and CIS AWS Foundations Benchmark.
Further Expectations of the Role:
- Disciplined in approach to work product completion and timelines.
- Must have analytical thought processes to make independent decisions.
- Ability and willingness to work extended hours on a sporadic basis as required.
- Ability and willingness to work on multiple assignments/projects in a fast-paced environment and meet tight deadlines.
- Ability and willingness to work independently.
- Ability to effectively formulate and communicate problems and ideas.
- Demonstrate and provide outstanding customer and employee relation