About the Role
Join us as a Lead Data Engineer. At Barclays, we don’t just adapt to the future, we create it. As a Lead Data Engineer you will design, develop and improve software, utilising various engineering methodologies, that provides business, platform, and technology capabilities for our customers and colleagues.
This role is based in Pune.
Purpose of the role
To build and maintain the systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure.
Accountabilities
- Build and maintenance of data architectures pipelines that enable the transfer and processing of durable, complete and consistent data.
- Design and implementation of data warehoused and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures.
- Development of processing and analysis algorithms fit for the intended data complexity and volumes.
- Collaboration with data scientist to build and deploy machine learning models.
Requirements
To be a successful Lead Data Engineer, you should have experience with:
- Data Engineering & ETL: Strong SQL skills for complex queries, optimization, and performance tuning. Knowledge of batch and real-time data processing patterns including understanding with streaming technologies (like Kafka). Experience with Spark for distributed data processing. Experience building and maintaining robust ETL/ELT pipelines. Data modeling expertise (dimensional modeling, star/snowflake schemas). Hands-on experience with data quality frameworks and validation.
- Public Cloud Platform Expertise (AWS or similar): Proven Experience in designing and implementing scalable data solutions on public cloud. Deep knowledge of data services like: AWS S3, Redshift, Athena. Understanding of security, IAM, VPC, and cost optimization.
- Programming & Development: Proficient in Python or Java for data processing. Version control and CI/CD pipelines. Understanding of software engineering best practices. (testing, code reviews, documentation)
- Leadership & Architecture: Proven track record leading data engineering teams. Experience designing end-to-end data architectures. Ability to translate business requirements into technical solutions. Strong stakeholder management and communication skills.
Additional Relevant Skills Given Below Are Highly Valued:
- Advanced Analytics & Visualization: Power BI development experience & creating interactive dashboards and reports.
- Data Governance & Quality: Implementing data governance frameworks & understanding of data lineage and metadata management tools.
- Modern Data Stack: Understanding of data Lakehouse architectures (Delta Lake, Iceberg) & experience with containerization using Docker.
- DevOps & Infrastructure as Code: CloudFormation for infrastructure automation, Monitoring and observability tools (CloudWatch, Datadog), Performance optimization and troubleshooting & Disaster recovery and backup strategies.
- Business Acumen: Financial services or reconciliation domain knowledge (given your SQL context), Agile/Scrum methodology experience & Cross-functional collaboration with analytics and business teams.
Vice President Expectations
- To contribute or set strategy, drive requirements and make recommendations for change. Plan resources, budgets, and policies; manage and maintain policies/ processes; deliver continuous improvements and escalate breaches of policies/procedures.
- If managing a team, they define jobs and responsibilities, planning for the department’s future needs and operations, counselling employees on performance and contributing to employee pay decisions/changes. They may also lead a number of specialists to influence the operations of a department, in alignment with strategic as well as tactical priorities, while balancing short and long term goals and ensuring that budgets and schedules meet corporate requirements.
- If the position has leadership responsibilities, People Leaders are expected to demonstrate a clear set of leadership behaviours to create an environment for colleagues to thrive and deliver to a consistently excellent standard. The four LEAD behaviours are: L – Listen and be authentic, E – Energise and inspire, A – Align across the enterprise, D – Develop others.
- OR for an individual contributor, they will be a subject matter expert within own discipline and will guide technical direction. They will lead collaborative, multi-year assignments and guide team members through structured assignments, identify the need for the inclusion of other areas of specialisation to complete assignments. They will train, guide and coach less experienced specialists and provide information affecting long term profits, organisational risks and strategic decisions.
- Advise key stakeholders, including functional leadership teams and senior management on functional and cross functional areas of impact and alignment.
- Manage and mitigate risks through assessment, in support of the control and governance agenda.
- Demonstrate leadership and accountability for managing risk and strengthening controls in relation to the work your team does.
- Demonstrate comprehensive understanding of the organisation functions to contribute to achieving the goals of the business.
- Collaborate with other areas of work, for business aligned support areas to keep up to speed with business activity and the business strategies.
- Create solutions based on sophisticated analytical thought comparing and selecting complex alternatives. In-depth analysis with interpretative thinking will be required to define problems and develop innovative solutions.
- Adopt and include the outcomes of extensive research in problem solving processes.
- Seek out, build and maintain trusting relationships and partnerships with internal and external stakeholders in order to accomplish key business objectives, using influencing and negotiating skills to achieve outcomes.