remote
Senior Engineer (Automation and Web Crawling) - Triadique Technologies Private Limited
Automation Engineer
Senior Automation Engineer to build and optimize web crawling systems for large-scale data extraction.
About the role
Key Responsibilities
- Develop and maintain web crawling and data extraction systems
- Optimize automation pipelines for performance and reliability
- Design scalable solutions for large-scale data processing
- Collaborate with data teams to ensure data quality and integrity
- Implement best practices in error handling and monitoring
Requirements
- 5+ years of experience in automation and web crawling
- Proficiency in Python and frameworks like Scrapy or BeautifulSoup
- Experience with distributed systems and parallel processing
- Knowledge of HTML, CSS, and JavaScript for web interaction
- Strong problem-solving skills and attention to detail
Skills
pythonweb crawlingautomationscrapydata processing