remote
Web Scraping Engineer II - YipitData
Software Engineer
Seeking a Web Scraping Engineer II to develop and maintain data extraction systems. This role involves leveraging Python and cloud technologies to build robust scraping solutions for market research.
About the role
Key Responsibilities
- Design, develop, and maintain scalable web scraping systems.
- Implement data extraction pipelines for various online sources.
- Ensure data quality, accuracy, and integrity throughout the extraction process.
- Collaborate with data analysts to understand data requirements and deliver insights.
- Optimize scraping performance and troubleshoot issues.
- Develop and maintain APIs for data access.
Requirements
- Proven experience in web scraping and data extraction techniques.
- Proficiency in Python and relevant libraries (e.g., Scrapy, BeautifulSoup).
- Experience with SQL databases for data storage and retrieval.
- Familiarity with cloud platforms (e.g., AWS, GCP) for deployment and scaling.
- Understanding of data processing and cleaning methodologies.