companybluefishai logo

Senior Data Acquisition Engineer

bluefishaiNew York
Hybrid Full-time

Clicking Apply Now takes you to AutoApply where you can tailor your resume and apply.


Unlock Your Potential

Generate Job-Optimized Resume

One Click And Our AI Optimizes Your Resume to Match The Job Description.

Is Your Resume Optimized For This Role?

Find Out If You're Highlighting The Right Skills And Fix What's Missing

Experience Level

Senior

Qualifications

What You’ll Be Doing Develop a scalable web data acquisition platform beneficial to multiple teams, enabling safer and more efficient data ingestion. Evaluate trade-offs between cost, reliability, and performance in essential architectural decisions. Create shared abstractions and tools that simplify the addition, maintenance, and operation of scrapers in production. Ensure system reliability by implementing smart retries, backoff strategies, error handling, and failure recovery. Continuously enhance the resilience, accuracy, and maintainability of scraping and data ingestion infrastructure. Integrate observability into data pipelines through logging, metrics, and alerting. Monitor data quality and enhance end-to-end data pipelines. Work cross-functionally to support new data sources and adapt to evolving product requirements. Qualifications Minimum of 5 years of experience in building and operating production-grade backend, data, or platform systems, with hands-on experience in web data acquisition or scraping. Experience managing systems in dynamic environments with frequent changes, partial failures, and external constraints. A product-oriented engineering mindset with strong problem-solving skills and a sense of ownership in production environments. Solid understanding of networking fundamentals, including TLS/SSL behavior, timeouts, and failure modes. Hands-on experience with browser automation tools. Proficiency in using CSS selectors and XPath for data extraction. Experience designing systems with robust retry logic, error handling, testing, and observability. Proficient in SQL and NoSQL databases.

About the job

About the Position

As a Senior Data Acquisition Engineer, you will take ownership of and enhance the systems designed for large-scale web data collection. Your responsibilities will include the design and maintenance of high-performance scraping and ingestion infrastructure that allows various teams to systematically add and manage data sources with reliability.

This role is integral to the Data Engineering team, concentrating on creating scalable, observable, and resilient systems that effectively manage production data. You will collaborate closely with data engineers and product teams to elevate data pipelines and ensure that data acquisition processes scale alongside business growth.

Join us to shape the future of AI-driven technologies and contribute significantly to impactful real-world applications.

This position is based in our NYC office and operates under a hybrid working model.

About bluefishai

bluefishai is at the forefront of AI-driven technology, pioneering solutions that leverage data to create impactful applications in various industries. Our commitment to innovation and excellence drives our team to push the boundaries of what's possible with data.

Similar jobs

Tailoring 0 resumes

We'll move completed jobs to Ready to Apply automatically.