Senior Python Data Scraping Engineer

Role overview Jobgether is hiring a Senior Python Data Scraping Engineer for a freelance, remote position based in Portugal. This role focuses on developing scalable systems to extract web data from complex and changing online sources. The work blends hands-on engineering with creative problem-solving, supporting workflows that combine AI tools and human ove…

Join Toloka AI as a Freelance Data Scraping Engineer and contribute to innovative projects in the field of data extraction and analysis. In this role, you will leverage your Python skills to develop robust scraping tools and gather valuable data from various online sources.

We are seeking a talented and motivated Python Developer (Data Engineer) to join our dynamic team at inetum2 in beautiful Lisbon. In this role, you will leverage your expertise in Python programming and data engineering to develop and maintain robust data pipelines, ensuring seamless data integration and processing.Your primary responsibilities will include building scalable data architectures, collaborating with cross-functional teams to understand data requirements, and optimizing existing data workflows for performance and efficiency.If you are passionate about data-driven solutions and thrive in a fast-paced environment, we want to hear from you!

About GapstarsGapstars, a premier software development services firm based in the Netherlands, specializes in cultivating remote, agile teams in Sri Lanka and Portugal for groundbreaking tech companies. Our workforce of over 275 TechStars is dedicated to transforming scalable software into high-performance products that redefine industry standards. We partner with ambitious tech firms aiming to dominate their markets.Position OverviewWe are looking for a Senior Software Engineer to enhance a rapidly expanding FinTech’s cloud-based data and analytics platform utilized by leading global market players. You will design and implement dependable, secure, and high-quality systems that facilitate extensive data distribution, with a strong emphasis on hands-on engineering, automation, and data visualization.This role is hybrid, located in Lisbon, Portugal.Key ResponsibilitiesDrive technological advancements within a fast-growing fintech company by shaping its tech solutions.Interpret business requirements and convert high-level goals into well-structured, scalable solutions utilizing the best practices of GCP.Develop robust, secure, and reliable systems, implementing practical workarounds to ensure service continuity during failures.Plan and execute well-structured releases while minimizing disruption and establishing clear rollback strategies.Guide and support junior team members, promoting a culture of learning and professional development.QualificationsAt least 5 years of experience in software development using Python and/or Java.Strong expertise in databases and cloud data warehouses (SQL, PostgreSQL, MySQL, etc.).

Role overview Natixis in Portugal seeks a Python Data Engineer to join the BPCE SI team in Lisbon. This position focuses on managing and improving data pipelines that support business decisions. Fluency in French is essential for daily work and collaboration. What you will do Build, maintain, and optimize data pipelines using Python Support data processing needs for analytics and reporting Work closely with colleagues to strengthen data-driven decision-making What we are looking for Fluent French language skills Strong programming experience with Python This role is based in Lisbon.

Jobgether is looking for a Senior Data Engineer based in Portugal. This position centers on building and maintaining data solutions that support analytics and reporting across the company. What you will do Design scalable data models and pipelines to handle growing data needs Implement processes for integrating data from various sources Work closely with data scientists and analysts to clarify requirements and deliver useful datasets Monitor and improve data quality, while maintaining security standards Collaboration This role involves regular interaction with cross-functional teams. Expect to partner with colleagues across analytics, engineering, and business functions to ensure data is accurate, accessible, and secure. Location This position is based in Portugal.

Join Altersolutions as a Senior Data Engineer and play a pivotal role in shaping the future of data management. In this dynamic position, you will leverage your expertise to design, develop, and maintain robust data pipelines and architectures, ensuring seamless data flow and accessibility across various platforms. You will collaborate with cross-functional teams to transform data into actionable insights, driving decision-making processes and enhancing business outcomes.

Join our innovative team at inetum2 as a Senior Data Engineer, where you will play a pivotal role in designing and implementing data solutions that drive business insights and enhance decision-making processes. You will leverage your expertise in data architecture, ETL processes, and database management to build robust data pipelines and ensure data integrity across various platforms.

Fundraise Up seeks a Senior Data Engineer located in Portugal for a remote position. The main focus is building and maintaining data systems that power fundraising technology. Role overview This position centers on designing and implementing scalable data solutions. The goal is to support clients as they work to improve their fundraising strategies. Key responsibilities Build and maintain reliable data systems tailored for fundraising technology Design scalable solutions that handle growing data needs Implement systems that help clients optimize their fundraising efforts Location This role is remote and open to candidates based in Portugal.

Join the Revolution in Healthcare at knokAt knok, we are pioneers in humanizing the digital transformation of healthcare. Our vision is to create a world where everyone has timely access to quality healthcare through innovative digital technology, fostering a more equitable society. Our commitment to this mission is evident in every team member who shares our passion.With our Digital Front Door strategy, knok unites patients, providers, and healthcare professionals in a seamless experience. Our API-first, white-label platform delivers a continuous, engaging, and personalized healthcare journey for all conditions through our advanced Patient Journey Engine.Utilizing insights from regular clinical practice, we harness readily available data to enhance care automation and drive financial efficiencies. Since our inception in 2015, we have facilitated over 2.5 million clinical interactions across more than 12 countries. Our scalable platform is AI-ready, empowering data-driven care to improve outcomes at every stage of life.Are you prepared to join us in transforming healthcare and making a real difference in the lives of many?About the RoleWe are seeking a Senior Data Engineer to join our dynamic team. In this role, you will be instrumental in designing robust, scalable, and high-quality data systems. You will build data pipelines, convert data into actionable insights, and ensure that the organization’s decision-making is supported by solid data foundations. If this resonates with you, we would love to hear more!Integrate raw data from diverse sources into consistent and machine-readable formats.Design, build, and maintain data systems and ETL pipelines.Monitor, test, and optimize data flows for performance.Analyze large datasets to discover significant trends and insights.Assess business needs and translate them into effective data solutions.Identify opportunities to enhance data quality, accessibility, and reliability.Create and sustain analytical tools and reporting frameworks.Collaborate with analysts, product managers, and other stakeholders across various projects.Oversee the cost and efficiency of data infrastructure.About YouTo be a successful candidate for this role, we are looking for:A degree in Computer Science, IT, or a related field (Master’s degree is a plus).A proven track record as a Data Engineer or in a comparable position.Expertise in Python (OOP, unit testing, APIs, etc.).Strong understanding of ETL processes and data modeling.Familiarity with SQL and orchestration tools (e.g., Airflow).Experience with columnar databases (e.g., Redshift, BigQuery, Snowflake).Proficient in version control (e.g., GitHub).Experience with dbt and cloud services (AWS, Heroku) is advantageous.A solid analytical mindset and problem-solving skills.Strong communication skills and the ability to work collaboratively.

About UsConstructor is a cutting-edge platform revolutionizing search and discovery in e-commerce, specifically designed to enhance key metrics such as revenue, conversion rates, and profitability. Our proprietary search engine, developed entirely in-house, leverages advanced transformers and generative LLMs to optimize functionalities from search capabilities to personalized recommendations and shopping agents. As the largest department within Constructor, our engineering team has crafted an unparalleled engine that consistently outperforms competitors in A/B testing. We are dedicated to staying at the forefront of AI technology.Built to handle extreme scalability, our engine processes over 1 billion queries daily across multiple languages, serving clients in diverse countries, including industry giants like Sephora, Under Armour, and Petco.Our team thrives on solving complex problems and enhancing the experiences of our clients and colleagues alike. We prioritize values such as empathy, openness, curiosity, and continuous improvement, believing that empowering every team member leads to remarkable achievements.Established in 2019 by Eli Finkelshteyn and Dan McCormick, Constructor is a U.S.-based company committed to innovation and excellence.Job DescriptionThe Constructor Data Platform is essential for our internal data and ML teams, managing the ingestion of over 1 TB of compressed events each day and overseeing more than 6 PB of data within our data lake.Key Responsibilities: Develop and maintain comprehensive tools and infrastructure utilized daily by our data scientists and ML engineers. Create public-facing APIs for event ingestion (FastAPI) and real-time analytics (ClickHouse, Cube). Manage data storage in optimal formats (S3, ClickHouse, Delta). Facilitate data processing using technologies such as Python, Spark/Databricks, ClickHouse, AWS Lambda, and Kinesis. Implement robust monitoring solutions (Prometheus, OpenTelemetry, PagerDuty, Sentry). Ensure automated testing of data pipelines and maintain data quality. Provide cost observability and optimization capabilities. Offer comprehensive tools for developers to design, execute, test, and schedule data pipelines, along with necessary support and documentation. This platform is developed by our dedicated Data Lake Team and Data Infrastructure Team.About the Data Infrastructure TeamJoin us as a Senior Data Engineer on our Data Infrastructure Team, responsible for: Job scheduling and orchestration for data pipelines. Deployment and management of BI tools. Real-time analytics infrastructure (ClickHouse, AWS services).

About Indicium AI Indicium AI is a globally recognized consultancy specializing in artificial intelligence. We empower the world’s foremost enterprises to seamlessly integrate AI into their operations at scale. With a proven track record across diverse sectors including Financial Services, Energy & Utilities, Healthcare & Life Sciences, Retail & CPG, and Manufacturing, we provide unparalleled clarity, speed, and capability in unlocking AI-driven value. Supported by a team of over 600 AI specialists across five international offices, we collaborate with premier partners such as Anthropic, Databricks, AWS, OpenAI, and Microsoft to deliver impactful AI solutions swiftly and efficiently. About the Role We are seeking a Senior Data Engineer who will act as the technical lead during client projects, overseeing architectural decisions, managing the delivery of intricate data platforms, and establishing engineering standards throughout engagements. This position demands a blend of advanced engineering expertise and exceptional communication abilities to influence stakeholders, mentor team members, and uphold the highest standards of data platform delivery. Key Responsibilities Architecture & Technical Leadership Champion the comprehensive design and implementation of scalable, production-ready data architectures, including Data Warehouses, Data Lakes, and Lakehouses. Set forth data engineering standards, patterns, and guidelines for the team and client projects. Exercise complete technical independence—propose, challenge, and refine architectural solutions both with clients and internally. Advanced Pipeline Engineering Design, construct, and sustain mission-critical ELT pipelines. Enhance complex data workflows for optimal performance, cost-effectiveness, and long-term sustainability. Infrastructure & Platform Engineering Manage data infrastructure provisioning and lifecycle oversight using Infrastructure as Code (Terraform), ensuring scalability, security, and reliability at an enterprise level. Spearhead cloud architecture strategies across AWS and/or Azure, aligning platform decisions with client objectives and commercial considerations. Governance, Security & Reliability Develop and enforce data governance frameworks, including access policies, data quality standards, lineage, and observability, particularly in complex, regulated environments.

Join Sword Health as a Senior Data Engineer and be a key player in revolutionizing healthcare through our cutting-edge AI Care platform. In this role, you'll be instrumental in architecting a scalable lakehouse by migrating workloads to Apache Iceberg. You'll develop a robust Jobs API, construct Spark and Flink pipelines, and establish clear data contracts. We seek a candidate proficient in Python and Kafka, who embodies a collaborative, platform-first mindset. At Sword Health, data is central to our mission of creating a pain-free world, empowering our Data Team to drive decisions that impact millions of lives globally.

Location: UK or Portugal Department: Engineering Employment Type: Permanent About YLD: At YLD, our mission is to empower our clients to continuously advance. We partner with them to cultivate the skills and capabilities necessary to outpace the competition, recognizing that our success is intrinsically linked to theirs. We are a collective of inquisitive and talented professionals dedicated to making YLD a desirable workplace for growth and retention. As a remote-first consultancy, we specialize in software engineering, product design, and data solutions. With teams spread across London, Lisbon, and Porto, we develop digital solutions that provide lasting value long after project completion. About the Role: As a Senior Data Engineer, you will collaborate with a client team to develop innovative products. We seek individuals who are curious, passionate, driven, and enthusiastic. Proficiency in problem-solving and the ability to design scalable, resilient, and fault-tolerant architectures are essential. Strong communication skills and teamwork are vital as you navigate a fast-paced engineering environment, achieving remarkable results and delivering exciting projects. About You: A Senior Data Engineer at YLD possesses advanced skills across both data engineering and analytics engineering, understanding the nuances between the two. You have experience in building data pipelines and transformation layers, have improved chaotic data projects, and possess thoughtful opinions on data team dynamics, with the flexibility to adapt these views based on context. Your passion for your craft drives you to actively learn and embrace the diverse challenges that consultancy presents: varying clients, technology stacks, and problems. Your Day-to-Day Responsibilities Will Include: Delivering data solutions on client engagements: pipelines, transformation layers, models, and supporting infrastructure. Adapting to diverse client contexts, including legacy warehouse migrations and rescuing struggling transformation layers. Defining and evolving internal standards for data work, including testing, documentation, project structure, and code reviews. Mentoring and developing data engineers within the organization. Participating in client team activities (internal presentations, learning sessions, social events) to foster strong working relationships. Contributing to proposals and aiding in shaping project approaches.

DataCamp helps people and organizations build essential data and AI skills. Over 17 million learners and more than 6,000 companies, including most of the Fortune 1000, use DataCamp’s platform to close the data and AI skills gap. Role overview The Senior Data Engineer will join the Data Platform Engineering team, part of the Platform Engineering department. This team builds and maintains the data platform that underpins all of DataCamp’s operations, rather than focusing on analytics for a single business unit. The work is central to the company’s ability to make informed, data-driven decisions. What you will do Develop and maintain DataCamp’s Datalakehouse on Google Cloud’s BigQuery. Support reporting and analytics through Looker BI tools, which guide company OKRs and strategic decisions. Maintain a fully automated data pipeline using Terraform and Ansible for provisioning data engineering tools. Manage data ingestion with Airbyte and schedule data transformation tasks with Airflow. Build and update data marts with DBT for finance, engineering KPIs, engineering costs, and infrastructure costs. Ensure daily data refreshes run smoothly for leadership, data scientists, analysts, and engineers. Advise on best practices for data ingestion, data mart design, and product design. Contribute to enabling AI agents and Model Control Points (MCPs) as DataCamp adopts an agent-first engineering approach. The data platform will support both human users and AI functionalities. Requirements At least 3 years of experience as a data engineer. Experience with cloud-based data platforms, ideally Google Cloud’s BigQuery. Familiarity with data pipeline automation using tools like Terraform and Ansible. Hands-on experience with Airbyte for data ingestion and Airflow for orchestration. Experience building data marts with DBT. Strong understanding of BI tools, especially Looker. Ability to support both human and AI-driven data use cases. This position is based in Portugal.

At ciandt, we are leading the way in tech transformation, merging human expertise with Artificial Intelligence to develop scalable technological solutions.With a dedicated team of over 8,000 professionals globally, we have established successful partnerships with more than 1,000 clients over our 30-year journey. Our commitment to innovation in Artificial Intelligence drives our success.We are seeking a passionate Senior Data Engineer to be part of our dynamic global team. In this pivotal role, you will be instrumental in designing, developing, and maintaining high-quality, scalable data pipelines and infrastructure. Collaborating with multidisciplinary teams in Brazil, Europe, the US, and beyond, you will transform data into actionable insights that inform digital products and strategic business decisions.Key Responsibilities:● Architect, construct, and sustain robust data pipelines that handle both structured and unstructured data.● Execute efficient data ingestion, transformation, and storage processes utilizing cloud technologies.● Collaborate with stakeholders to align business objectives with effective data engineering solutions.● Monitor, troubleshoot, and enhance data pipelines to ensure optimal reliability and performance.● Facilitate data validation, testing, and comprehensive documentation processes.● Contribute to the design and implementation of contemporary data architectures, including data lakes, lakehouses, and data warehouses.● Employ Infrastructure-as-Code (IaC) methodologies for cloud resource management and provisioning.● Integrate innovative tools and frameworks to modernize existing data environments.● Uphold security, governance, and compliance throughout all phases of data handling.● Participate in agile teams, fostering continuous improvement and mentoring junior colleagues.Qualifications:● Extensive experience in developing data pipelines using Python and SQL.● Proven background in cloud platforms like AWS (Glue, Redshift, EMR) or Azure (ADF, Synapse).● Proficient in ETL/ELT tools and frameworks (e.g., Talend, Azure Data Factory, Airflow, DBT).● Familiar with modern data technologies and data architecture principles.

Join Inetum, a leading innovator in digital services, where we drive growth and transformation through advanced technology solutions. We are seeking a highly skilled and motivated Senior Data Engineer to become part of our dynamic team. In this role, you will play a critical part in designing, developing, and optimizing data solutions that enhance business intelligence and analytics capabilities.If you are passionate about data, excel in collaborative settings, and relish tackling complex problems, this is your chance to make a significant impact!

About UsAt Riskified, we empower businesses to unlock their ecommerce potential by mitigating risks. Our platform is trusted by some of the world’s leading brands and publicly traded companies to safeguard against chargebacks, combat fraud, and enhance customer loyalty. Our AI-driven fraud and risk intelligence solutions analyze interactions in real-time, offering robust identity insights that help our partners succeed in the competitive online landscape. We proudly collaborate with renowned companies across various industries, including Acer, Gucci, Lorna Jane, GoPro, and many more.We cultivate a collaborative and dynamic work environment, where innovative ideas flourish and meaningful contributions are valued. Our commitment to teamwork and excellence drives us to make a lasting impact in the ecommerce space, making Riskified a workplace of choice.About the Role*** Please note that this position will be recruited, hired, and employed by a third party.We are in search of a skilled and passionate Senior Data Platform Engineer to enhance and maintain our core data platform products, ensuring optimal performance, scalability, and stability. As a data-driven organization, we are committed to managing an increasing influx of data daily.In this role, you will be a vital part of the Data Platform team, responsible for overseeing a large-scale production data platform that serves as the backbone of our products. You will develop self-service tools to enhance platform accessibility for users in our R&D and Analytics departments, and act as a knowledge hub, consulting and sharing your expertise with your colleagues.What You’ll Be DoingLead the development of AI-ready foundations for our data platform, focusing on data quality, contracts, observability, and self-service tools that empower machine learning and AI-driven initiatives.Manage, provision, and monitor various platform products.Enhance the application infrastructure tightly integrated with our Data Platform tools.Provide guidance to development teams and mentor fellow team members.Utilize your creativity to solve complex, high-scale data challenges.Research and evaluate new technologies, starting from POC to deployment.

Join the dynamic Business Intelligence team at SIXT, where we design and maintain the cutting-edge SIXT Data Platform and its Data Catalogue, catering to thousands of users around the globe. Our flagship offering, the SIXT Data Shop, processes tens of millions of events daily, transforming them to address various consumer needs and delivering insights through a robust toolbox—from self-service batch processes to real-time ML model endpoints.Our current priorities include enhancing operational automation, broadening self-service data ingestion capabilities, optimizing ETL workflows, and enabling innovative applications for our clients. Our dedicated team of 20 Data Engineers and Business Intelligence Engineers embraces collaboration and is constantly on the lookout for the latest technological advancements. At SIXT, we celebrate success, agility, ownership, and cross-cultural collaboration. Join us and become a valued member of the SIXT family.YOUR ROLE AT SIXTBuild CRM Data Foundations: Create analytical infrastructure for our CRM initiatives and automate data pipelines to establish a premier customer intelligence environment.Drive Innovation: Investigate and apply the latest AWS and big data technologies to unveil hidden opportunities, enable new functionalities, and forge integrations for the SIXT Data Shop.Collaborate Cross-Functionally: Work alongside Data Engineers, BI Analysts, and Data Scientists to design optimal solutions for various analytical applications, including dashboarding, ad hoc analytics, data-as-a-product, and machine learning.Shape Platform Strategy: Contribute your expertise and innovative ideas to the vision and roadmap of the Data Platform.Ensure Data Integrity: Uphold exceptional work ethics and integrity while managing sensitive customer data, adhering to the highest standards of data protection.

QIMA seeks a Senior Manager of Data & Web Engineering based in Lisbon. This position leads a team dedicated to applying data and technology in support of marketing objectives. The role involves guiding the design and development of web applications that serve client requirements and help grow QIMA’s digital footprint. Role overview This leadership role centers on managing a team responsible for both data and web engineering. The focus is on building solutions that align with marketing strategies and deliver value to clients. Main responsibilities Oversee the design and development of web applications Direct efforts to use data and technology to advance marketing goals Ensure projects meet client needs and support QIMA’s digital presence Location This position is based in Lisbon.