Senior Data Engineer - Informatica & PySpark Expertise

Role Overview qodeworld is hiring a Senior Data Engineer with deep experience in Informatica and PySpark. This role is based in Pennsylvania and requires working onsite three days each week. What You Will Do Apply 8–10 years of hands-on experience in data engineering and data analysis. Design, develop, and optimize ETL processes using Informatica PowerCenter and Informatica Data Quality (IDQ). Build and maintain large-scale data processing and analytics solutions with advanced PySpark skills. Work with Hadoop technologies, including HDFS, Hive, Sqoop, and MapReduce. Develop streaming and batch data pipelines using Python and Kafka. Use strong knowledge of database concepts, data modeling, and ETL workflows to support data architecture and design. Manage the full ETL lifecycle: data extraction, ingestion, quality checks, normalization, and loading. Contribute to Agile projects, using Jira for tracking and delivery. Engage directly with clients, coordinating across the software development lifecycle and communicating clearly with stakeholders. Preferred Qualifications Experience with AWS data services and analytics tools. Familiarity with machine learning models and AI concepts. Knowledge of data modeling tools such as Erwin.

About the Role qodeworld is hiring a Senior Data Engineer with deep experience in Informatica and PySpark. This position focuses on building and optimizing data solutions for large-scale analytics. The role is based in either Cleveland, OH or Pittsburgh, PA and requires onsite work three days per week. Main Responsibilities 8–10 years of experience in data engineering and data analysis. Hands-on expertise with Informatica PowerCenter and Informatica Data Quality (IDQ) for ETL design, development, and optimization. Advanced proficiency with PySpark for processing, transforming, and analyzing large datasets. Strong knowledge of Hadoop technologies, including HDFS, Hive, Sqoop, and MapReduce. Solid programming skills in Python and Kafka to build both streaming and batch data pipelines. Thorough understanding of database concepts, data modeling, data design, and ETL workflows. Experience across all phases of the ETL lifecycle: data extraction, ingestion, quality checks, normalization, and loading. Comfort working in Agile environments, using tools such as Jira to support project delivery. Proven background in client-facing roles, with strong communication and leadership skills to manage the software development lifecycle. Preferred Skills Familiarity with AWS data components and analytics. Understanding of machine learning models and AI concepts. Experience with data modeling tools like Erwin. Qualifications Bachelor’s or Master’s degree in Computer Science or a related discipline. Strong problem-solving skills and a collaborative approach to cross-functional teamwork.

Join our dynamic team as an Informatica MDM Consultant in Philadelphia! We are seeking a skilled professional who is passionate about data management and looking to make a significant impact within our organization. In this role, you will leverage your expertise in Informatica Master Data Management to help our clients optimize their data strategies.

At CapTech, our Data Engineering consultants empower clients to construct and sustain sophisticated data systems that integrate data from diverse sources to facilitate informed decision-making. We specialize in developing pipelines and preparing data for utilization by data scientists, analysts, and other data systems. Our passion lies in problem-solving and delivering innovative solutions tailored to our clients' needs. As Cloud Data Engineers, we leverage the cloud infrastructure of our clients to provide immediate value and ensure scalability for future growth. We thrive in a collaborative environment, offering numerous opportunities to learn from and share insights with fellow developers, architects, and clients.Your Impact and Responsibilities:Serve as a trusted advisor to customers, offering best practices, methodologies, and technologies for implementing robust data engineering solutions.Design, implement, and maintain modern data pipelines that deliver optimal solutions using appropriate cloud technologies.Collaborate with product owners and business subject matter experts (SMEs) to analyze customer requirements and deliver sustainable engineered solutions.Provide technical leadership and collaborate across teams to ensure alignment of technical solutions with customer needs.Stay abreast of the latest cloud technologies, patterns, and methodologies; effectively communicate results and ideas to key stakeholders.

Join Capital One, a leading financial services company, as a Senior Data Engineer - Principal Associate based in Scranton, Pennsylvania. Are you enthusiastic about innovating within the tech space? Do you thrive in a dynamic, collaborative, and iterative environment? Here at Capital One, you will collaborate with a diverse team of creators, problem solvers, and innovators dedicated to addressing real-world challenges and fulfilling customer needs.As a Data Engineer at Capital One, you will be pivotal in spearheading significant transformations across the organization, merging data with cutting-edge technologies.

Join Veeam Software as a Senior Sales Engineer - Data & AI Security, where you will play a pivotal role in delivering innovative solutions in the field of data protection and AI security. You will collaborate closely with both sales and technical teams to drive customer success and ensure the implementation of our cutting-edge technologies.

Join the largest pharmaceutical company in the USA as a Senior Network/Data Engineer!Immediate interviews and hiring for multiple positions available.As a Senior Network/Data Engineer, you will be responsible for designing and implementing LAN networks within a sophisticated pharmaceutical data center and campus setting, supporting various installation and migration projects.You will ensure the quality and performance of network design and implementation, demonstrating exceptional troubleshooting skills during LAN implementation activities.Your role will involve participating in and executing low-level design, installation planning, change control, documentation creation, physical installation, connectivity, service activation, and testing of new network hardware.Required: Minimum Cisco CCNP level experience with Cisco LAN Switches, Cisco Access Points, and Wireless LAN Controllers in a campus environment. Familiarity with the Cisco Nexus platform is highly desirable.Experience with Cisco Prime is extremely desirable, along with the implementation of complex wireless solutions in the enterprise.Excellent communication skills are essential. Proficiency in Microsoft Office products, including SharePoint and Visio, is required.

Join Veeam Software as a Senior Sales Engineer specializing in Data & AI Security. In this pivotal role, you will be responsible for driving sales success through your technical expertise in our cutting-edge data security solutions. You will work closely with sales teams and customers to ensure they understand the value of our products, helping them to secure their data environments effectively.

Join our dynamic team at Hudson Manpower as a Senior Security Engineer specializing in Workday. We seek a hands-on developer who possesses deep knowledge of HCM modules and can adeptly customize workflows and fine-tune the system. This role places a strong emphasis on Performance Management and Performance Metrics within the Workday environment.As a key member of our Information Technology department, you will utilize your extensive experience with industry-standard Red Team testing tools (such as Cobalt Strike, Mythic C2, Rubeus, Bloodhound, and Covenant) to safeguard our financial institution against advanced persistent threats without relying on phishing techniques.Your expertise in Red Team concepts, tools, and automation strategies will be pivotal in defending our infrastructure. A thorough understanding of the MITRE ATT&CK framework, vulnerability measurement, and system hardening for both Windows and Linux environments is essential.We are looking for someone who can creatively modify payloads to evade detection and has strong skills in at least one scripting language (Python, Ruby, PowerShell, Bash, etc.). Experience with cloud environments (AWS, GCP, Azure) and conducting offensive security exercises against various operating systems is highly desirable.Successful candidates will have a history of delivering Red Team projects, including the creation and execution of statements of work and risk mitigation strategies while collaborating with stakeholders to remediate findings. Familiarity with automated configuration management tools like Chef and the discovery and exploitation of vulnerabilities in AI systems will be advantageous.If you hold recognized industry certifications such as GPEN, GXPN, GREM, eCPTX, eCPPT, OSCP, OSWE, CISSP, CPSA, CRT, or similar, we encourage you to apply. Knowledge of industry security standards (TIBER-EU, CBEST, NIST Cyber Security Framework, ISO27002, etc.) is also a plus.

About Shift4 Payments Shift4 Payments, Inc. streamlines complex payment systems for businesses around the world. As a leader in commerce-enabling technology, Shift4 supports billions of transactions each year for hundreds of thousands of clients across many industries. Learn more at www.shift4.com. Role Overview Shift4 is growing quickly and is hiring a Senior AWS Cloud Data Engineer to join its in-house engineering team. This position plays a key part in shaping technology, architecture, and engineering delivery within the AWS Cloud Data Services group. What You Will Do Design, build, and maintain data infrastructure solutions on AWS Support data-driven projects across the organization Collaborate with project managers, business analysts, and software engineers Help ensure systems are available, reliable, and scalable Take ownership of assigned projects Who We’re Looking For Motivated, self-starting engineers who enjoy working on challenging projects Strong background in AWS cloud data engineering Comfort working with cross-functional teams Location This is an on-site role based at one of the following Shift4 offices: Tampa, FL; Atlanta, GA; Las Vegas, NV; or Center Valley, PA.

About UsAt Aurora, our mission is to provide safe, fast, and widespread access to self-driving technology.Explore The Aurora Driver which aims to usher in a new era of mobility and logistics, making transportation safer, more efficient, and more accessible to everyone.Join a team of passionate and intelligent individuals as we tackle complex challenges and expand our expertise. For the latest updates, visit aurora.tech or follow us on LinkedIn.Aurora is committed to hiring individuals from diverse backgrounds to build a transportation ecosystem that enhances road safety, ensures timely delivery of goods, and improves mobility for all.We are seeking a Senior Software Engineer to join our Vehicle Data Management team. This team is responsible for managing data flows, including autonomy sensor data, vehicle logs, simulations, models, and training sets. The ideal candidate will design and implement systems that facilitate efficient communication between physical offloaders and cloud infrastructure.Your Role:Design and implement scalable on-vehicle/offload and cloud data systems that can handle hundreds of petabytes of multimodal data (sensor, log, simulation, and ML datasets).Enhance the reliability and performance of offline data workflows, focusing on throughput, latency, and discoverability (indexing, metadata, search).Create internal tools, libraries, and operational patterns to evolve and scale data pipelines.Manage key services end-to-end, including design, implementation, rollout, operational management, observability, and incident response.Collaborate directly with stakeholders across onboard systems, autonomy, ML, and infrastructure to deliver effective solutions.

Join our dynamic team at Sonsoft Inc. as a Big Data Engineer. In this role, you will leverage cutting-edge technologies to design, develop, and manage large-scale data processing systems. You will work closely with data scientists and analysts to ensure our data infrastructure supports robust data analytics and reporting.

Join Sonsoft Inc. as a Big Data Engineer and take your career to the next level! We are looking for talented individuals who are passionate about leveraging large data sets to drive impactful business decisions. In this role, you will collaborate with cross-functional teams to design, develop, and maintain robust data processing systems that enable our organization to harness the full potential of big data.

Role overview Shift4 Payments is looking for an AWS Cloud Data Engineer to help build and maintain cloud-based data processing systems. The position involves designing, developing, and supporting solutions that keep data flowing smoothly and analytics reliable. Locations Atlanta, Georgia Center Valley, Pennsylvania Las Vegas, Nevada Tampa, Florida What you will do Design and develop data processing systems on AWS Maintain and improve cloud-based analytics infrastructure Support consistent data flow for analytics and reporting

As a Lead Machine Learning Engineer at CapTech, you will be at the forefront of designing and implementing innovative, data-driven solutions tailored to our clients' needs. Your primary focus will be on developing and deploying scalable machine learning systems within enterprise settings. Enjoy a collaborative atmosphere where you can learn from and share insights with fellow analysts, architects, and clients.Key Responsibilities:Collaborate with clients, data scientists, engineers, and cross-functional teams to develop comprehensive machine learning solutions and identify novel data science strategies that address business challenges.Offer technical leadership and work closely with teams to ensure alignment of technical solutions with customer requirements.Translate client specifications into data-centric processes, models, and analytical metrics.Analyze and transform extensive datasets across various enterprise-level platforms (e.g., AWS, Azure, GCP).Design, build, and deploy sophisticated analytical solutions utilizing client data (including recommender systems, natural language processing, and risk assessment).Optimize and scale machine learning systems to meet client needs effectively.Advance CapTech’s Machine Learning and Data Science practices by delivering client presentations, crafting proposals, participating in business development events, and mentoring junior data scientists and engineers.

About Us:At Perpay, a certified B Corp and one of Philadelphia’s most influential growth-stage startups, our mission is to enhance the financial stability of everyday Americans. Over the past decade, we've achieved a strong product-market fit and an efficient operating model across various offerings, establishing Perpay as the top financial partner for individuals with subprime credit.With over 500,000 customers utilizing more than $1 billion in spending power, we are at a crucial turning point. As we scale our operations, launch new products, and increase our impact, we seek passionate teammates ready to embark on this journey with us.We are proud to partner with industry leaders such as First Round Capital and L Catterton.Our impactful products include:Perpay Marketplace: A platform that merges interest-free payments with modern e-commerce, making ownership more affordable while promoting healthy repayment behaviors.Perpay+: A tool that uses Marketplace repayment history to help members track and improve their credit scores with all three major credit bureaus.Perpay Credit Card: A credit card offering enhanced access to the benefits of a World Mastercard, eliminating barriers like high security deposits and low approval rates.Our team thrives on in-person collaboration in our unique center-city Philadelphia office, designed to be a comfortable "home away from home". This space not only offers stunning river views but also promotes rapid product development, strong relationships, and career growth. The excitement from achieving significant milestones is palpable. While we primarily work in the office, we provide reasonable flexibility for personal needs, such as caring for sick children or handling urgent errands, and we organize official remote weeks around major holidays. If you are passionate about making a meaningful impact, fostering collaboration, and enjoying generous perks, Perpay is the ideal place for you in Philadelphia.About the Role:As a Senior Data Scientist at Perpay, you will play a key role in our business's daily success, driving both new and existing product initiatives with your analytical prowess and machine learning expertise. Additionally, you will help shape the future of our data science team by developing scalable processes and systems to support our rapid growth. This is a thrilling opportunity to positively influence Perpay’s mission and improve the lives of our customers. We seek an individual who is enthusiastic about all facets of data science, including machine learning, exploratory data analysis, and the effective communication of results for real-time deployments. This pivotal position is part of our data science team and reports to the Data Science Manager.

Suvoda is looking for a Data Engineer to join the Product Development team onsite in Conshohocken, Pennsylvania. This role is well-suited for recent graduates who have a solid technical background and want to work with data in support of clinical trial technology. The team focuses on building and maintaining a cloud-native data platform that underpins critical solutions for the clinical research field. What you will do Create domain-oriented data products using GraphQL APIs Work with AWS DMS replication to Aurora PostgreSQL to enable near real-time reporting Collaborate closely with other team members to deliver reliable and scalable solutions Who should apply Recent graduates with a technical foundation Those eager to begin a career in data engineering Individuals interested in contributing to projects that support human health This is a full-time, onsite position in Conshohocken, Pennsylvania.

IntegriChain is seeking a talented Senior Software Engineer to join our expanding Engineering team in Philadelphia. This role is pivotal in enhancing our capabilities to process extensive healthcare and pharmaceutical data at an enterprise scale. As a hands-on engineer, you will utilize your programming expertise to focus on automation, performance optimization, and maintaining high-quality standards. You will be responsible for designing, developing, and delivering robust product functionalities that meet reliability and quality benchmarks.In this role, you will collaborate closely with product managers, QA teams, and other stakeholders to translate business requirements into effective technical solutions. Your leadership in anticipating challenges, improving engineering practices, and ensuring system scalability, performance, and maintainability will be essential. You will take ownership of significant components, conduct thorough code reviews, and actively contribute to our long-term technical roadmap.Develop and maintain stable products using the latest technologies and frameworks while adhering to Agile and DevOps methodologies.Deliver high-quality solutions on time, addressing problems without compromising deadlines.Utilize AI tools like Copilot for code generation, test case creation, code analysis, and optimization.Assist in redesigning and upgrading applications to incorporate the latest technologies.Be a proactive problem solver, a leader, and a collaborative team player.Engage in all phases of the software development lifecycle, including design, development, testing, code review, and deployment.Embrace learning opportunities from team members and day-to-day experiences.

About UsAurora Innovation is dedicated to safely and swiftly advancing self-driving technology for everyone. Our goal is to establish a transformative era in mobility and logistics that enhances safety, efficiency, and accessibility.At Aurora, you will work on intricate challenges alongside a team of passionate and intelligent professionals, developing your expertise and expanding your knowledge. For the latest updates from Aurora, visit aurora.tech or follow us on LinkedIn.We are looking for a remarkable Senior Data Analyst to join our Data Solutions team. In this role, you will focus on automating the monitoring and analysis of the Aurora Driver’s on-road performance and operational efficiency. Your data-driven insights will play a significant role in achieving our product and commercialization objectives. The ideal candidate is adept at collaborating across departments and managing complex projects with Business Development, Operations, Product, Software, and Strategy teams. If you are enthusiastic about leveraging cutting-edge Data Analytics to create impactful tools and workflows that revolutionize transportation, we want you on our team.

Join Dun & Bradstreet: Pioneering Data for a Better FutureAt Dun & Bradstreet, we believe in the transformative power of data to forge a brighter tomorrow. As a global leader in business decisioning data and analytics, we empower companies around the globe to grow, mitigate risks, and drive innovation. For over 180 years, organizations have relied on us to convert uncertainty into opportunity. Our diverse, global team champions creativity, collaboration, and bold thinking. Are you prepared to make a significant impact and help define the future? Join us! Discover career opportunities at dnb.com/careers.As a Senior Microsoft Fabric Engineer, you will play a vital role in overseeing the governance and security of data within the Fabric platform, ensuring its readiness for analytics and AI-driven workloads. This position involves leveraging Microsoft Purview and Fabric’s inherent governance capabilities to uphold security, compliance, and data quality standards across all data and reports. Your responsibilities will include implementing data classification, access controls, and data loss prevention strategies in accordance with established guidelines. As the Senior Microsoft Fabric Engineer, you will monitor the impact of Copilot-driven activities on capacity and performance across Fabric workloads, supporting necessary adjustments to ensure optimal operations. By maintaining consistent governance, scalable design, and robust platform stewardship, you will contribute to a stable and AI-ready environment that fosters high-quality data and analytics outcomes.