Spark Sql

Spark Sql Developer Jobs

4 jobs found

10d ago

Pyspark/Python Data Engineer

Tata Consultancy Services Irving, TX

Irving, TX

10d ago

$100k - $120k/yr Full time Python Amazon Web Services Apache Airflow

Apply Now

Job Description

Join Tata Consultancy Services as a PySpark and Python Data Engineer to help shape scalable data pipelines that power analytics across the organization. You will design, build, and optimize ETL and ELT processes with PySpark, collaborating with data analysts, architects, and DevOps to ensure top-tier data quality, performance, and reliability. This onsite role is based in Irving, Texas with a salary range of USD 100,000 to 120,000 per year.

Responsibilities

Design, develop, and maintain ETL/ELT pipelines using PySpark
Write optimized and scalable PySpark transformations using DataFrames and Spark SQL
Develop reusable and efficient Python-based data processing components
Ensure data quality, integrity, and performance across pipelines
Perform debugging, performance tuning, and optimization of PySpark jobs
Collaborate with cross-functional teams (Data Analysts, Architects, DevOps)
Contribute to CI/CD pipelines and deployment workflows for data applications
Monitor and troubleshoot data workloads in production environments

Requirements

Strong hands-on experience in PySpark (Spark SQL, DataFrame API)
Advanced proficiency in Python (data processing, performance tuning, modular coding)
Solid understanding of ETL design patterns and data pipeline architecture
Good working knowledge of SQL for data transformation and analysis
Experience with data processing in distributed environments
3–8 years of experience in Data Engineering / PySpark development
Proven hands-on project experience in PySpark + Python
Bachelor of Computer Science

Technologies

PySpark
Spark SQL
DataFrame API
Python
SQL
Amazon S3
AWS Glue
AWS EMR
Airflow
Snowflake
Git

Experience

3–8 years of experience in Data Engineering / PySpark development
Proven hands-on project experience in PySpark + Python

Salary Range

USD 100,000 - 120,000 per year

Location

Irving, TX (onsite)

Job Function

Technology

Role

Engineer

Job ID

416300

Desired Skills

Python

Desired Candidate Profile

Qualifications: Bachelor of Computer Science

View full job page →

11d ago

Data Engineer

INSPYR Solutions Spring, TX

Spring, TX

11d ago

$70 - $85/hr Contract Analytics Azure Azure Data Factory

Apply Now

Job Description

INSPYR Solutions offers an onsite contract opportunity in Spring, Texas for a Data Engineer who will design scalable data models and Azure based pipelines to empower analytics and business intelligence across the organization. This role is carved for collaboration, technical growth, and delivering reliable data infrastructure. The compensation range is USD 70 to 85 per hour, with the work onsite three days per week.

Benefits

Comprehensive medical benefits
Competitive pay and 401(k)
Retirement plan
Onsite collaboration with a dedicated team and access to ongoing learning opportunities
Contracts designed to support career growth and practical data engineering impact

Responsibilities

Design and implement data models that support enterprise reporting and analytics, maintaining dimensional models and semantic layers while applying data warehousing best practices
Build scalable, modular ETL and ELT pipelines using Azure Data Factory, Synapse Pipelines, and Databricks
Transform and structure raw data with Spark SQL, DBT, and Azure Synapse to deliver analytics-ready data, including data quality checks and schema evolution strategies
Leverage Azure technologies such as SQL Server, Synapse Analytics, Databricks, and Delta Lake to construct and manage scalable data infrastructure and pipelines
Enable CI/CD and DevOps by integrating data workflows with Azure DevOps and Git for version control, automated testing, and continuous deployment
Support reporting and visualization efforts by collaborating with analysts and BI developers to optimize data models for Power BI
Troubleshoot and optimize data pipelines and models to ensure high availability and performance of data systems
Stay current with evolving data engineering practices, tools, and technologies to maintain a forward-looking data stack

Requirements

5 to 7 years of experience in data engineering or related roles
Proven expertise in data modeling, particularly dimensional modeling, and data warehouse design
Hands-on experience with Azure Data Factory, Synapse Analytics, Databricks, and DBT
Strong SQL and Spark SQL skills
Experience with CI/CD, DevOps, and version control (Git, Azure DevOps)
Familiarity with Power BI or similar visualization tools
Solid understanding of data integration, data quality, and data transformation best practices
Strong problem-solving abilities and a collaborative mindset

Technologies

Azure Data Factory
Synapse Analytics
Databricks
DBT
Spark SQL
SQL Server
Delta Lake
Power BI
Azure DevOps
Git

View full job page →

1mo ago

Senior Python Developer with Spark

CGI Group, Inc. Reston, VA

Reston, VA

1mo ago Closed

$99k - $154k/yr Full time Python Senior Apache Spark

This position is no longer accepting applications (closed Jun 22, 2026).

Job Description

Senior Python Developer with Spark to design, build, and optimize large-scale data processing systems in a cloud-native AWS environment, based in Reston, VA with hybrid work options.

Responsibilities

Design, implement, and optimize expansive data processing pipelines within a cloud-native AWS framework.
Build high-performance data pipelines using Spark and Python while handling complex relational datasets.
Identify bottlenecks in data workflows, optimize Spark jobs, and author advanced SQL to support hierarchical and analytical use cases.
Collaborate with cross-functional teams to deliver scalable, reliable data solutions that support critical business functions in a financial services setting.

Requirements

8+ years of hands-on Python for data engineering, including building and maintaining data pipelines.
Deep expertise in Apache Spark, including partitioning, caching, broadcast joins, shuffle optimization, understanding of DAGs/stages/tasks, and memory/resource management.
Extensive experience with big data ecosystems such as Hadoop, Hive, and EMR.
Advanced proficiency in SQL, including recursive CTEs for hierarchical data, query optimization, indexing strategies, and execution plan analysis.
Strong experience with AWS services including EMR, Lambda, Step Functions, EventBridge, Redshift, S3, and Glue.
Experience building and consuming APIs, along with data transformation and ingestion workflows.
Proven ability to work with large-scale datasets, performing data analysis and extracting actionable insights.
Familiarity with data modeling concepts (normalized/denormalized structures, handling hierarchical data).
Hands-on experience with CI/CD pipelines and tools such as GitLab and Terraform.
Strong understanding of performance troubleshooting in distributed systems and identifying bottlenecks.
Ability to clearly explain technical decisions, especially around Spark optimization and SQL logic.
Strong analytical thinking, problem-solving skills, and attention to detail.
Effective collaboration skills in cross-functional, matrixed environments.
Education: Bachelor's degree in Computer Science, Information Systems, or a related field.
Desired: Experience in financial services or regulated environments.
Desired: Exposure to data visualization tools.
Desired: Familiarity with event-driven architectures on AWS.

Technologies

Python
Apache Spark
Hadoop
Hive
EMR
AWS
Lambda
Step Functions
EventBridge
Redshift
S3
Glue
SQL
GitLab
Terraform
APIs

Benefits

Competitive compensation
Comprehensive insurance options
Matching contributions through the 401(k) plan and the share purchase plan
Paid time off for vacation, holidays, and sick time
Paid parental leave
Learning opportunities and tuition assistance
Wellness and Well-being program

Skills

Amazon Web Services Cloud
Detail-oriented
GitHub
Problem Solving
Python
SQL
Terraform

View full job page →

1mo ago

Lead Software Engineer - Python/PySpark/SQL

JPMorganChase Jersey City, NJ

Jersey City, NJ

1mo ago Closed

$152k - $215k/yr Full time Python Apache Spark Big Data

This position is no longer accepting applications (closed Jun 20, 2026).

Job Description

The Lead Software Engineer role within JPMorgan Chase's Asset & Wealth Management division centers on delivering secure production code and building data pipelines in a cloud-based AWS environment. The position emphasizes hands-on development alongside technical leadership, using Python, PySpark, and SQL to support wealth management marketing solutions.

Location and Compensation

Location: Jersey City, NJ (onsite)

Salary: USD 152,000 - 215,000 per year

Minimum experience: 5 years

Responsibilities

Designs and implements innovative software solutions, handling complex problems beyond routine approaches and breaking down technical challenges for actionable outcomes.
Leads wealth management marketing software initiatives with a strong grasp of business concepts in the domain.
Develops secure, production-grade code and conducts reviews and debugging of colleagues' work.
Proactively identifies opportunities to automate recurring remediation tasks to improve overall system stability.
Steers evaluation sessions with external vendors, startups, and internal teams to assess architectures, credentials, and fit within current information ecosystems.
Champions communities of practice across the software engineering organization to promote adoption of new and advanced technologies.
Contributes to an inclusive team culture that emphasizes diversity, opportunity, and respect.

Requirements

Formal training or certification in software engineering accompanied by five or more years of applicable experience.
Hands-on experience delivering system design, application development, testing, and operational stability for wealth management marketing software.
Practical development experience with Python, PySpark, SQL, and AWS services.
Capability to develop and optimize scalable data pipelines and applications using Python and PySpark within the AWS cloud, enabling efficient data processing.
Solid understanding of AWS services such as S3 and EMR to design robust data solutions that meet business needs and improve data access and analytics.
Familiarity with software applications and processes within a cloud-centric architecture.
Ability to identify improvement opportunities in existing applications to boost stability and streamline the platform.
Proven ability to collaborate with engineers and developers to ensure a standardized, optimized, secure, reliable, and accessible platform that supports business and technology needs.
Commitment to operational excellence through root cause analysis and continuous improvement.
Proficiency across the full Software Development Life Cycle.
Experience working with partner teams to build reusable services that meet solution requirements.

Technologies

Python
PySpark
SQL
AWS (S3, EMR)
Step Functions
Lambda
Spark
Terraform

Benefits

Comprehensive health care coverage
On-site health and wellness centers
Retirement savings plan
Backup childcare
Tuition reimbursement
Mental health support
Financial coaching
Commission-based pay
Discretionary incentive compensation

Preferred Qualifications, Capabilities, and Skills

Exposure to cloud technologies such as Step Functions, Lambda, and Spark
Terraform experience is a plus
Experience in the financial services industry

View full job page →

Spark Sql Developer Jobs

Pyspark/Python Data Engineer

Data Engineer

Senior Python Developer with Spark

Lead Software Engineer - Python/PySpark/SQL

Get Spark Sql Developer Jobs alerts