Pyspark/Python Data Engineer
Python
Amazon Web Services
Apache Airflow
AWS
Aws Glue
Big Data
Cloud
Cloud Operations
Cloud Platform
Cloud Platforms
Data Engineer
Data Integration
Data Pipeline
Data Processing
Data Warehouse
Data Warehousing
Database
DevOps
Engineer
ETL
Git
Pyspark
Snowflake
Software Development
Spark
Spark Sql
SQL
Job Description
Join Tata Consultancy Services as a PySpark and Python Data Engineer to help shape scalable data pipelines that power analytics across the organization. You will design, build, and optimize ETL and ELT processes with PySpark, collaborating with data analysts, architects, and DevOps to ensure top-tier data quality, performance, and reliability. This onsite role is based in Irving, Texas with a salary range of USD 100,000 to 120,000 per year.
Responsibilities
- Design, develop, and maintain ETL/ELT pipelines using PySpark
- Write optimized and scalable PySpark transformations using DataFrames and Spark SQL
- Develop reusable and efficient Python-based data processing components
- Ensure data quality, integrity, and performance across pipelines
- Perform debugging, performance tuning, and optimization of PySpark jobs
- Collaborate with cross-functional teams (Data Analysts, Architects, DevOps)
- Contribute to CI/CD pipelines and deployment workflows for data applications
- Monitor and troubleshoot data workloads in production environments
Requirements
- Strong hands-on experience in PySpark (Spark SQL, DataFrame API)
- Advanced proficiency in Python (data processing, performance tuning, modular coding)
- Solid understanding of ETL design patterns and data pipeline architecture
- Good working knowledge of SQL for data transformation and analysis
- Experience with data processing in distributed environments
- 3–8 years of experience in Data Engineering / PySpark development
- Proven hands-on project experience in PySpark + Python
- Bachelor of Computer Science
Technologies
- PySpark
- Spark SQL
- DataFrame API
- Python
- SQL
- Amazon S3
- AWS Glue
- AWS EMR
- Airflow
- Snowflake
- Git
Experience
- 3–8 years of experience in Data Engineering / PySpark development
- Proven hands-on project experience in PySpark + Python
Salary Range
USD 100,000 - 120,000 per year
Location
Irving, TX (onsite)
Job Function
Technology
Role
Engineer
Job ID
416300
Desired Skills
- Python
Desired Candidate Profile
Qualifications: Bachelor of Computer Science
Similar Jobs
