Data Engineer (Contract to Perm)
Big Data
Bigdata
Cloud Data Engineering
Cloud Data Platform
Cloud Platform
Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Integration
Data Integration Tools
Data Lakehouse
Data Pipeline
Data Platform
Data Processing
Data Warehouse
Database
Databases
Databricks
ETL
Informatica
Integration
Reporting and Analytics
Software Development
Spark
SQL
Job Description
Locality is seeking a Senior Data Engineer for a contract to perm opportunity in New York, NY (onsite). In this role, you will be the primary internal owner of the Audience Engine, taking charge of ingestion, identity resolution, automation, and the data foundation needed for applied AI enablement.
You will work hands-on in Databricks, owning pipelines across Bronze/Silver/Gold layers and delivering audience publication with partner integrations, governance, and reliability in mind.
Responsibilities
- Establish and own the technical management of the Audience Engine, transitioning knowledge from external partners while building a scalable, internally operated platform
- Decrease reliance on third-party vendors while maintaining platform reliability, uptime, and data integrity across pipelines and integrations
- Architect, own, and continuously optimize Databricks Bronze/Silver/Gold pipelines for large-scale audience ingestion, enrichment, and activation workflows
- Improve scalability and cost efficiency through pipeline optimization and effective resource management
- Monitor, troubleshoot, and resolve ingestion failures, schema drift, and data quality issues across batch and near real time pipelines
- Lead identity resolution across datasets including HHID, DSID, MAID, IP, and campaign logs, strengthening matching logic for the Household Identity Graph
- Ensure accurate and timely audience publication by maintaining downstream delivery processes and meeting SLAs across activation platforms and partners
- Own and maintain integrations with external data and activation partners (including Experian, FreeWheel, and DSPs) to ensure seamless data exchange and operational reliability
- Collaborate cross functionally with Product, Analytics, and Ad Operations to onboard new data sources and activation endpoints for new use cases and revenue opportunities
- Support and operationalize applied AI/ML workflows in Databricks, including integration with Akkio, and convert business needs into production-ready data features and datasets
- Design and implement scalable data models and query-ready datasets that support BI reporting, audience insights, and predictive modeling, aligned with downstream analytics requirements
- Enforce data governance, lineage, and documentation standards, ensuring compliance with privacy, consent, and data usage regulations across audience and identity systems
Requirements
- 4+ years of experience as a Data Engineer or Data Platform Engineer, ideally in advertising
- Strong hands-on experience with Databricks, Spark, and modern cloud data platforms
- Proven experience designing and maintaining multi-layer data architectures (Bronze / Silver / Gold)
- Deep expertise in SQL and proficiency in Python or another scripting language
- Experience with identity graphs, audience data, and complex join logic across large datasets
- Hands-on experience building automated data pipelines and third-party data integrations
- Familiarity with adtech, martech, or audience activation platforms, including exposure to identity providers such as Experian or similar vendors
- Practical experience supporting AI/ML workflows or AI-enabled analytics, including data quality, schema design, and performance optimization in cloud-scale environments
Technologies
- Databricks
- Spark
- SQL
- Python
- Akkio
- BI
Benefits
- Competitive base salary in the range of $60-$80/hr (dependent on years of experience, level of education obtained, and applicable skillset)
- Excellent benefits package including health care benefits, retirement benefits, and paid days off
Compensation: USD $60 - $80 per hour. Job type: contract (contract to perm). Experience: 4+ years.
Similar Jobs
L