Aws Data Lakehouse

Aws Data Lakehouse Developer Jobs

2 jobs found

Today

Data Engineer

VISUAL SOFT, INC Huntsville, AL

Huntsville, AL

Today

Full time 3d Design Tools Amazon Web Services AWS

Apply Now

Job Description

VISUAL SOFT, INC is seeking a Data Engineer to support secure, mission-critical data environments within a classified cloud infrastructure. This role is fully onsite in Huntsville, Alabama and requires an active Top Secret clearance as well as U.S. citizenship.

In this position (Mid-Level or Senior), you will build and maintain ETL and data integration capabilities across on-premises and AWS environments, including real-time and batch processing, automation, and data governance activities.

What you will do

Design, develop, and maintain ETL workflows to extract, transform, and load large-scale structured and unstructured datasets.
Build data migration solutions between legacy and modern systems using SQL, Java, and cloud-native tools.
Create data integration frameworks using AWS services including Glue, Lambda, S3, RDS, Redshift, and Kinesis.
Develop automation scripts with Python, Shell, or Bash for deployment, data validation, and maintenance.
Maintain and enhance data pipelines for real-time and batch data processing.
Support data quality, metadata management, and data governance efforts.
Participate in Agile/Scrum sprints, contributing to design, code reviews, testing, and documentation.
Troubleshoot and resolve data-related issues across on-premises and AWS environments.

Required education

Bachelor’s Degree (B.S.) in Computer Science, Information Systems, Engineering, or a related field.

Required experience

5+ years (Mid-Level) or 8+ years (Senior-Level) in data engineering or database development.

Skills and qualifications

Strong hands-on experience with ETL tools, such as Informatica, Talend, Pentaho, AWS Glue, or custom Java ETL frameworks.
Proficiency in SQL and at least one major RDBMS, including Oracle or PostgreSQL.
Experience with data migration projects and data quality validation.
Proficiency in Java or Python for data processing and automation.
Cloud experience, preferably AWS (including RDS, S3, Lambda, Redshift, Glue).
Working knowledge of Linux/Unix environments and shell scripting.
Experience working in an Agile/Scrum development environment.
Excellent problem-solving, analytical, and communication skills.

Technologies

ETL, AWS Glue, AWS Lambda, Amazon S3, Amazon RDS, Amazon Redshift, Amazon Kinesis
SQL, Java, Python, Shell, Bash
Informatica, Talend, Pentaho
Oracle, PostgreSQL, MongoDB
Linux/Unix, Agile/Scrum, and AWS cloud services

Location and clearance

Huntsville, Alabama (100% onsite)
Clearance required: Active Top Secret Clearance (U.S. Citizenship Required)

Employment type

Full-Time / Permanent
1099 Consultant

Benefits

3 weeks of Paid time off (PTO) that includes sick leave
2 floating and 8 public holidays
Company covers 50% of health and dental insurance for all full-time employees (dependents at extra cost)
STD, LTD, and one-time salary equivalent of life insurance at no cost to all full-time employees
401k program after 90 days with a company match of 4% and immediate vesting
Profit sharing for w-2 employees with no benefits and no employee contributions required
Floating and holidays expire at the end of every year of service

View full job page →

1d ago

Lead Data Engineer

Capital Technology Group Washington, DC

Washington, DC

1d ago

$150k - $200k/yr Full time AWS Aws Bedrock Aws Data Lakehouse

Apply Now

Job Description

Capital Technology Group is looking for a Lead Data Engineer to design, build, and operate scalable data pipelines and platforms in a Washington, DC hybrid environment. In this role, you will partner with other Data Engineers to evaluate and prototype new data tools and technologies while supporting mission-critical analytics in AWS-based federal environments governed by FedRAMP and NIST SP 800-53 security controls.

You will help shape AWS-native data systems, including ingestion, transformation, and orchestration workflows, and contribute to the reliability, performance, and maintainability of enterprise data platforms. You will also provide technical guidance through architecture discussions and code reviews in an Agile development setting.

What You’ll Do

Design, build, and maintain scalable data pipelines, ETL/ELT workflows, and data models using Python, Apache Spark (PySpark), SQL (PostgreSQL), and AWS Glue.
Develop and optimize AWS-native data platforms using AWS Glue, Amazon EMR, Amazon MWAA (Apache Airflow), Amazon S3, RDS, and CloudWatch.
Build high-performance ingestion, transformation, and orchestration workflows for structured and semi-structured datasets using Apache Iceberg and Parquet.
Design and optimize analytical data platforms using Amazon Athena, Trino, Hive, OpenSearch, and enterprise data catalog technologies.
Build AI-enabled data solutions using Amazon Bedrock, RAG pipelines, and vector search technologies such as Amazon S3 Vectors and OpenSearch vector indexes.
Develop cloud infrastructure using CloudFormation (Infrastructure as Code) and GitHub with enterprise CI/CD pipelines.
Improve reliability, scalability, performance, and maintainability through monitoring, troubleshooting, automation, and continuous optimization.
Support mission-critical analytics and reporting solutions in large-scale AWS-based federal environments while implementing controls aligned to FedRAMP and NIST SP 800-53.
Mentor junior engineers via technical guidance, architecture discussions, and code reviews while promoting engineering best practices.
Collaborate cross-functionally in an Agile environment to deliver high-quality data solutions and communicate technical concepts to technical and non-technical stakeholders.

Required Qualifications

Bachelor’s degree in Computer Science, Engineering, or a related technical field.
15+ years of professional experience in data engineering, data architecture, or a related field.
Strong hands-on experience with Apache Spark (PySpark) (required), Python, SQL (PostgreSQL), and dbt for large-scale data engineering, ETL/ELT, data transformation, and data modeling.
Experience with AWS Glue, Amazon EMR, Amazon MWAA (Apache Airflow), AWS Lambda, Amazon S3, and Amazon RDS.
Experience developing scalable data pipelines, workflow orchestration, and data integration solutions across enterprise environments.
Experience with modern data lake technologies and formats such as Parquet and Iceberg.
Experience designing and optimizing solutions using relational and NoSQL databases.
Ability to build reliable, high-performance data platforms using performance tuning and enterprise-scale ETL/ELT architecture.
Strong analytical and problem-solving skills.
Experience working in Agile, iterative software development environments.
Ability to quickly learn and apply new technologies and domain knowledge.
Excellent written and verbal communication skills, including the ability to explain complex topics to diverse audiences.

Technologies You’ll Work With

Python, Apache Spark (PySpark), SQL (PostgreSQL)
AWS Glue, Amazon EMR, Amazon MWAA (Apache Airflow), Amazon S3, RDS, CloudWatch
Apache Iceberg, Parquet, Amazon Athena, Trino, Hive, OpenSearch, enterprise data catalog technologies
Amazon Bedrock, RAG pipelines, vector search technologies (Amazon S3 Vectors, OpenSearch vector indexes)
CloudFormation (Infrastructure as Code), GitHub, CI/CD pipelines
FedRAMP, NIST SP 800-53, AWS Lambda, dbt, NoSQL databases

Eligibility / Client Requirements

Applicants must be US Citizens and be able to obtain Public Trust clearance.

Salary and Benefits

Salary: USD 150,000 - 200,000 per year (final offer may vary based on experience, skills, and other factors; stated range is not a guarantee and is subject to change).

Medical, Dental, and Vision
Life Insurance, Short/Long Term Disability
Employee Assistance Program
401(k) with 4% matching
Liberal PTO vacation policy
Generous Annual Continuing Education
Annual Wellness Budget
Bonus Incentive Programs (employee referrals and performance-based rewards)
Remote work (hybrid roles will be specified in the job post)

Nice to Have

Experience supporting analytics, data engineering, or modernization initiatives for financial regulators, capital markets, or other highly regulated environments.
Experience with Apache Iceberg and modern data lakehouse architectures.
Experience with unstructured data processing, including document/text processing, embeddings, vector search, and LLM-based data solutions.
Exposure to integrating LLMs and generative AI capabilities into enterprise data pipelines and platforms.
Experience designing data architectures that support both structured and unstructured data at scale.

View full job page →

Aws Data Lakehouse Developer Jobs

Data Engineer

Lead Data Engineer

Get Aws Data Lakehouse Developer Jobs alerts