Data Engineer

Scale Marketing is hiring a Data Engineer (onsite) to expand and optimize its data pipeline architecture, data flows, and extraction/transform/load processes.

Responsibilities

• Create and maintain data pipeline architecture for reliable, scalable data delivery
• Assemble large, complex data sets that align with functional and non-functional business requirements
• Identify, design, and implement internal process improvements (for example: automating manual work, optimizing data delivery, redesigning infrastructure for scalability)
• Build ETL infrastructure to extract, transform, and load data from a wide variety of data sources
• Use SQL and big data technologies to support end-to-end data movement and transformation
• Develop analytics tools that leverage the pipeline to deliver actionable insights for customer acquisition, operational efficiency, and key performance metrics
• Partner with stakeholders across the Partner, Client, and Data teams to support data-related technical issues and infrastructure needs
• Ensure data is separated and secured across national boundaries through multiple data centers
• Create data tools that help analytics and data science teams build and optimize products
• Collaborate with data and analytics experts to improve functionality across data systems

Requirements

• Advanced SQL knowledge, including query authoring and working familiarity with relational databases
• Experience building and optimizing big data pipelines, architectures, and data sets
• Proven ability to perform root cause analysis on internal and external data and processes to answer business questions and identify improvement opportunities
• Strong analytic skills working with unstructured datasets
• Experience implementing processes for data transformation, data structures, metadata, dependency management, and workload management
• Track record of manipulating, processing, and extracting value from large disconnected datasets
• Working knowledge of message queuing, stream processing, and highly scalable big data data stores
• Project management and organizational skills
• Experience supporting and working with cross-functional teams in a dynamic environment
• 5+ years of experience in a Data Engineer role
• Graduate degree in Computer Science, Statistics, Informatics, Information Systems, or another quantitative field

Required tools and technologies

• Big data tools: Apache Hadoop, Apache Hive, Apache Spark, Apache Pig (and similar)
• Relational SQL / NoSQL databases (experience with at least three): MongoDB, HBase, Oracle, SQL Server, Cassandra, Neo4j, Redis, Riak
• Pipeline/workflow management tools: Apache Kafka and/or Apache Spark, Azure DevOps, AWS Data Pipeline
• Cloud: AWS and/or Microsoft Azure
• Stream-processing systems: AWS Kinesis, Azure Stream Analytics, and/or Apache Flink (or similar)
• Scripting languages (experience with at least three): Python, Java, Scala, C++, Linux/Unix (and similar)

Location: Chicago, IL (onsite)