Wyandotte Technologies is seeking a Senior Splunk Engineer and subject matter expert to support mission-critical systems for the Southwestern Power Administration (SWPA), a U.S. Department of Energy customer. The position emphasizes hands-on Splunk enterprise architecture and administration, advanced SPL development, secure operations, and complex troubleshooting in highly restricted environments, including real-time SCADA operations.
Role Overview
In this onsite role in Springfield, MO, you will serve as a senior technical owner for enterprise and operational Splunk capabilities. Responsibilities include designing and optimizing distributed Splunk environments, engineering operational intelligence through advanced SPL, and building reliable automation and observability workflows that improve detection, diagnosis, and resolution of production issues.
Key Responsibilities
- Act as the senior Splunk SME and technical owner for enterprise and operational Splunk capabilities.
- Design, administer, optimize, and troubleshoot distributed Splunk environments, including search heads, indexers, forwarders, deployment components, and clustered architectures as applicable.
- Develop and optimize advanced SPL for troubleshooting, analytics, correlation, alerting, and search performance at scale.
- Own data onboarding and normalization, including indexes, sourcetypes, props/transforms, field extractions, forwarder configuration, retention, access controls, and platform health.
- Engineer dashboards, reports, and actionable alerts to reduce time to detect, diagnose, and resolve operational issues.
- Build production-grade automation using PowerShell, Python, Bash, REST APIs, and other suitable technologies to support Splunk and enterprise operations.
- Automate deployments, configuration, validation, monitoring, reporting, remediation, and recurring support activities.
- Integrate Splunk with applications, ServiceNow, automation workflows, infrastructure, security tooling, and other enterprise systems.
- Analyze application, infrastructure, security, and operational telemetry to identify trends, anomalies, failures, performance issues, and root causes.
- Establish reusable engineering standards, documentation, configuration practices, and support procedures.
- Mentor engineers and administrators and provide technical leadership for complex Splunk and observability issues.
- Identify manual, repetitive, and error-prone processes and engineer reliable automated solutions.
- Design, implement, and improve CI/CD pipelines and deployment automation, including build, test, release, deployment, validation, and rollback processes.
- Develop, enhance, integrate, and support custom enterprise applications, and modernize legacy applications and operational processes into secure, supportable architectures.
- Troubleshoot complex production issues and perform detailed root-cause analysis.
- Participate in Agile/Scrum activities, including sprint contributions and creation of clear technical documentation.
- Support systems operating in secure, highly restricted environments, including real-time operational and SCADA-related systems.
- Participate in a 24/7 on-call rotation and support occasional evening/weekend maintenance or deployment activities.
- Work closely with SWPA stakeholders, technical teams, and leadership to translate operational needs into reliable technical solutions.
Required Qualifications
- Bachelor’s degree in Computer Science, Software Engineering, Information Technology, Cybersecurity, or a related technical field plus 7+ years of relevant professional experience; OR 11+ years of relevant Splunk, infrastructure, systems engineering, automation, DevOps, or implementation experience.
- 7+ years of enterprise infrastructure, security, systems, observability, automation, or related engineering experience.
- 5+ years of hands-on Splunk engineering or administration in enterprise or production environments, with demonstrated senior-level ownership.
- Advanced SPL expertise, including complex search development, troubleshooting, correlation, and search optimization.
- Hands-on experience administering core Splunk components such as indexers, search heads, universal/heavy forwarders, deployment server, and distributed or clustered environments.
- Demonstrated data onboarding and normalization experience, including indexes, sourcetypes, props/transforms, field extractions, forwarders, and retention.
- Strong Linux and/or Windows systems troubleshooting ability, including diagnosing platform, application, network, and data-ingestion issues.
- Strong scripting and automation experience using PowerShell, Python, Bash, REST APIs, or comparable technologies.
- Experience with Git and CI/CD or configuration/deployment automation.
- Strong communication, client relationship, root-cause analysis, documentation, mentoring, and independent technical ownership skills.
- U.S. Citizenship required due to federal customer/system-access requirements.
- Must successfully complete a 7-year background investigation.
- 100% onsite in Springfield, Missouri. Candidates must be able and willing to work onsite; this is not remote or hybrid.
- Must participate in an on-call rotation supporting mission-critical systems.
- Must be available for occasional evening or weekend maintenance and deployment activities.
Preferred Qualifications
- Splunk Enterprise Certified Admin, Splunk Enterprise Certified Architect, or comparable advanced Splunk certification.
- Splunk Enterprise Security (ES), security monitoring/SIEM, or advanced operational observability experience.
- Experience with large-scale distributed Splunk architecture, clustering, capacity and performance optimization, upgrade/migration, and platform lifecycle.
- Experience with automated remediation, event-driven automation, REST APIs, ServiceNow integrations, and enterprise workflow automation.
- Experience supporting Linux and Windows in secure, restricted, federal, or other highly controlled computing environments.
- SCADA, industrial control systems, OT/IT integration, utilities, energy, real-time operational systems, or other critical-infrastructure experience.
- Technical leadership, architecture ownership, or mentoring of Splunk engineers and administrators.
Technologies
- Splunk, SPL
- PowerShell, Python, Bash
- REST APIs
- ServiceNow
- CI/CD, Git
- Linux, Windows
Compensation
USD 110,000 - 115,000 per year.
Benefits
- 401(k)
- 401(k) matching
- Dental insurance
- Employee assistance program
- Flexible spending account
- Health insurance
- Life insurance
- Paid time off
- Vision insurance
Important Applicant Screening
This position requires 5+ years of hands-on Splunk engineering or administration, U.S. Citizenship, and 100% onsite work in Springfield, Missouri. Please apply only if you meet all three requirements.
Employment Requirements
- U.S. Citizenship is required due to federal customer/system-access requirements.
- Must successfully complete a 7-year background investigation.
- Must be 100% onsite in Springfield, Missouri; remote or hybrid is not available.
- Must participate in an on-call rotation supporting mission-critical systems.
- Must be available for occasional evening or weekend maintenance and deployment activities.