Note: The job is a remote job and is open to candidates in USA. Pennant Solutions is seeking a skilled and experienced Senior Kafka/Kong Administrator to join their IT operations team. The role involves deploying, configuring, maintaining, and monitoring Kafka clusters and Kong gateways to ensure high availability and optimal performance for real-time platforms.
Responsibilities
- Deploy and configure Kafka clusters following best practices for scalability, security, and performance
- Collaborate with cross-functional teams to gather requirements and design Kafka infrastructure to meet data streaming needs
- Manage topics, partitions, replication factors, and broker configurations to ensure efficient data distribution and fault tolerance
- Implement monitoring and alerting solutions to proactively identify and address performance bottlenecks, resource constraints, and anomalies
- Conduct regular performance testing, load testing, and capacity planning to ensure clusters can handle anticipated workloads
- Design and implement high-availability strategies, including failover mechanisms and data replication across multiple data centers or cloud regions
- Develop and maintain disaster recovery plans and procedures to minimize data loss and downtime in the event of failures
- Implement and manage security measures such as encryption, authentication, and authorization to ensure data privacy and compliance with industry standards
- Stay updated on security vulnerabilities and patches, applying necessary updates to maintain a secure Kafka environment
- Diagnose and resolve Kafka-related issues, including performance degradation, data loss, and connectivity problems
- Collaborate with development and data engineering teams to troubleshoot consumer/producer application integration with Kafka
- Maintain thorough documentation of Kafka configurations, deployment processes, and troubleshooting procedures
- Provide training and knowledge sharing sessions to junior team members and other stakeholders
- Collaborate with cross-functional teams, including data engineers, developers, and system administrators, to ensure smooth integration of Kafka into our data ecosystem
- Communicate effectively with stakeholders to provide updates on Kafka performance, maintenance, and improvements
- Hands on experience installing and configuring Kong API Gateway as well as integration with existing API frameworks
- Describe and configure dataplane security using RBAC, Roles, Workspaces and Teams
- Configure upstreams and load balancing
- Create Services, Routes, and Consumers
- Working knowledge of Kong Ingress with Kubernetes/Dockers
- Excellent knowledge of API design standards, patterns, and best practices especially with OpenAPI 2.0, REST, JSON, XML
- Distinguish between downstream and upstream networking issues
- Collecting troubleshooting/debug information
- Using log files and health and traffic metrics
- Deep understanding of JWTs and their usage
- Use Kong Vitals to monitor Kong Gateway’s health and performance
- Pull metrics using Vitals API
Skills
- Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent work experience)
- 5+ years of hands-on experience in administering Apache Kafka clusters in production environments
- 5+ years proficiency in Kafka/Kong architecture, installation, configuration, and tuning
- Strong understanding of Kafka internals, including topics, partitions, replication, and consumer/producer APIs
- 5+ years of experience with Kafka ecosystem tools such as Kafka Connect, Kafka Streams, and Schema Registry
- Knowledge of Linux systems and shell scripting
- Strong DevOps background
- 5+ years of familiarity with cloud platforms (e.g., AWS, Azure, GCP) and containerization (e.g., Docker, Kubernetes)
- Excellent problem-solving skills and the ability to troubleshoot complex issues efficiently
- Strong communication skills and the ability to work collaboratively in a team environment
- Certification as a Confluent Certified Administrator for Apache Kafka (CCAAK)
- Preferred certification in Kong Gateway Certified Associate (KGCA)
Company Overview