DataOps Engineer

Описание

The IT company Andersen invites an experienced DataOps Engineer to work on a project for one of our customers in Georgia.

The customer is the first fully digital neobank in Georgia, existing solely on mobile phones. Since 2015, the company has been providing convenient and secure financial services, including quick loans, money transfers, utility payments, and card management, all without bureaucracy or hidden fees.

The project is modernizing data infrastructure by migrating legacy pipelines to a new platform, implementing an on-premises LakeHouse architecture, applying PII anonymization/tokenization, and developing analytics applications for users and automated systems.

Responsibilities:
- Migrating legacy data pipelines to a modern LakeHouse-based data platform.
- Designing and maintaining an on-premise LakeHouse architecture using tools like Spark, Delta Lake, and Trino.
- Implementing anonymization and tokenization mechanisms to protect personally identifiable information (PII).
- Building and managing CI/CD pipelines to automate deployment of data workflows and infrastructure components.
- Developing and maintaining infrastructure using Infrastructure-as-Code tools such as Terraform and Ansible.
- Setting up monitoring and logging systems to ensure the reliability and observability of data pipelines.
- Supporting the development and operation of analytics applications for both users and machine clients.

Must-haves:
- Bachelor's or Master's degree in Computer Science, Engineering, Information.
- Proven experience in a DataOps, Data Engineering, or DevOps role with a focus on data systems for 4+ years.
- Hands-on experience with Apache Spark for batch and stream processing.
- Proficiency in deploying and managing applications on Kubernetes (e.g., using Helm charts, operators).
- Experience with object storage solutions, specifically Dell ECS or similar S3- compatible storage.
- Solid understanding and practical experience with Trino (formerly PrestoSQL) for distributed SQL querying.
- Experience with Delta Lake architecture, including table management, optimization, and its ecosystem.
- Strong programming skills in languages such as Python, Scala, or Java.
- Experience with CI/CD tools (e.g., Jenkins, GitLab CI, Argo CD) and version control systems (e.g., Git).
- Familiarity with Infrastructure-as-Code tools (e.g., Terraform, Ansible, CloudFormation).
- Experience with data pipeline orchestration tools (e.g., Apache Airflow, Kubeflow Pipelines, Prefect).
- Knowledge of monitoring and logging tools (e.g., Prometheus, Grafana, ELK stack).
- Understanding of Data Warehousing, Data Lakes, and Data LakeHouse concepts.
- Strong analytical and problem-solving skills with a proactive and collaborative mindset.
- Excellent communication and interpersonal skills.
- Level of English – from Intermediate+.

Nice-to-haves:
- Experience with other big data technologies (e.g., Kafka, Flink).
- Knowledge of other data storage and database systems (SQL and NoSQL).
- Understanding of data security principles and experience implementing them in data platforms.
- Certifications in relevant technologies (e.g., Kubernetes, Spark, Cloud platforms).
- Experience working in an Agile development environment.

Reasons why this job would be interesting to you:
- Experience in teamwork with leaders in FinTech, Healthcare, Retail, Telecom, and others. Andersen cooperates with such businesses as Samsung, Siemens, Johnson & Johnson, BNP Paribas, Ryanair, Mercedes, TUI, Verivox, Allianz, T-Systems, etc.;
- The opportunity to change the project and/or develop expertise in an interesting business domain;
- Job conditions – you can work both fully remotely and from the office or can choose a hybrid variant;
- Guarantee of professional, financial, and career growth! The company has introduced systems of mentoring and adaptation for each new employee;
- The opportunity to earn up to an additional 1,000 EUR per month, depending on the level of expertise, which will be included in the annual bonus, by participating in the company's activities;
- Access to the corporate training portal, where the entire knowledge base of the company is collected and which is constantly updated;
- Bright corporate life (parties / pizza days / PlayStation / fruits / coffee / snacks / movies);
- Certification compensation (AWS, PMP, etc);
- Referral program;
- English courses;
- Private health insurance and compensation for sports activities.

Your personal data is protected in accordance with GDPR regulations.

Join us!