Data Engineer
inDrive
This job is no longer accepting applications
See open jobs at inDrive.See open jobs similar to "Data Engineer" General Catalyst.Data Engineer
Cyprus, Limassol
- Hybrid
- Full-time
- Middle
We are looking for a Data Engineer to the Antifraud team. Antifraud fights many types of fraud. Such as monetary fraud in cashless payment or referral bonuses. And non-monetary fraud, such as bots and autoclickers.
The Antifraud team is a fully product driven and data driven team. Data it’s oxygen of antifraud. We have a lot of analytical rules and pipelines of data processing. We expect you to participate in architecture construction, data modeling, share knowledge and receive this knowledge.We are not afraid to use new infrastructure solutions (Google Cloud Platform, AWS).
We are free to make decisions, being limited only by laws on the protection of personal data. We make infrastructure and architectural decisions exclusively with a sober look at the problem and appealing to data driven.
Our stack:
- Airflow2 for ETL process orchestration
- Python/Scala to develop for Spark
- Go for writing services
- BigQuery as an OLAP DBMS
- MySQL as an OLTP DBMS
- Apache Kafka and Google Pub/Sub as an integration layer between micro-services
Responsibilities
- Develop our antifraud-systems: The Streaming Antifraud and The Batching Antifraud Create a unified system for processing, storing and validating data
- Design and build processes for storing, processing, cleaning and enriching data
- Contribute to our internal data processing framework
- Participate in the development of the data pipeline at all stages, ranging from discussions with data sources regarding the data capture format to the presentation of the new feature to consumers
- Develop APIs (REST, gRPC) for high-load data access services
- Participate in system design and architectural solution development together with the development team
- Write integration and unit tests, develop automation tools for validation and alerting based on data status, and share knowledge with the developers during in-house meet-ups
Qualifications
- Expert level knowledge of Python 3.7+
- Experience developing of data flows on Airflow2
- Experience developing ETL processes on Spark/Flink
- High level of SQL proficiency
- Understand data processing algorithms and principles ️
- Good knowledge of general programming concepts (design patterns, OOP, modularity, pure architecture, 12-Factor App)
- Responsible and proactive (we believe that ideas should come not only from the business, but also from the company’s employees)
Conditions & Benefits
- Stable salary, official employment, health insurance.
- Relocation package offered to candidates from different regions.
- Full coverage of business trip expenses, including daily allowances for employees engaged in travel-related tasks.
- Hybrid work mode and flexile schedule.
- All the necessary work equipment.
- A diverse array of internal training programs.
- Partially or fully funded additional training courses.
- Monthly corporate 'Fun Days' in various engaging formats.
- Meaningful gestures on special occasions - company gifts to celebrate weddings, new arrivals, and birthdays.
This job is no longer accepting applications
See open jobs at inDrive.See open jobs similar to "Data Engineer" General Catalyst.