Data Engineer
Datumo specializes in providing Big Data and Cloud consulting services to clients from all over the world, primarily in Western Europe, Poland and the USA.
Core industries we support include e-commerce, telecommunications and life science. Our team consists of exceptional people whose commitment allows us to conduct highly demanding projects .
Our team members tend to stick around for more than 3 years, and when a project wraps up, we don't let them go - we embark on a journey to discover exciting new challenges for them.
It's not just a workplace; it's a community that grows together!
What we expect : Must-have :
Must-have :
- 2 years of commercial experience in Big Data
- proven record with a selected cloud provider (GCP, Azure or AWS)
- good knowledge of Python or JVM (Scala preferred, can be Java)
- strong understanding of Spark or similar distributed data processing framework
- experience with BigQuery, Snowflake, Hive or similar distributed datastore
- designing and implementing Big Data systems following best practices
- ensuring solution quality through automatic tests, CI / CD and code review
- proven collaboration with businesses
- English proficiency at B2 level, communicative in Polish
Nice to have :
- experience in Snowflake / Databricks platform
- familiarity with Airflow or similar pipeline orchestrator
- knowledge of Apache Kafka, Docker and Kubernetes technologies
- experience in Machine Learning projects
- willingness to share knowledge (conferences, articles, open-source projects)
What’s on offer :
- 100% remote work, with workation opportunity
- 20 free days
- onboarding with a dedicated mentor
- project switching possible after a certain period
- individual budget for training and conferences
- benefits : Medicover private medical care, co-financing of the Medicover Sport card
- opportunity to learn English with a native speaker
- regular company trips and informal get-togethers
Development opportunities in Datumo :
- participation in industry conferences
- establishing Datumo's online brand presence
- support in obtaining certifications (e.g. GCP, Azure, Snowflake)
- involvement in internal initiatives, like building technological roadmaps
- training budget
- access to internal technological training repositories
Discover our exemplary projects :
IoT data ingestion to cloud
The project integrates data from edge devices into the cloud using Azure services. The platform supports data streaming via either the IoT Edge environment with Java or Python modules, or direct connection using Kafka protocol to Event Hubs.
It also facilitates batch data transmission to ADLS. Data transformation from raw telemetry to structured tables is done through Spark jobs in Databricks or data connections and update policies in Azure Data Explorer.
Petabyte-scale data platform migration to Google Cloud
The goal of the project is to improve scalability and performance of the data platform by transitioning over a thousand active pipelines to GCP.
The main focus is on rearchitecting existing Spark applications to either Cloud Dataproc or Cloud BigQuery SQL, depending on the Client’s requirements and automate it using Cloud Composer.
Data analytics platform for investing company
The project centers on developing and overseeing a data platform for an asset management company focused on ESG investing.
Databricks is the central component. The platform, built on Azure cloud, integrates various Azure services for diverse functionalities.
The primary task involves implementing and extending complex ETL processes that enrich investment data, using Spark jobs in Scala.
Integrations with external data providers, as well as solutions for improving data quality and optimizing cloud resources, have been implemented.
Realtime Consumer Data Platform
The initiative involves constructing a consumer data platform (CDP) for a major Polish retail company. Datumo actively participates from the project’s start, contributing to planning the platform’s architecture.
The CDP is built on Google Cloud Platform (GCP), utilizing services like Pub / Sub, Dataflow and BigQuery. Open-source tools, including a Kubernetes cluster with Apache Kafka, Apache Airflow and Apache Flink, are used to meet specific requirements.
This combination offers significant possibilities for the platform.
Recruitment process :
- Quiz - 15 minutes
- Soft skills interview - 30 minutes
- Technical interview - 60 minutes
Find out more by visiting our website - https : / / www.datumo.io
If you like what we do and you dream about creating this world with us - don’t wait, apply now!
Data Engineer
Job title. Data Engineer Department. Engineering Reporting to. Engineering Manager Location. Poland Purpose of Role We're looking for a Data Engineer to join our Market Performance Group in transforming What you will be doing Own and improve our big data pipelines in AWS Spark. Build scalable and resilient data infrastructure, as well as tools to extract and transform data used Work closely with our data scientists and our data engineers to create and evolve products that measure
Data Engineer
Dla naszego klienta. globalnej organizacji. poszukujemy osoby na stanowisko Data Engineer. Wymagania. 5 lat doświadczenia jako Data Engineer z wiedzą na temat Pythona, Scali lub Javy. AWS Redshift, Google BigQuery, Azure Data Lake). Udokumentowane projekty implementacji systemów big data, w tym doświadczenie z real time data processing
Big Data Engineer
Develop and implement data pipelines for ingesting, processing, and analyzing large volumes of data. Implement data governance and security best practices to ensure data integrity and compliance. Proven experience as a Big Data Engineer or similar role, with a strong portfolio showcasing your contributions to big data projects. Familiarity with data warehousing concepts, ETL processes, and data modeling.
Data Engineer
To support the design, build and roll out of the Data Lake Data Warehouse we are recruiting Data Engineers The Data Engineer will develop content in the Data Lake Data Warehouse for reporting USE Cases Work with Data Governance to map and classify the physical data sets in a Data Governance Tool You are meant for this job if. Prior experience as a Data Engineer with 2 4 years' experience Educated to degree level in IT or Computing Science or proven experience as a Data Engineer.
Data Engineer
znajomości zagadnień z zakresu Big Data lub hurtowni danych Doświadczenia w obszarze Big Data problemów MILE WIDZIANE Znajomość technologii Spark oraz Airflow Doświadczenie w projektach Data
Data Engineer
It also facilitates batch data transmission to ADLS. or data connections and update policies in Azure Data Explorer. Data analytics platform for investing company The project centers on developing and overseeing a data Integrations with external data providers, as well as solutions for improving data quality and optimizing Realtime Consumer Data Platform The initiative involves constructing a consumer data platform
Data Engineer
Responsibilities. Design and develop data structures, pipelines and ETL processes Execute Data Governance practices, keeping our data secure, transparent and in good quality. Optimize data infrastructure for performance, reliability, and scalability. with a strong understanding of data warehousing, ETL processes, and data modeling. Experience with Big Data technologies like Hadoop, Spark, and distributed data storage systems.
DATA Engineer
PAYBACK is a top player in data driven marketing. digital business models and implement technical trends, we are one of the leading companies in the data Your responsibilities. Within the Data Engineering Team you can work with a huge amount of data and help Your Profile. You have 4. years of work experience as a Data Engineer with huge datasets or in a similar Mass data doesn't scare you.
Junior Data Engineer
In this position you will join our Data Team , tasked with development and operations of company wide data warehousing solution. data artifacts to downstream systems. As a Junior Data Engineer you will be responsible for various elements of our product's lifecycle. from developing new features and optimising existing data pipelines to debugging and improving production
Data Engineer
You will be. developing, constructing, testing and maintaining architectures (such as data warehouses , ensuring that the developed architecture supports our clients' business requirements, developing data set processes for data modeling, mining, and production, employing a variety of languages and tools efficiency and quality. Your profile. 3. years of professional experience as Data Engineer, able understanding and data importing, data cleaning and manipulation, proficiency in Python, experienced