Senior Data Engineer
Job description
Senior Data Engineer at Mex Digital.
About the role
You will design, construct, install, test, and maintain highly scalable data pipelines with a focus on machine learning models and analytics. You will work closely with data scientists, ML engineers, and stakeholders to ensure data is accessible, consistent, and reliable for ongoing projects. You will develop and maintain APIs for data access and manipulation, integrating with external data services as required. You will manage and optimize data storage solutions for both structured and unstructured data, covering text, image, audio, and video to support machine learning model requirements. You will implement data quality monitoring processes to ensure production data is always accurate and available. You will collaborate with ML engineers on data-related technical issues and provide architectural guidance. You will support metadata management and data governance using cataloging tools such as OpenMetadata. You will ensure compliance with data security and privacy policies across all data systems.
Key facts
What you'll do
- Design, construct, install, test, and maintain highly scalable data pipelines with a focus on machine learning models and analytics
- Work closely with data scientists, ML engineers, and stakeholders to ensure data is accessible, consistent, and reliable for ongoing projects
- Develop and maintain APIs for data access and manipulation, and integrate with external data services as needed
- Manage and optimize data storage solutions for both structured and unstructured data, including relational databases, NoSQL databases, search engines such as Elasticsearch, and object storage, covering text, image, audio, and video data to support machine learning model requirements
- Implement data quality monitoring processes to ensure production data is always accurate and available to key stakeholders
- Collaborate with ML engineers on data-related technical issues and provide architectural guidance and solutions
- Support metadata management and data governance using cataloging tools such as OpenMetadata
- Ensure compliance with data security and privacy policies across all data systems
- Maintain clear and up-to-date documentation including data dictionaries, metadata, and architectural diagrams
- Contribute to real-time and streaming data processing requirements where applicable
Requirements
- Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field, or equivalent professional experience
- 8 or more years of experience in a data engineering role
- Proficiency in Python, Java, and Scala, with experience managing large-scale data at terabyte to petabyte scale
- Hands-on experience with big data technologies including Hadoop, Spark, and Flink
- Familiarity with machine learning frameworks such as TensorFlow, PyTorch, or similar
- Strong understanding of data warehousing, ETL processes, and data modeling concepts
- Experience with cloud platforms, with AWS as the primary platform including S3, Redshift, Glue, and Lambda; Azure, GCP, and VMware experience is beneficial
- Experience with API development using RESTful or GraphQL approaches
- Knowledge of DevOps practices, CI/CD, and containerization using Docker or Kubernetes
- Familiarity with BI tools such as Metabase and data catalog tooling such as OpenMetadata
- Experience with streaming platforms such as Kafka or RabbitMQ
- Fintech sector experience is an advantage
Nice to have
None specified.
Practical notes
MultiBank Group is a global financial pioneer established in 2005 in California and now proudly headquartered in Dubai, UAE. We specialize in delivering cutting-edge trading technology, unparalleled liquidity, and exceptional customer service. Our extensive range of financial products includes Forex, Metals, Shares, Indices, Commodities, and Cryptocurrency CFDs. Join our thriving community of over 2 million clients across 100 countries, contributing to a daily trading volume exceeding US$ 35 billion. As a heavily regulated institution with oversight from 18+ financial regulators across 5 continents, and recipient of over 80 financial awards, MultiBank Group is devoted to innovation, excellence, and empowering our clients to achieve their financial goals.
Senior Data Engineer at Mex Digital.
About the role
You will design, construct, install, test, and maintain highly scalable data pipelines with a focus on machine learning models and analytics. You will work closely with data scientists, ML engineers, and stakeholders to ensure data is accessible, consistent, and reliable for ongoing projects. You will develop and maintain APIs for data access and manipulation, integrating with external data services as required. You will manage and optimize data storage solutions for both structured and unstructured data, covering text, image, audio, and video to support machine learning model requirements. You will implement data quality monitoring processes to ensure production data is always accurate and available. You will collaborate with ML engineers on data-related technical issues and provide architectural guidance. You will support metadata management and data governance using cataloging tools such as OpenMetadata. You will ensure compliance with data security and privacy policies across all data systems. You will maintain clear and up-to-date documentation including data dictionaries, metadata, and architectural diagrams. You will contribute to real-time and streaming data processing requirements where applicable.
Key facts
What you'll do
- Design, construct, install, test, and maintain highly scalable data pipelines with a focus on machine learning models and analytics
- Work closely with data scientists, ML engineers, and stakeholders to ensure data is accessible, consistent, and reliable for ongoing projects
- Develop and maintain APIs for data access and manipulation, and integrate with external data services as needed
- Manage and optimize data storage solutions for both structured and unstructured data, including relational databases, NoSQL databases, search engines such as Elasticsearch, and object storage, covering text, image, audio, and video data to support machine learning model requirements
- Implement data quality monitoring processes to ensure production data is always accurate and available to key stakeholders
- Collaborate with ML engineers on data-related technical issues and provide architectural guidance and solutions
- Support metadata management and data governance using cataloging tools such as OpenMetadata
- Ensure compliance with data security and privacy policies across all data systems
- Maintain clear and up-to-date documentation including data dictionaries, metadata, and architectural diagrams
- Contribute to real-time and streaming data processing requirements where applicable
Requirements
- Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field, or equivalent professional experience
- 8 or more years of experience in a data engineering role
- Proficiency in Python, Java, and Scala, with experience managing large-scale data at terabyte to petabyte scale
- Hands-on experience with big data technologies including Hadoop, Spark, and Flink
- Familiarity with machine learning frameworks such as TensorFlow, PyTorch, or similar
- Strong understanding of data warehousing, ETL processes, and data modeling concepts
- Experience with cloud platforms, with AWS as the primary platform including S3, Redshift, Glue, and Lambda; Azure, GCP, and VMware experience is beneficial
- Experience with API development using RESTful or GraphQL approaches
- Knowledge of DevOps practices, CI/CD, and containerization using Docker or Kubernetes
- Familiarity with BI tools such as Metabase and data catalog tooling such as OpenMetadata
- Experience with streaming platforms such as Kafka or RabbitMQ
- Fintech sector experience is an advantage
Nice to have
None specified.
Practical notes
MultiBank Group is a global financial pioneer established in 2005 in California and now proudly headquartered in Dubai, UAE. We specialize in delivering cutting-edge trading technology, unparalleled liquidity, and exceptional customer service. Our extensive range of financial products includes Forex, Metals, Shares, Indices, Commodities, and Cryptocurrency CFDs. Join our thriving community of over 2 million clients across 100 countries, contributing to a daily trading volume exceeding US$ 35 billion. As a heavily regulated institution with oversight from 18+ financial regulators across 5 continents, and recipient of over 80 financial awards, MultiBank Group is devoted to innovation, excellence, and empowering our clients to achieve their financial goals.
Senior Data Engineer at Mex Digital.
About the role
You will design, construct, install, test, and maintain highly scalable data pipelines with a focus on machine learning models and analytics. You will work closely with data scientists, ML engineers, and stakeholders to ensure data is accessible, consistent, and reliable for ongoing projects. You will develop and maintain APIs for data access and manipulation, integrating with external data services as required. You will manage and optimize data storage solutions for both structured and unstructured data, covering text, image, audio, and video to support machine learning model requirements. You will implement data quality monitoring processes to ensure production data is always accurate and available. You will collaborate with ML engineers on data-related technical issues and provide architectural guidance. You will support metadata management and data governance using cataloging tools such as OpenMetadata. You will ensure compliance with data security and privacy policies across all data systems. You will maintain clear and up-to-date documentation including data dictionaries, metadata, and architectural diagrams. You will contribute to real-time and streaming data processing requirements where applicable.
Key facts
What you'll do
- Design, construct, install, test, and maintain highly scalable data pipelines with a focus on machine learning models and analytics
- Work closely with data scientists, ML engineers, and stakeholders to ensure data is accessible, consistent, and reliable for ongoing projects
- Develop and maintain APIs for data access and manipulation, and integrate with external data services as needed
- Manage and optimize data storage solutions for both structured and unstructured data, including relational databases, NoSQL databases, search engines such as Elasticsearch, and object storage, covering text, image, audio, and video data to support machine learning model requirements
- Implement data quality monitoring processes to ensure production data is always accurate and available to key stakeholders
- Collaborate with ML engineers on data-related technical issues and provide architectural guidance and solutions
- Support metadata management and data governance using cataloging tools such as OpenMetadata
- Ensure compliance with data security and privacy policies across all data systems
- Maintain clear and up-to-date documentation including data dictionaries, metadata, and architectural diagrams
- Contribute to real-time and streaming data processing requirements where applicable
Requirements
- Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field, or equivalent professional experience
- 8 or more years of experience in a data engineering role
- Proficiency in Python, Java, and Scala, with experience managing large-scale data at terabyte to petabyte scale
- Hands-on experience with big data technologies including Hadoop, Spark, and Flink
- Familiarity with machine learning frameworks such as TensorFlow, PyTorch, or similar
- Str