Big Data Development Engineer

About the role

Bybit is a global digital asset trading platform that serves millions of users across the world with real-time market data and trading services. The Big Data Development Engineer will join the data engineering team based in Kuala Lumpur, Malaysia, where they will play a central role in shaping the company's data infrastructure. This position focuses on building and maintaining the large-scale data pipelines and processing systems that power trading analytics, risk management, and business intelligence operations. The engineer will collaborate with cross-functional teams across the organization to design reliable data solutions that meet the needs of both internal stakeholders and external market participants.

Key facts

What you'll do

• Design and implement large-scale data processing pipelines for trading and market data ingestion.
• Build and optimize ETL workflows that reliably ingest data from multiple exchange and market sources.
• Develop and maintain comprehensive data models that support complex analytics and business reporting requirements.
• Collaborate closely with the trading and risk management teams to understand their evolving data needs.
• Monitor data pipeline performance continuously and troubleshoot production issues to ensure high availability and reliability.
• Write efficient, well-structured code in Python or Java for data transformation and processing tasks.
• Ensure data quality by implementing automated validation checks and monitoring alerts across all pipelines.
• Participate actively in code reviews and contribute to the team's engineering best practices and standards.
• Document data architecture decisions thoroughly and share technical knowledge with team members through internal sessions.
• Work with the platform engineering team to deploy data services reliably into production environments.
• Architect scalable data lake solutions to store and serve both historical and real-time data efficiently.
• Drive continuous improvement by evaluating new tools and technologies for the evolving data stack.
• Optimize existing data workflows for performance, cost efficiency, and scalability across growing data volumes.
• Partner with data scientists and analysts to deliver clean, accessible datasets for their projects.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related technical discipline from an accredited institution.
• Three or more years of professional experience in data engineering, software development, or a closely related field.
• Strong proficiency in at least one general-purpose programming language such as Python, Java, or Scala.
• Hands-on experience with distributed data processing frameworks like Apache Spark, Flink, or equivalent technologies.
• Solid understanding of SQL, database design principles, and relational data modeling best practices.
• Familiarity with major cloud platforms such as AWS or Google Cloud Platform for building data infrastructure.
• Demonstrated experience building and maintaining real-time or near-real-time data pipelines in a production setting.
• Good understanding of data modeling techniques, ETL design patterns, and data warehouse architecture concepts.
• Ability to work effectively in a fast-paced environment with competing priorities and tight deadlines.
• Strong problem-solving skills combined with a meticulous attention to detail in data quality assurance.

Nice to have

• Prior experience working with cryptocurrency exchanges or financial market data systems and platforms.
• Knowledge of stream processing tools such as Apache Kafka, Kinesis, or comparable messaging systems.
• Familiarity with containerization technologies including Docker and orchestration platforms like Kubernetes for deployment.
• Exposure to data governance practices, data cataloging tools, and metadata management frameworks.
• Prior experience working in a high-throughput, low-latency trading or financial technology environment.
• Understanding of data security practices and regulatory compliance requirements relevant to financial services.

Skills & tools

• Python or Java programming for building data-intensive applications and services.
• Apache Spark, Flink, or similar distributed processing frameworks for large-scale data workloads.
• SQL and NoSQL databases including PostgreSQL, MySQL, Cassandra, or Redis for data storage.
• Apache Kafka or equivalent tools for event streaming, message queuing, and data ingestion.
• Cloud services on AWS or GCP including S3, BigQuery, Redshift, or equivalent data warehouses.
• Docker and Kubernetes for containerized deployment and orchestration of data processing services.
• Airflow or similar workflow orchestration tools for scheduling and managing data pipelines.
• Git and version control practices for collaborative software development and code management.

Practical notes

• This position is based in the Kuala Lumpur office and is expected to be performed on-site.
• The role reports directly to the Head of Data Engineering within the Kuala Lumpur-based team.
• Standard business hours apply, with some flexibility required for on-call responsibilities during critical incidents.
• Bybit offers competitive compensation and benefits packages along with a collaborative work environment for data professionals.
• Employees in this role may be required to participate in on-call rotations for production data systems and services.

About the company

Bybit Fintech Limited, known as Bybit, is a Dubai based centralized cryptocurrency exchange. The platform has faced regulatory warnings in several jurisdictions.