Data Engineering - Senior
Job description
Data Engineering - Technical Lead
About Us
Paytm stands as India's leading digital payments and financial services company. Our mission is to connect consumers and merchants through diverse payment use cases. For merchants, we supply acquiring devices such as Soundbox, EDC, QR scanners, and Payment Gateway, with payment aggregation managed via PPI and other bank instruments. We expand merchant capabilities through advertising and the Paytm Mini app store. Using this platform use, we extend credit offerings including merchant loans, personal loans, and BNPL, funded by our financial partners.
About the Role
This position focuses on executing complex technical initiatives alongside collaborative peers in a fast-moving setting. The ideal candidate combines strong product design instincts with deep specialization in Hadoop and Spark technologies. You will influence analytics direction while ensuring system robustness, scalability, and performance.
Requirements
Between four and eight years of hands-on experience with Big Data technologies.
Capacity to grow our analytics capabilities through faster, more dependable tools that manage petabyte-scale data with low latency and horizontal scalability.
Ability to brainstorm and build platforms that serve cluster users in varied forms while maintaining efficiency.
Ownership of diagnosing problems across the full technical stack.
Capacity to design and develop a real-time events pipeline for data ingestion to power real-time dashboards.
Skill in developing complex, efficient functions that convert raw data sources into reliable data lake components.
Experience designing and implementing new components using Hadoop ecosystem technologies and successful project execution.
Embodiment of Paytm values, staying hungry, humble, and relevant in every contribution.
Skills That Support Success
Hands-on expertise with Hadoop, MapReduce, Hive, Spark, and PySpark.
Strong programming and debugging capabilities in Python, Java, and Scala.
Experience with at least one scripting language such as Python or Bash.
Exposure to NoSQL databases like HBase and Cassandra.
Competence in multithreaded application development.
Understanding of databases, SQL, and messaging queues such as Kafka.
Background in streaming applications like Spark Streaming, Flink, or Storm.
Familiarity with AWS and cloud services including S3.
Experience with caching architectures such as Redis.
Why Join Us
You gain the chance to make a meaningful impact while working in an energizing environment. You will tackle work that matters to you and to the customers we serve. Join us if you genuinely consider what technology can achieve for people. Our achievements stem from the collective energy of our people and an unwavering focus on the customer, a principle that will continue to guide us.
Compensation
If you are the right fit, we are committed to creating value for you. Our ecosystem includes a large user base and extensive data assets that enable credit democratization for deserving consumers and merchants. This role places you at the center of India's digital lending evolution.
Job Details
Location: India
Engagement: Full-time Employment
What You Will Do
Design real-time event ingestion pipelines that enable instant dashboard visibility from varied sources.
Build frameworks that process petabytes of data daily while ensuring low latency and horizontal scalability.
Perform full-stack diagnosis to maintain reliability of data lake components.
Transform raw inputs into powerful, dependable data building blocks.
Introduce and execute emerging components from the Hadoop ecosystem.
Collaborate with partners to create ingestion workflows serving multiple consumer needs.
Support analytics growth with robust pipelines that underpin merchant credit decisions.
Monitor streaming applications to sustain performance and high availability.
Requirements
Four to eight years of hands-on experience with Big Data technologies and distributed processing systems.
Strong coding skills in Python, Java, or Scala for demanding data challenges.
Ability to work with multithreaded applications and automation scripts.
Understanding of NoSQL stores such as HBase and Cassandra, plus SQL and messaging queues.
Nice to Have
Experience with streaming frameworks such as Flink or Storm.
Familiarity with AWS services including S3 and caching via Redis.
Skills & Tools
Hadoop, MapReduce, Hive, Spark, PySpark, Python, Java, Scala, Bash, NoSQL, HBase, Cassandra, Kafka, SQL, AWS, S3, Redis, Flink, Storm.