SDE II - Data Engineer

Required Nice to have

-

Build data ingestion and transformation workflows using Python, SQL, Apache Spark/PySpark, and relevant data engineering frameworks.

-

Work with data lakehouse architectures and contribute to ingestion, storage, transformation, and serving layers.

-

Implement data quality checks, schema validation, error handling, and monitoring to ensure data accuracy and reliability.

-

Support data modelling and query optimisation for analytical and reporting workloads.

-

Contribute to pipeline performance improvements, resource utilisation, and efficient compute and storage usage.

2. Distributed Processing and Streaming

-

Work with distributed data processing technologies such as Apache Spark, PySpark, and Apache Kafka .

-

Develop and support streaming workflows for processing event-driven data, with attention to data freshness, reliability, and processing efficiency.

-

Implement mechanisms for handling duplicate, delayed, or failed events, including retries, checkpointing, and recovery where applicable.

-

Investigate production issues and troubleshoot data pipeline failures, performance bottlenecks, and data quality discrepancies.

-

Contribute to the adoption and effective use of technologies such as Apache Flink, Spark Structured Streaming, and open table formats where relevant to the platform.

3. Data Storage and Analytics

-

Work with cloud-based data platforms and storage services to support scalable data processing and analytics.

-

Develop and optimise SQL queries, data transformations, partitioning strategies, and table structures.

-

Contribute to analytical data stores such as StarRocks, ClickHouse, or similar technologies, based on project requirements.

-

Work with data lake table formats such as Apache Iceberg, Delta Lake, or Apache Hudi, where applicable.

-

Support data modelling, metadata management, access controls, and data retention practices in line with established engineering standards.

4. Collaboration and Engineering Practices

-

Collaborate with backend engineers, ML engineers, and other stakeholders to understand data requirements and deliver reliable data solutions.

-

Work with workflow orchestration tools such as Apache Airflow or equivalent to schedule, monitor, and manage data pipelines.

-

Participate in code reviews, technical discussions, testing, and deployment activities.

-

Follow engineering best practices for version control, CI/CD, observability, documentation, and production readiness.

-

Contribute to continuous improvement by identifying opportunities to enhance pipeline reliability, scalability, performance, and cost efficiency.

Required Skills and Qualifications

Experience and Education

-

2–5 years of experience in software engineering, data engineering, or big data development.

-

Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field.

-

Experience developing and maintaining data pipelines in a production environment.

-

Experience handling large datasets or distributed data processing workloads is preferred.

-

Exposure to ad-tech, programmatic advertising, real-time bidding (RTB), or event-driven systems is an added advantage.

Core Data Engineering Skills

-

Strong programming skills in Python .

-

Good command of SQL , including joins, aggregations, data transformations, and query optimisation.

-

Hands-on experience with Apache Spark/PySpark or a comparable distributed data processing framework.

-

Experience building and maintaining ETL/ELT pipelines.

-

Understanding of batch processing and streaming data concepts.

-

Exposure to workflow orchestration tools such as Apache Airflow, Prefect, or Dagster.

-

Understanding of data modelling, partitioning, schema evolution, and data quality practices.

Streaming, Cloud, and Data Platform Skills

-

Familiarity with Apache Kafka or similar messaging and streaming technologies.

-

Exposure to AWS, GCP, or Azure data services.

-

Familiarity with data lakes, lakehouse architectures, or analytical databases.

-

Exposure to Apache Iceberg, Delta Lake, Apache Hudi, StarRocks, or ClickHouse is an advantage.

-

Understanding of monitoring, logging, error handling, and production troubleshooting.

Preferred Skills

-

Experience processing high-volume event data or building near-real-time data pipelines.

-

Exposure to ad-tech datasets such as bid requests, impressions, clicks, conversions, campaign performance, or attribution data.

-

Familiarity with Java or Scala in addition to Python.

-

Basic understanding of REST APIs, microservices, and event-driven architectures.

-

Exposure to Docker, Kubernetes, and CI/CD tools is a plus.

-

Experience collaborating with ML teams on feature engineering or training data pipelines is an advantage.

Since 2007, InMobi has built the extraordinary. Beyond just technology. It has built a way of working.

Across 16 offices globally, through every pivot and breakthrough, one thing has remained constant: we are builders, dreamers, doers. We are fluid enough to move fast, grounded enough to own outcomes, and human enough to care deeply about the work and about one another.

Culture is at the core of that ethos. It has been co-created by the people who built this company and by those continuing to shape it every day. It lives in how we work, how decisions are made, how teams show up for one another, and how seriously we take the responsibility to build the best.

This is a place where ambition and belonging occupy the same space. Where the bar is high and the support is real. Where freedom comes with trust, and trust comes with accountability. Where you are valued not only for what you do, but for the perspective, judgment and humanity you bring to the work. Where you will be challenged by the work and supported by the communities, passions and experiences that shape who you are beyond your role.

If this sounds like the place you have been looking for, join us.

Find out more here: InMobi Culture