Business Analyst

Data modeling, Data warehousing, ETL pipelines, Flink, Kafka, Spark, Kinesis, Airflow, Python, Java, Scala, Monitoring

Years of Exp : 6+
Education : Bachelor's degree in Computer Science or related field.

Key Responsibilities:
* Design, develop, and maintain scalable data pipelines and ETL processes
* Optimize data flow and collection for cross-functional teams
* Build infrastructure required for optimal extraction, transformation, and loading of data
* Ensure data quality, reliability, and integrity across all data systems
* Collaborate with data scientists and analysts to help implement models and algorithms
* Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, etc.
* Create and maintain comprehensive technical documentation
* Evaluate and integrate new data management technologies and tools
* Monitoring using tools like Datadog/Grafana

Mandatory Skills:
* Extensive experience with big data technologies (e.g., Spark, Flink, Hadoop), Terraform, Cloud Formation, Kubernetes, DataDog/Grafana, CI/CD
* Experience in monitoring using any one tool is mandatory - DataDog/Grafana and the like
* Experience with containerization and orchestration tools (Kubernetes)
* Experience with data modeling, data warehousing, and building ETL pipelines
* Experience with cloud platforms (AWS, Azure, or GCP) and their data services. AWS Preferred
* Experience with building streaming pipelines with flink, Kafka, Kinesis. Flink Preferred.
* Strong knowledge of data pipeline and workflow management tools (e.g., Airflow, Luigi, NiFi)
* Expert knowledge of SQL and experience with relational databases (e.g., PostgreSQL, Redshift, TIDB, MySQL, Oracle, Teradata)
* Proficiency in at least one programming language such as Python, Java, or Scala
* Understanding of data governance and data security principles
* Experience with version control systems (e.g., Git) and CI/CD practices

Preferred Skills :
* Basic knowledge of machine learning workflows and MLOps
* Experience with NoSQL databases (MongoDB, Cassandra, etc.)
* Familiarity with data visualization tools (Tableau, Power BI, etc.)
* Experience with real-time data processing
* Knowledge of data governance frameworks and compliance requirements (GDPR, CCPA, etc.)