Data Engineer With Python

Job Role - Data Engineer
1. At least 4 to 6 years of relevant experience in data engineering team with 2-3 years hands-on experience in supporting Azure cloud technologies
2. Hands-on Expertise in Azure Data Engineering tool stack primarily Azure Data Factory, Azure Data Lake, logic Apps and Azure SQL,MPP (Azure Synapse)
3. Deep knowledge and hands-on on creating Azure Databricks notebooks (Good knowledge on PySpark, Databricks SQL, Delta Lake concepts) or Synapse Spark
4.Good understanding and exposure to Production Support of data integration and ETL systems with an understanding of SLAs, cross team collaboration and expectation setting.
5.Good understanding and experience around Data Warehousing, ETL/ELT process, Relational Databases, Slowly changing dimension considerations etc.
6.Should be familiar with Azure Synapse Workspace and good understanding of the Best practices for Data Loading to various target systems like Azure Synapse, data lake etc from various source systems
7.Understanding and able to build complex data transformation logic using Azure Databricks Pyspark for structured, semistructured data with different formats
8.Good understanding of No-SQL databases and experience with the same
9.Good understanding of SQL, Can write stored procedures, CTE queries, functions, understanding of normalization
10.Understanding of data loading from different source file formats (parquet, xml, json, csv etc), Experience with data Extraction from different data sources (Relational, FTP, SAP etc)
11.Good understanding of security requirements of different Azure components
12.Experience in best practices to resolve issues due to ingestion of varied data volumes by code related changes, optimal usage of clusters, partitioning and other methods
13.Worked on 1-2 projects involving migration of large scale projects from On-Prem to cloud and also having a decent understanding of remediation of performance issues
14.Understanding of Devops (CICD processes for automated deployments) and as applicable for data engineering (ADF, ADB, SQL deployments)
15.Very good understanding of Agile to deliver consistent project outcome
16.Very high data affinity and strong analytical skills
17.Good understanding of data modelling concepts
18.Analyzing and translating business needs into long-term solution data models
19.Evaluating and analysis (reverse engineering) of existing data systems.
20.Creation of Conceptual data models and data flows, Physical data models and data strategies
20.Nice to have:- Certification in DP-203 (Microsoft Certified: Azure Data Engineer Associate),Databricks Data Engineering Certification
LEVEL OF EXPERTISE
Python - 2 years
Azure - 5 years
Azure Data Factory - 3 years
Azure DataBricks - 3 years
ETL(Extract, Transform, Load) - 3 years