Tata Consultancy Services
|Systems Engineer - Data Engineer and Analyst (Azure)
Kolkata, West Bengal, India
Summary
Served as a Systems Engineer, Data Engineer, and Azure Analyst at Tata Consultancy Services, leading end-to-end data solution development and administration for major clients like Becton, Dickinson and Company and GlaxoSmithKline.
Highlights
Administered and developed on Denodo Platform, managing data virtualization and integration for critical business applications across multiple client projects.
Designed and implemented complex data virtualization solutions using Denodo Virtual DataPort, creating Base, Integration, and Report Views from diverse sources (Hive, SQL, SharePoint) to enable real-time analytics for Qlik and Power BI.
Managed data security and authentication within Denodo, enhancing data governance, and utilized Denodo Data Catalog for metadata management.
Streamlined SDLC by migrating Denodo Views and tables across Dev, QA, and Production environments using Denodo Solution Manager, ensuring SSL certification and Java-based system updates.
Developed and monitored Azure Data Factory (ADF) pipelines and datasets for clients, ensuring real-time data ingestion and processing, and providing real-time monitoring support.
Created Hive and HQL tables using DB Visualizer, integrating data from diverse sources including Attunity Replication, Oracle ETL, JDE, Everest, SAP, and Salesforce.
Integrated Microsoft SQL Server data with Power BI and Qlik dashboards via dedicated Azure Data Factory pipelines, enhancing business intelligence capabilities for end-users.
Leveraged Azure Databricks and PySpark to integrate SQL services within a Python framework, executing Scala code in a distributed environment for large-scale data processing and modeling.