Freelance (Self employed)

Working as Azure data engineer responsible for data migration from MS SQL server to Microsoft fabric data warehouse incorporating full load and incremental load using SQL stored procedure through SSMS
• Copy activity in fabric is used for data migration from lakehouse to warehouse (bronze and silver layer for business logic transformation-joins) and PySpark notebook code is used to transform the stage/raw data before loading it into the data warehouse (gold layer) where update and insert to dim and fact tables is done (full load and master tables are only inserted based on created on date and incremental tables are updated based on I or U records on created and modified date)
• Python MySQL connector is used when the source is Azure SQL database and pandas library is used to store the table in dataframe objects
• Work responsible for raw data migration, cleaning, transforming and archiving from csv files in azure storage container to Azure synapse analytics data factory using PySpark notebook into master, demographic, dim and fact tables
• Existing data is left as it is, updated data is appended with lsk modified in scheme table and new data is inserted in masters table using copy activity New (insert) and old (existing and updated and common) data is identified based on new and old records in members table using inner and full sql joins
• Data is inserted by first writing it into parquet files then loading into dedicated sql
pool tables using copy activity based on partitioned records

Freelance Data Engineer

IIT Roorkee

DAMAGE ANALYSIS OF 2.5 D WOVEN COMPOSITES USING PHASE FIELD METHOD
(PRESENTED AT 4th SICE CONFERENCE IIT HYDERABAD DEC 2022 WITH EXPERIENCE
CERTIFICATE)- Formulated a plan of action for the given research problem of
damage analysis (fiber, matrix and interface debonding damage) of woven
composites under uniaxial loading and boundary conditions using phase field
method (finite element method) and worked in collaboration with PhD researchers
to arrive at the solution
Performed simulations for phase field damage in composites using ABAQUS CAE
and performed a parametric study of the damage and strength of 2.5 D woven
composites used in aerospace industry
Used MATLAB to construct RVE(Representative Volume Element) model of the 2.5 D
woven composite from micro-CT images and applied image processing(
grayscale) to filter and clean the images before construction
Used Solidworks/AutoCAD/Abaqus CAE to convert the RVE model into finite
element (FEM) model
Used Fortran programming language to simulate the damage of the 2.5 D woven
composite using phase field model

Student

Indian Institute of Technology, Guwahati

Key Qualifications and Responsibilities:
Collaborated with other M.Tech students to create teaching syllabus for teaching AutoCAD and
ABAQUS to B.Tech Mechanical students, prepare simulation tests for Engineering Drawing and select
additional teaching materials.
Provided additional support to students with learning difficulties during Engineering Drawing classes
using compass and rulers and provide classroom management by identifying strength and weakness.
Part of organizing event to create effective M.Tech teaching programs, schedules and routines for
students from other visiting engineering colleges to create interest and facilitate learning.
Created engaging AutoCAD simulations of engineering drawing syllabus by introducing new teaching
method and materials and real life examples such as Car Jack machine increasing student retention by
25%.
Responsible for checking the engineering drawing sheets of students and grading them according to
standard grading system and maintain success report of each student on a regular basis using weekly
tests and improve performance of students by 50%.
Effectively communicate with fellow colleagues, administration and parents to meet and achieve
individual student goals by providing effective feedback

Teaching at M.Tech IIT Guwahati

Gameskraft

PG FEE ETL JOB - I worked on a delta etl job that would run every 10 minutes. The etl would also do
backfilling for the past 12 months payments and would get the settled amount and adjust for data before
October. It covered the payment gateways which were used for transaction for the users to add cash
before registering to our rummy culture website.
DELAYED VIEW ETL JOB - I worked on daily etl that would provide us with the one hour max delayed
view of all installs and downloads of our rummy culture app apk that the users would be doing. The etl
would pick up data from transactional database of rummy culture (MYSQL) and ship the data to amazon
redshift using amazon S3 as cloud storage.
COST DATA WAREHOUSE JOB - I worked on daily etl that would provide us with the user acquisition
cost involved for our rummy culture website through google and facebook ads. The etl would also use
the google and facebook APIs to load the data from google and facebook marketing campaigns and
ship the daily level data (costs, clicks, impressions etc. ) for a particular keyword associated with an ad
to amazon redshift.
All the code was reviewed and pushed to production.

Senior Data Engineer

Walmart Global Tech India

JDA - I developed a Data Lake repository solution by extracting data for markets from various
sources and constructed a ETL flow using yaml files and automic as scheduling object for specific
business transformations and finally load output data to Object Store data repository, which in turn was
subscribed and used by JDA application.
Data was extracted from sources - Teradata, Mainframe, Informix, DB2 using TDCH Connectors in case
of Teradata and a combination of load ready files, copybook and mainframe logic for Mainframe. For
Infomix and DB2 data was loaded using SqoopConnector. Target destination was HDFS (Data lake)
and external and internal tables were used over the HDFS extracted data and join queries to load the
data using hive to final catalog tables.
Before the final stage checksum validation was applied between HDFS and Swift to check the accuracy
of data.
Finally the data was loaded to swift object store repository (data objects accessed using URIs) from
HDFS by constructing external object store tables over the data residing in HDFS.
GOVERNANCE - I worked on getting insights of data stored in HDFS by performing profiling operations
on it. From this, we got to know understanding of data in the tables such as -
Number of zeros
Negatives in an integer column
Max, Min value for a column
String Frequency Distribution
Pattern Frequency Distribution
Length Frequency Distribution
PET - I developed a data lake solution by scraping weather data, store data, holiday data from URL
and loading it to external files and construct an ETL flow given business transformations (halo and
cannibalization logic) and finally load output data to data lake which in turn will feed data to promotions
forecasting analytics.
DQ FRAMEWORK - I worked on implementing data quality framework rules as a check on quality of
data between staging and landing tables. The rules applied are -
Not-null check
Duplicate check
All the codes were reviewed and pushed to production.

Data Engineer

Practo

PRACTO PARTNER APP - Worked on developing the Caller-ID feature for Practo Partner app.Worked
on redesigning the reports widget fragment for Practo Partner app. Handled fragments andactivity life
cycles. Implemented Services, Intent receivers, Async Tasks, RestApi integration as awrapper around
Volley Networking Library and Bottom Sheet Layouts. All code
was reviewed and pushed into production.

Software Development Intern

ONGC Foundation

ONGC Internship - Underwent an internship at ONGC plant in Sivasagar Assam. The training involved
understanding of various equipment and systems used in drilling and oil wells and various site work in
the following sections- Compressor and Tc shop, Diesel shop, Heavy equipment repair shop, Ultrasonic
testing shop and Machine shop.

Internship Trainee

Adept Data Engineer with a proven track record at Gameskraft, leveraging skills in PySpark, SQL, and Azure Synapse analytics to enhance data migration and transformation processes. Collaborated effectively with teams, demonstrating strong communication and problem-solving abilities, to push significant code to production, optimizing ETL workflows and data warehousing strategies.

Taplio

Devorshi Bhattacharjee's Linkedin Analytics

Want to drive more opportunities from LinkedIn?

Taplio

.css-1jgreyo{background-image:linear-gradient(284deg, rgba(39,43,72,1) 0%, rgba(33,47,183,1) 100%);color:transparent;-webkit-background-clip:text;background-clip:text;}Devorshi Bhattacharjee's Linkedin Analytics

Want to drive more opportunities from LinkedIn?

Devorshi Bhattacharjee's Linkedin Analytics