Dr Emmanuel Ogungbemi

Head of Data Engineering | I help you improve your productivity with data and AI

From being a data analyst to leading the development of data pipelines, my journey in data engineering and analytics has been both challenging and rewarding.

When I began my career in data, I faced the daunting task of turning vast amounts of raw data into actionable insights.

The problem? Many organisations struggle with fragmented data and inefficient processes.

Here's how I tackled these challenges and built a successful career in data:

1. Crafted Data Solutions: Designed and implemented a Common Data Platform, ensuring seamless data integration and access.
2. Led Innovative Projects: At NATS, I contributed to the world's first AI system for air traffic control, utilising digital twinning and machine learning.
3. Developed Future Talent: Shaped data science and engineering education at Multiverse by developing comprehensive curricula for degree-level apprenticeships.
4. Consulted on High-Impact Projects: Provided AI and data science consultancy for industry giants like BP, Sword and Chaucer (now BIP UK), driving data-driven decision-making and innovation.

Now, I'd like to talk about you.

Are you aspiring to become a successful data analyst/data scientist/data engineer but need help figuring out where to start?

Here's a roadmap to guide you:

1. Master the Fundamentals:
- Learn key programming languages like Python and SQL.
- Understand data structures, algorithms, and database management.

2. Gain Practical Experience:
- Work on real-world projects to apply your skills.
- Use tools like Power BI and Tableau for data visualisation.

3. Join the Community:
- Participate in data science forums and attend industry events.
- Network with professionals to exchange knowledge and opportunities.

4. Continuous Learning:
- Stay updated with the latest trends and technologies in data analytics.
- Take advanced courses to deepen your expertise.

To help you get started, I've created a comprehensive Data Analyst Roadmap that outlines the essential steps and resources you need to succeed.

Join my email list to register your interest in my free Data Analyst Roadmap.

You have the potential to excel in the data industry, and I'm here to support your journey.

Let's connect and share our experiences in data engineering and analytics!

Head of Data Engineering

Excited to share an essential resource: Linear Regression!

This comprehensive guide delves into the fundamentals and advanced techniques of linear regression. 

Perfect for students, data scientists, and anyone interested in understanding this crucial statistical method, this document provides detailed explanations and practical examples.

Key topics include:
- Introduction to Linear Regression: Understanding the basics and importance of linear regression in data analysis.
- Mathematical Foundations: A detailed explanation of the linear regression model, including equations and assumptions.
- Data Visualization: The Importance of Visualizing Data and Interpreting Linear Regression Results.
- Model Evaluation: Techniques for evaluating the performance of linear regression models.
- Practical Applications: Real-world examples and applications of linear regression in various fields.


Enhance your understanding of linear regression and apply this knowledge to your data analysis projects with this detailed guide!

- Repost to share with others
- Save to revisit down the road
- Add your comment!

Dr Emmanuel O.

I help you break into data science and AI with practical tips, real-world insights, and the latest trends.

Linear Regression

Excited to share an essential resource: Python Machine Learning!

This comprehensive guide delves into the fundamentals and advanced techniques of machine learning using Python. 

Perfect for students, data scientists, and anyone looking to enhance their understanding of machine learning, this document provides detailed explanations and practical examples.

Key topics include:
- Getting Started with Python Machine Learning: Introduction to machine learning and Python, essential libraries, and the basics of data handling.
- Classification with Real-world Examples: Using the Iris dataset, visualization techniques, building and evaluating classification models, and cross-validation.
- Clustering: Finding related posts, preprocessing text data, using bag-of-words and KMeans clustering.
- Topic Modeling: Latent Dirichlet Allocation (LDA), building and evaluating topic models, and applying them to real-world datasets.
- Regression and Recommendations: Building recommendation systems, using regression models, and improving recommendations with advanced techniques.
- Computer Vision and Pattern Recognition: Introduction to image processing, pattern recognition, and building classifiers for image data.
- Dimensionality Reduction: Techniques for feature selection and extraction, principal component analysis (PCA), and other dimensionality reduction methods.
- Handling Big Data: Strategies for processing large datasets, using tools like Jug and Amazon Web Services (AWS).


Enhance your machine-learning skills and apply this knowledge to real-world projects with this detailed guide!

- Repost to share with others
- Save to revisit down the road
- Add your comment!

Building Machine Learning Systems with Python

I am excited to share an essential resource: Exam Ref 70-761, Querying Data with Transact-SQL by Itzik Ben-Gan!

This comprehensive guide is perfect for anyone preparing for the Microsoft Exam 70-761, covering key topics in T-SQL querying and programming constructs. 

Whether you're a developer, DBA, or data analyst, this book provides the knowledge and practice you need to succeed.

Key topics include:
- Creating Transact-SQL SELECT Queries: Understanding logical query processing, filtering data, sorting data, and combining sets with set operators.
- Querying Multiple Tables Using Joins: Cross joins, inner joins, outer joins, and composite joins.
- Implementing Functions and Aggregating Data: Type conversion functions, date and time functions, character functions, CASE expressions, and aggregate functions.
- Modifying Data: Inserting, updating, deleting data, merging data, and using the OUTPUT option.
- Querying Data with Advanced T-SQL Components: Subqueries, APPLY operator, table expressions, pivoting, and window functions.
- Programming Databases Using Transact-SQL: Creating views, user-defined functions, stored procedures, error handling, transactions, and handling NULLs.


Enhance your T-SQL skills and prepare effectively for your certification with this detailed guide!

- Repost to share with others
- Save to revisit down the road
- Add your comment!

Querying Data with Transact-SQL

Understanding the key terms in data analysis is essential for anyone working with data. 

This concise guide breaks down fundamental concepts, making it easier for you to navigate the world of data. 

Whether you're a beginner or looking to refresh your knowledge, this resource is perfect for you.

Key terms include:
- Data Profiling: Examining and summarizing data characteristics
- Data Interpretation: Making sense of data and deriving insights
- Data Manipulation: Transforming data for analysis
- Exploratory Data Analysis (EDA): Visualizing data to uncover patterns
- Data Mining: Discovering patterns in large datasets
- Data Storytelling: Conveying insights in a compelling narrative
- Data Visualization: Graphical representation for better understanding
- Data Validation: Ensuring data accuracy and reliability

Master these terms and enhance your data analysis skills!

- Repost to share with others
- Save to revisit down the road
- Add your comment!

Data Analysis Vocabulary

Excited to share an essential resource: SQL: Comprehensive Guide and Examples!

This detailed guide covers a wide range of SQL topics, from getting started with basic queries to advanced techniques. 

Perfect for students, professionals, and anyone looking to deepen their SQL knowledge, this document provides valuable insights and practical examples.

Key topics include:
- Getting Started with SQL: Overview, Syntax, and Examples.
- ALTER TABLE: Modify columns, add/drop constraints, and alter table structure.
- AND & OR Operators: Combining conditions in SQL queries.
- Cascading Delete: Maintaining data integrity with cascading deletes.
- CASE Statements: Implementing conditional logic in queries.
- Clean Code in SQL: Best Practices for Writing Readable and Maintainable SQL Code.
- Common Table Expressions (CTEs): Simplifying complex queries with CTEs.
- CREATE DATABASE and CREATE TABLE: Setting up databases and tables.
- Data Types: Understanding different data types in SQL.
- DELETE Statements: Removing data from tables.
- JOINs: Combining data from multiple tables with various types of joins.
- Subqueries: Using subqueries in SELECT, FROM, and WHERE clauses.
- Transactions: Managing data consistency with transactions.
- Indexes: Improving query performance with indexing.
- Stored Procedures: Creating and using stored procedures.


Enhance your SQL skills and master database management with this comprehensive guide!

- Repost to share with others
- Save to revisit down the road
- Add your comment!

undefined undefined

The award of this certificate is in recognition of my participation in the SEEP 2019 conference held between 18 - 21 November 2019 at the University of Sharjah, United Arab Emirates where I delivered two (2) papers.

Certificate of Participation

The certificate is a result of my participation in the 5th International Conference on Material Science & Smart Materials where I presented a poster titled "Characterisation of Solar Materials and Electronic Devices using Cadmium Telluride (CdTe) and Zinc Sulphite (ZnS) as a Case Studies"

Certificate

The certificate is a result of my participation in the SEEP 2018 conference where I presented a paper titled "Effect of Membrane Thickness on the Performance of PEM Fuel Cell"

Certificate of National Service

I received this letter due to my invaluable contribution to the successful delivery of the first historic comprehensive training programme for the Volunteer Cadets in Nigeria, the OYES Volunteer Cadets' Training Course.  

Letter of Appreciation

The certificate is a recognition of my service to the club as the Vice-President (Business) in 2010.

Certificate of Fellowship

The award is a recognition of my service to the student society as General Secretary - 2007/2008, Vice-President - 2008/2009, President - 2009/2010 and National Vice-President (West) - 2010/2011.

Certificate of Service

This certificate is due to my participation in the 40th annual general meeting and conference of the society.

Certificate of Attendance

I received this certificate as a result of my participation in the international essay competition organised by the World Bank. The title is "How does youth unemployment affect you? How can you tackle youth unemployment through youth-led solutions"

Certificate of Merit

Marketing: Copywriting for Social Media

Developing CI/CD Solutions with Azure DevOps

Python Projects

Microsoft Certified: Azure Data Engineer Associate

Microsoft Certified: Azure AI Fundamentals

Microsoft Certified: Azure Data Fundamentals

Microsoft Certified: Azure Fundamentals

Data Engineering Foundations

[PCAP-31-03] PCAP – Certified Associate in Python Programming

Agile at Work: Building Your Agile Team

Scrum: The Basics

Stay Lean with Kanban

Machine Learning Essentials for Seismic Interpretation

Starting a Memorable Conversation

Artificial Intelligence Foundations: Neural Networks

Diversity, Inclusion, and Belonging

Machine Learning and AI Foundations: Classification Modeling

Body Language for Leaders

Industrial/Oilfield Health, Safety & Environmental Management Appreciation

Introduction to Auditing and Evaluating Behaviour Change (IEMA approved training)

International Association of Engineers

Institution of Chemical Engineers (IChemE)

The Institution of Engineering and Technology

Emmanuel is dedicated to helping you break into data science and AI with practical tips, real-world insights, and the latest trends.

Emmanuel Ogungbemi | Substack

We are delighted to be back for the second edition of ML Club Live!

This event is all about getting hands-on with GenAI, with some coding examples and also some discussions of interesting papers.

As always, this is community-driven, so if you want to come and speak about anything related to ML or AI, even for 10 minutes, please reach out to Emmanuel (bunmieogungbemi@gmail.com) or Andy (andrewpmcmahon629@gmail.com).

ML Club - July 2024 - Hands on with GenAI, Tue, Jul 2, 2024, 5:30 PM | Meetup

I coordinate Real Projects Talk for AFBE-UK Scotland

Organising Data Science and Engineering events
- To help newbies get into data science
- Organise boot camps to support learning and development
- Support working professionals by providing mentorship and advice
- https://www.meetup.com/ml-club/

Learning SQL

Excited to share an invaluable resource: Comprehensive Python Cheatsheet by Jure Šorn!

This extensive cheat sheet covers the fundamental concepts and advanced techniques of Python programming. 

Perfect for beginners and experienced developers, this document provides a detailed reference for a wide range of Python functionalities.

Key topics include:
- Collections: Lists, dictionaries, sets, tuples, ranges, and more.
- Types: Strings, regular expressions, numbers, and datetime.
- Syntax: Arguments, inline functions, closures, decorators, and classes.
- System Operations: Exit, print, input, command-line arguments, file handling, and OS commands.
- Data Handling: JSON, Pickle, CSV, SQLite, bytes, and memory views.
- Advanced Topics: Threading, operators, metaprogramming, evaluation, and coroutines.
- Libraries: Progress bars, plotting, tables, web scraping, profiling, NumPy, and more.


Enhance your Python programming skills and keep this comprehensive cheatsheet as a quick reference guide!

- Repost to share with others
- Save to revisit down the road
- Add your comment!

Comprehensive Python Cheatsheet

I am excited to share an invaluable resource: Basic SQL!

This comprehensive guide provides a deep dive into SQL fundamentals, covering everything from data definition and constraints to basic retrieval queries and set operations. 

Perfect for students, professionals, and anyone looking to strengthen their SQL skills, this document offers detailed explanations and practical examples.

Key topics include:
- SQL Data Definition and Data Types: Creating tables, specifying data types, and understanding schemas.
- Specifying Constraints in SQL: Primary keys, unique keys, foreign keys, and check constraints.
- Basic Retrieval Queries in SQL: Using SELECT statements, WHERE clauses, and basic data retrieval techniques.
- Set Operations in SQL: UNION, EXCEPT, and INTERSECT operations.


Enhance your database management skills and master SQL with this detailed guide!

- Repost to share with others
- Save to revisit down the road
- Add your comment!

Basic SQL

Excited to share an invaluable resource: Docker Tutorial by Anthony Baire!

This comprehensive tutorial covers everything you need to know about Docker, from the basics of managing containers to advanced security and ecosystem considerations. 

Whether you're new to Docker or looking to enhance your skills, this guide provides clear and detailed insights into effective containerization practices.

Key topics include:
- Introduction to Docker and its components
- Managing Docker containers: creation, interaction, and lifecycle
- Handling inputs/outputs and data volumes
- Building and managing Docker images
- Security considerations and best practices
- Exploring the Docker ecosystem and future trends


Unlock the full potential of Docker for your projects with this detailed tutorial!

- Repost to share with others
- Save to revisit down the road
- Add your comment!

Docker Tutorial

I am excited to share an essential resource: Scalable Machine Learning with Apache Spark!

This comprehensive guide is perfect for anyone looking to build and scale machine-learning models using Apache Spark. 

Covering everything from data processing pipelines to distributed hyperparameter tuning, this document provides valuable insights and practical knowledge for data scientists and engineers.

Key topics include:
- Data Processing Pipelines: Creating and managing efficient data workflows with Spark.
- Machine Learning Models: Building, Tuning, and Deploying ML Models with Spark ML.
- Model Tracking and Deployment: Using MLflow to track, version, and deploy machine learning models.
- Hyperparameter Tuning: Performing distributed hyperparameter tuning with Hyperopt.
- Scaling Inference: Scaling the inference of single-node models with Spark.


Enhance your machine learning capabilities and streamline your data engineering processes with this detailed guide!

- Repost to share with others
- Save to revisit down the road
- Add your comment!

Scalable Machine Learning with Apache Spark

Excited to share an essential resource: SQL Fundamentals!

This comprehensive guide covers the foundational concepts of SQL, making it perfect for students, professionals, and anyone looking to deepen their understanding of SQL and database management. 

From basic queries to advanced techniques, this document provides a thorough introduction to SQL.

Key topics include:

Introduction to SQL SELECT Statements: Learn how to retrieve data from databases using basic SELECT statements.
Basic SELECT Statement Syntax: Understand the structure and components of a basic SELECT statement, including SELECT, FROM, and WHERE clauses.
Writing SQL Statements: Guidelines for constructing clear and readable SQL statements.
Selecting All Columns and Rows: Techniques for retrieving all data from tables.
Selecting Specific Columns: How to display specific columns of a table.
Column Heading Defaults and Aliases: Customizing column headings for better readability.
Arithmetic Expressions and Operator Precedence: Using arithmetic operators and understanding their precedence in expressions.
Handling Null Values: Defining and working with null values in SQL.
Concatenation and Literal Character Strings: Combining columns and adding literal strings in SQL queries.
Eliminating Duplicate Rows: Using the DISTINCT keyword to remove duplicate rows in query results.
SQL and SQL*Plus Interaction: Overview of SQL*Plus commands and their differences from SQL statements.
Displaying Table Structure: Using the DESCRIBE command to view table structures.
Enhance your SQL skills and master database management with this detailed guide!

Repost to share with others
Save to revisit down the road
Add your comment!

Writing Basic SQL Statements

I am excited to share an insightful resource: Improving Python and Spark Performance and Interoperability with Apache Arrow by Julien Le Dem and Li Jin!

This comprehensive guide delves into optimizing Python and Spark interoperability using Apache Arrow. 

Perfect for data scientists, engineers, and anyone working with big data, this document provides a deep dive into enhancing performance and efficiency in data processing.

Key topics include:

Current Limitations of PySpark UDFs: Understanding the constraints and performance issues with existing user-defined functions.
Overview of Apache Arrow: Introduction to Arrow, its goals, and benefits for in-memory columnar storage.
Performance Improvements: Real-world examples and benchmarks showcasing significant speedups in PySpark integration.
Vectorized UDFs: Leveraging pandas DataFrames for efficient data processing and reducing overhead.
Group UDFs: Techniques for improving group-based operations and data normalization.
Future Roadmap: Upcoming enhancements and support for Arrow in the Apache ecosystem.


Enhance your data processing workflows and achieve higher performance with this detailed guide on Apache Arrow and PySpark!

Repost to share with others
Save to revisit down the road
Add your comment!

Improving Python and Spark Performance and interoperabi

Pre-Interview Checklist for Big Data Engineer Roles.

1. SQL Essentials:
- SELECT statements, including WHERE, ORDER BY, GROUP BY, HAVING
- Basic Joins: INNER, LEFT, RIGHT, and FULL
- Aggregate functions: COUNT, SUM, AVG, MAX, MIN
- Subqueries, Common Table Expressions (WITH clause)
- CASE statements, advanced JOIN techniques, and window functions (OVER, PARTITION BY, ROW_NUMBER, RANK)

2. Python Programming:
- Basic syntax, control structures, data structures (lists, dictionaries)
- Pandas & NumPy for data manipulation: DataFrames, Series, groupby

3. Hadoop Ecosystem Proficiency:
- Understanding HDFS architecture, replication, and block management.
- Mastery of MapReduce for distributed data processing.
- Familiarity with YARN for resource management and job scheduling.

4. Hive Skills:
- Writing efficient HiveQL queries for data retrieval and manipulation.
- Optimizing table performance with partitioning and bucketing.
- Working with ORC, Parquet, and Avro file formats.

5. Apache Spark:
- Spark architecture
- RDD, DataFrame, Datasets, and Spark SQL
- Spark optimization techniques
- Spark Streaming

6. Apache HBase:
- Designing effective row keys and understanding HBase’s data model.
- Performing CRUD operations and integrating HBase with other big data tools.

7. Apache Kafka:
- Deep understanding of Kafka architecture, including producers, consumers, and brokers.
- Implementing reliable message queuing systems and managing data streams.
- Integrating Kafka with ETL pipelines.

8.Apache Airflow:
- Designing and managing DAGs for workflow scheduling.
- Handling task dependencies and monitoring workflow execution.

9. Data Warehousing and Data Modeling:
- Concepts of OLAP vs. OLTP
- Star and Snowflake schema designs
- ETL processes: Extract, Transform, and load
- Data lake vs. data warehouse
- Balancing normalization and denormalization in data models.

10. Cloud Computing for Data Engineering:
- Benefits of cloud services (AWS, Azure, and Google Cloud)
- Data storage solutions: S3, Azure Blob Storage, and Google Cloud Storage
- Cloud-based data analytics tools: BigQuery, Redshift, and Snowflake
- Cost management and optimization strategies

If I had to start learning #dataengineering all over again

(I will follow this plan, mostly in order)

1. Learn SQL
— Aggregations with GROUP BY
— Joins (INNER, LEFT, FULL OUTER)
— Window functions and Common table expressions

2. Learn about data modeling
— read about data normalization and 3rd normal form
— read about fact, dimension, and aggregate tables
— read about efficient table designs such as cumulative

3. Learn Python
— Learn loops, if statements
— Try out some really solid libraries like Pandas, numpy, scikit-learn
— Will not waste time on Machine Learning

4. Learn about data quality
— how do you write a good data check?
— how can you implement the write-audit-publish pattern in your pipelines?

5. Learn about distributed compute, and job orchestration
— Read about MapReduce 
— learn about partitioning, skew, and spilling to disk
— try a scheduler like Airflow

If you follow this, you’ll feel way more confident in interviews
#DataEngineering

Understanding SQL: How We Write It vs. How It's Executed

Have you ever wondered why your SQL queries don't always run as expected? 
- There's a difference between how we write SQL and how the database processes it. 

Let's break it down:

When we write SQL queries, we typically follow this sequence:

- SELECT
- TOP
- DISTINCT
- FROM
- WHERE
- GROUP BY
- HAVING
- ORDER BY
- LIMIT

Execution Order:
However, the database engine processes these commands in a different order:

- FROM - Identifies the data source.
- WHERE - Filter records based on conditions.
- GROUP BY - Aggregates data into groups.
- HAVING - Filter groups based on conditions.
- SELECT - Retrieves specified columns.
- ORDER BY - Sorts the result set.
- TOP/LIMIT - Limits the number of rows in the result set.

Why This Matters:
- Misunderstanding the execution order can lead to unexpected results or inefficient queries.
- Knowing the execution order helps you write more effective and optimized SQL queries.
- Improved query performance and more accurate data retrieval.

Tips for Writing Efficient SQL:
- Always start by understanding your data and defining clear requirements.
- Use the execution order to optimize your queries for better performance.
- Practice writing and analyzing different queries to see how changes affect execution.

Mastering the execution order of SQL not only makes your queries more efficient and enhances your ability to troubleshoot and optimize complex data operations.

♻ Repost to share with others
🔖 Save to revisit down the road
💬 Add your comment!

Querying data with SQL

Understanding Git: Your Ultimate Version Control System 🚀

Ever wondered what makes Git so essential?

Let’s break it down for you:

1/ What is Git?
↳ Git is a version control system that tracks code changes.
↳ It’s a tool used by developers to collaborate on projects.
↳ With Git, you can revert to previous versions of your project anytime.

2/ Git vs. GitHub
↳ Git is NOT the same as GitHub.
↳ GitHub is a hosting service that makes use of Git.
↳ GitHub allows developers to collaborate globally.

3/ Why Git Matters
↳ Over 70% of developers use Git!
↳ It tracks every change made to your project.
↳ It allows multiple people to work on the same project from anywhere.

4/ Key Git Features
↳ Every time a file is changed, Git tracks it.
↳ You can see the entire history of changes (commits).
↳ Git allows you to revert back to earlier versions easily.

5/ Git Branching
↳ Create separate branches to work on different features.
↳ Merge branches when the feature is complete.
↳ Switch between branches without disrupting the main project.

6/ Git Commands to Know
↳ $ git init: Initializes a new Git repository.
↳ $ git add <filename>: Stages files for commit.
↳ $ git commit -m “message”: Commits changes with a message.

7/ Pushing to GitHub
↳ After committing changes locally, push them to GitHub.
↳ $ git push origin master: Sends your code to the master branch on GitHub.
↳ GitHub hosts your project so others can contribute or review it.

Git makes collaboration easier and code management more efficient. 
- Mastering Git is essential for every developer!
________________________________________
♻️ Repost to share this with your network!
🔔 Follow Emmanuel Ogungbemi for more insights on Git and development tools.

Struggling with Data Management? Discover SQL!

Managing and analyzing data effectively is crucial. Are you finding handling large datasets challenging or generating meaningful insights from your data?

Here’s a powerful tool that can help: SQL.

1. What is SQL?
- Structured Query Language (SQL) is a standard language for accessing and manipulating databases.
- It lets you retrieve, update, and manage data in a relational database.
- SQL is widely used across various industries for data analysis, reporting, and business intelligence.

2. Why Learn SQL?
- Efficient Data Handling: Quickly query and manipulate large datasets.
- Versatile Skills: SQL is used in various fields, from web development to finance.
- Career Growth: High demand for SQL skills in the job market.

3. Getting Started with SQL
- Basics First:
Learn fundamental commands like SELECT, INSERT, UPDATE, and DELETE.
Understand how to create and manage database tables.

- Practice with Real Data:
Use sample databases to practice your queries.
Explore platforms like Kaggle for datasets to work on.

- Advanced Techniques:
Dive into complex queries with JOIN, GROUP BY, and SUBQUERIES.
Learn about database optimization and indexing.

4. Why You Should Care:
- Problem: Handling data manually is time-consuming and prone to errors.
- Solution: SQL provides a systematic approach to data management, making it easier to analyze and derive insights.
- Value: Mastering SQL can streamline your data processes, improve accuracy, and enhance your decision-making capabilities.

SQL is an essential tool for anyone dealing with data.
Start learning today to unlock its full potential and take your data management skills to the next level.

♻ Repost to share with others
🔖 Save to revisit down the road
💬 Add your comment!

25 Most Common Interview Questions for Data Analyst Positions 🚀

I will hire you if you can answer these questions.

Here are 25 key questions to help you prepare:

Fundamentals:
1.	What is data analytics, and why is it important for businesses?
2.	Explain the difference between structured and unstructured data.
3.	What are the key steps in the data analysis process?
4.	How do you handle missing or corrupted data in a dataset?
5.	What are the key differences between quantitative and qualitative data?

Data Tools & Techniques:
6.	What is the role of SQL in data analysis?
7.	What are some key Excel functions used in data analysis?
8.	Explain the use of JOINs in SQL.
9.	What is data normalization in databases, and why is it important?
10.	How would you explain the concept of a relational database to someone new?

Data Visualization:
11.	Why is data visualization important in data analysis?
↳ Helps present complex data clearly to stakeholders.
12.	What are some popular data visualization tools?
↳ Mention Tableau, Power BI, and Excel as key tools.
13.	When would you use a bar chart vs. a line chart?
↳ Bar charts for comparing categories, line charts for trends over time.
14.	What are some best practices for creating a dashboard?
↳ Focus on clarity, simplicity, and relevance to the audience.
15.	Explain the use of heatmaps in data visualization.
↳ Heatmaps represent data using colour gradients to show patterns or density.

Statistical Concepts:
16.	What is the difference between correlation and causation?
↳ Correlation shows a relationship, while causation implies one variable causes the other.
17.	Explain the concept of standard deviation.
↳ Measures the amount of variation or dispersion in a dataset.
18.	What is the p-value in hypothesis testing?
↳ The p-value helps determine the significance of your results.
19.	How do you determine if a dataset is normally distributed?
↳ Use histograms, Q-Q plots, and skewness/kurtosis measures.
20.	What is A/B testing, and how is it used in decision-making?

Practical Implementation:
21.	How do you clean and prepare data for analysis?
↳ Explain steps like handling missing data, removing duplicates, and outlier treatment.
22.	What are the challenges of working with large datasets?
↳ Mention issues like performance, storage, and data quality.
23.	How do you handle outliers in your data?
↳ Depending on the context, you can remove, cap, or transform outliers.
24.	Explain a situation where you identified a problem and solved it using data.
↳ Be prepared to walk through a real-life example with clear steps.
25.	How do you ensure data accuracy and integrity throughout your analysis?
↳ Double-check data entry, validate sources and perform consistency checks.

________________________________________

Found these questions helpful?
👍 React
♻️ Share
💬 Comment

So more data analysts can prepare for interviews!

________________________________________

Good luck with the interviews! 💼

Pyspark can be HARD

But learning these concepts made it easier for you.

𝟭. 𝗜𝗻𝘁𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝘁𝗼 𝗣𝘆𝘀𝗽𝗮𝗿𝗸:
- Spark Architecture & Apache Spark Overview
- Setting up the PySpark Environment
- SparkContext and SparkSession

𝟮. 𝗥𝗗𝗗𝘀 (𝗥𝗲𝘀𝗶𝗹𝗶𝗲𝗻𝘁 𝗗𝗶𝘀𝘁𝗿𝗶𝗯𝘂𝘁𝗲𝗱 𝗗𝗮𝘁𝗮𝘀𝗲𝘁𝘀):
- Creating RDDs
- Transformations and Actions (Narrow vs Wide)
- RDD Operations (map, filter, reduce, etc.)

𝟯. 𝗗𝗮𝘁𝗮𝗙𝗿𝗮𝗺𝗲 𝗮𝗻𝗱 𝗗𝗮𝘁𝗮𝘀𝗲𝘁 𝗔𝗣𝗜𝘀:
- DataFrame operations (select, filter, groupBy, etc.)
- Reading, writing data & inferring schemas
- Spark File Layout
- How to refer columns in DF and DS

𝟰. 𝗦𝗽𝗮𝗿𝗸 𝗦𝗤𝗟 𝗮𝗻𝗱 𝗗𝗮𝘁𝗮 𝗦𝗼𝘂𝗿𝗰𝗲𝘀 :
- Running SQL Queries on DataFrames
- Registering DataFrames as Temporary Tables or Views
- SQL Functions and UDFs
- Working with various formats (CSV, JSON, Parquet, Avro)
- Hive Tables and External Databases (JDBC)

𝟱. 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝘁𝗲𝗰𝗵𝗻𝗶𝗾𝘂𝗲𝘀:
- Broadcast and Accumulator Variables
- Caching, Persistence, and Memory Management
- Data Skew Handling
- Partition Pruning and Shuffle Optimization

𝟲. 𝗣𝘆𝘀𝗽𝗮𝗿𝗸 𝗦𝘁𝗿𝗲𝗮𝗺𝗶𝗻𝗴:
- Structured Streaming Basics
- Windowing Operations
- Integrating with Kafka for Real-time Data Processing
- Fault Tolerance and Checkpointing

Data skills will always be relevant.

So learning skills focused on data will last a lifetime.

Here are 3 career options to consider in data.

𝗗𝗮𝘁𝗮 𝗔𝗻𝗮𝗹𝘆𝘀𝘁:
- SQL
- Python
- Excel
- Power BI / Tableau
- Statistical Analysis
- Data Warehousing

𝗗𝗮𝘁𝗮 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴:
- SQL
- Python
- Hadoop
- Hive
- HBase
- Kafka
- Airflow
- PySpark
- CI/CD
- Data Warehousing
- Data modeling
- AWS/Azure/GCP

𝗗𝗮𝘁𝗮 𝗦𝗰𝗶𝗲𝗻𝘁𝗶𝘀𝘁:
- SQL
- Python/R
- Artificial intelligence
- Statistics & Probability
- Machine Learning
- Deep Learning
- Data Wrangling
- Mathematics (Linear Algebra, Calculus)

If you've read so far, do SHARE the post.

An aspiring Data Engineer aspirant asked me a question.

Why is PySpark so important to learn?

I showed them this list:

- PySpark + DataFrame API = Data Manipulation
- PySpark + RDD = Distributed Datasets
- PySpark + filter() = Data Filtering
- PySpark + join() = Data Integration
- PySpark + groupBy() = Data Aggregation
- PySpark + orderBy() = Data Sorting
- PySpark + union() = Combining Datasets
- PySpark + withColumn() = Data Transformation
- PySpark + select() = Column Selection
- PySpark + SQL Queries = SQL Integration
- PySpark + createOrReplaceTempView() = Virtual Tables
- PySpark + map() = Data Mapping
- PySpark + reduceByKey() = Data Reduction
- PySpark + partitionBy() = Data Partitioning
- PySpark + broadcast() = Data Broadcasting
- PySpark + accumulators = Shared Variables
- PySpark + Spark SQL = Structured Data
- PySpark + DataFrame Caching = Performance Optimization
- PySpark + Window Functions = Advanced Analytics
- PySpark + UDFs = Custom Functions
- PySpark + Machine Learning = Scalable Models
- PySpark + GraphX = Graph Processing
- PySpark + Streaming = Real-Time Processing
- PySpark + DataFrame Joins = Efficient Merging
- PySpark + MLlib = Machine Learning
- PySpark + Structured Streaming = Continuous Processing
- PySpark + Pipeline API = Workflow Automation
- PySpark + Delta Lake = Reliable Lakes
- PySpark + Databricks = Cloud Platform
- PySpark + ETL Pipelines = Data Extraction
- PySpark + Performance Tuning = Query Efficiency
- PySpark + Cluster Management = Distributed Computing

With consistency and effort, you will get there.

LIKE and SHARE, if you can relate to this post.

Chief of Staff | Speaker | Strategic Growth & GTM Partner | Operational Excellence | Helping Companies Capture Opportunity & Scale Through Change

I help growth-stage companies turn strategy into scalable results — aligning revenue, operations, and execution as they grow, integrate, or enter new markets.With 18+ years leading complex initiatives across regulated and operationally intensive industries, I embed inside organizations to execute strategic priorities and align GTM, delivery, and operations to deliver measurable revenue outcomes.Who I Help:✦ SaaS and technology companies seeking a strategic partner to shape product positioning and execute growth-focused GTM strategies.✦ Growth-stage companies outgrowing their operating model and struggling to execute consistently at scale.✦ CEOs and executive teams who need structured operating rhythms and execution discipline to drive results.✦ Companies navigating expansion, integration, or restructuring across people, process, systems, and technology.How I Deliver Results:↳ Chief of Staff &amp; Strategic Advisory: Partner directly with the CEO and executive leadership to translate strategy into measurable roadmaps, structure operating rhythms, and drive aligned execution across product, delivery, and GTM.↳ GTM &amp; Growth StrategyLead the design and execution of GTM and growth strategies—serving as a strategic advisor and market interface across product, delivery, and buyer engagement.↳ Operational Excellence &amp; Operating Model Design:Build scalable operating models, clarify ownership and accountability, and establish prioritization systems that translate strategy into measurable outcomes.↳ Operational &amp; Data EnablementBridge operations and technology by designing performance dashboards, reporting frameworks, and technology-enabled workflows that increase visibility and strengthen decision-making.↳ Integration &amp; Value Creation: Lead post-merger integration, restructuring, and scaling initiatives to reduce risk, improve EBITDA, and strengthen long-term enterprise value.📬 Let’s Connect!For advisory, partnership, or Chief of Staff support, send a DM or contact FlightofThePhx@morgan-davis.com.🎤 Speaking &amp; Sponsorship Inquiries:I speak on enterprise Portfolio, Change &amp; Execution Leadership and Future of Work &amp; Career Reinvention.Contact FlightofThePhx@morgan-davis.com to inquire about speaking opportunities.🐦‍🔥 Flight of the Phoenix Business Community:Looking to grow your career or a business? Grow with us! We help leaders grow their careers and explore Portfolio Careers &amp; Fractional Leadership.Learn more: https://nas.io/flightofthephoenix

Morgan Davis, PMP, PROSCI, MBA

Moving mountains starts with moving stones.

Progress is incremental.

♻️ Reshare to remind your network that success is built through small, intentional actions.

➕ Follow Morgan Davis, PMP, PROSCI, MBA for more leadership insights.

Right before a plane takes off, the resistance is at its highest.

The engines are loud.
The cabin shakes.
You’re focused on takeoff.

It’s one of the most critical moments of the entire flight.

Then suddenly… you’re in the air.

If you’re feeling the resistance right now, you might be closer to flying than you think.

Hang on.
The takeoff is part of the journey.

♻️ Reshare to remind someone that resistance comes before takeoff.

➕ Follow Morgan Davis, PMP, PROSCI, MBA for more leadership insights.

Most organizations chase results. 

Few embed the change that makes them last.

Anybody can hustle their way to a result.

Close the deal.
Hit the number.
Improve the margin.
Ship the project.

If that result isn’t integrated into how you or your organization operates, it becomes people-dependent — and temporary.

When your A-player leaves, the result leaves with them.

If the change isn’t integrated into the operating model, the system will pull you back.

Friction always wins the long game.

So remove it.

↳ Redesign the workflow.
↳ Clarify ownership.
↳ Automate repeatable work.
↳ Institutionalize knowledge.
↳ Align incentives.
↳ Embed new behaviors into daily operations.

Engineer performance into the system.

If your results are cyclical, the issue isn’t effort.
It’s integration.

♻️ Reshare to remind your network that systems create success.

➕ Follow Morgan Davis, PMP, PROSCI, MBA for more operations excellence insights.

The grind after the grind is the real grind.

Sunday Funday. 

To all the hustlers out there: get some rest too.

♻️ Reshare to celebrate perseverance.

➕ Follow Morgan Davis, PMP, PROSCI, MBA for more leadership insights.

Not everything has to be perfect to still be good.

It’s National Don’t Cry Over Spilled Milk Day 🥛

The saying isn’t about pretending mistakes don’t matter.
It’s about knowing that once something’s happened… crying won’t unspill it.

You clean it up.
You take the lesson.
Then you move forward.

Because everyone spills something.
And what you do next matters more than the spill itself.

♻️ Reshare this to remind someone in your network that even when something goes wrong, something good can still come from it.

Follow Morgan Davis, PMP, PROSCI, MBA Davis for more leadership insights.

The professionals accelerating right now are not the most qualified.
They are the most visible, connected, and trusted.

Opportunities don’t come from job boards.
They come from conversations.

Promotions don’t come from perfect resumes.
They come from credibility and relationships.

Clients don’t buy the best service.
They buy the person they trust to help them deliver results.

Networking, personal branding, and communication skills are no longer optional.

They define career success and often determine whether you land the client or the next opportunity.

If you're not getting the opportunities or clients you want, this is your gap — and it is widening fast.

If you want support building these capabilities for yourself or your organization, DM Morgan Davis, PMP, PROSCI, MBA.

♻️ Reshare to remind your network that opportunity comes from relationships.

➕ Follow Morgan Davis, PMP, PROSCI, MBA for more leadership and career growth insights.

You can come up with a million reasons not to start.

And most of them are valid.

Someone else:
↳ Has more experience
↳ Has a bigger network
↳ Moves faster
↳ Knows the playbook

The real problem is not starting because you can’t fly (yet).

No one starts by flying.
They start by starting.

You crawl. Then you learn to walk. Then you run. Then you fly.
There are no shortcuts.

Don’t let perfection or fear of judgment keep you stuck.

What have you been waiting to start?

♻️ Reshare to remind someone that progress starts small.

➕ Follow Morgan Davis, PMP, PROSCI, MBA for more leadership insights.

Be obsessed with your customers.

Not your product.

Most founders start with what they want to build.
The real shift is starting with who you serve.

When you start with customers:

↳ You gain faster traction.
Because you’re solving a problem.

↳ You increase conversion rates.
Because your product or service clearly addresses a defined pain point.

↳ You strengthen your positioning.
Because when your offer is anchored in a real problem, customers can immediately see themselves in it.

↳ You lower customer acquisition cost.
Relevance reduces friction and hesitation in the buying decision.

When your solution aligns with a real pain point, it stands out.
And differentiation drives revenue.

Here’s what this looks like in practice:

✅ Start with the customer.

Define exactly who you serve and the specific pain they are trying to solve.

✅ Design around their problem.

Build solutions around existing pain points.

✅ Build value from their perspective.

Tie your offer to outcomes, results, and transformation.

✅ Lead with outcomes, not offers.

Sell the results before the offer.

If you’re struggling to gain traction, pause — and refocus on your customer. 
Realign your offer and positioning to the problem they need solved.

Are you building from your product — or from your customer?

♻️ Reshare to help your network start with the people they serve.

➕ Follow Morgan Davis, PMP, PROSCI, MBA for more growth strategy insights.

Recently, a colleague asked me if it was worth it.

Yes.

Two years ago, I started my LinkedIn journey after it was recommended by my career counselor at UNC Chapel Hill. 

At the time, I felt stuck in my career — up against a glass ceiling and completely burnt out. I was at a major turning point in both my career and my life.

I stopped relying on other people to tell my story and started listening to my advisors. 

I joined a creator accelerator, invested heavily in intentionally shaping my network, and began posting daily about my experiences leading change, executing strategy, and helping organizations grow.

It felt uncomfortable at first. It definitely turned heads. 

But that’s what growth feels like — betting on yourself, taking risks, and committing to the long game.

The opportunities, relationships, and momentum that followed have been incredible.

Today I’m celebrating my Favikon rankings:

 #8 Project Management Creator on LinkedIn in the United States
 #29 Project Management Creator on LinkedIn Worldwide

This journey has led to partnerships, speaking opportunities, clients, and a growing community of leaders navigating change, building portfolio careers, and developing their personal brands.

If you’re feeling stuck like I was, change is possible. 

Investing in your personal brand can open doors you don’t even see yet — whether you’re growing in your company or building something new.

Thank you to everyone who has been part of the journey so far. I appreciate your continued support.

♻️ Reshare to remind your network that visibility creates opportunities.

➕ Follow Morgan Davis, PMP, PROSCI, MBA for more career insights.

Everyone is carrying something you can’t see.

Choose patience today.

♻️ Reshare to inspire kindness in your network.

➕ Follow Morgan Davis, PMP, PROSCI, MBA for more leadership insights.

The pace of change in the workplace has accelerated dramatically.

What got you here may not get you where you want to go next — and many leaders and job seekers are feeling the gap.

Here are the trends:

👉  The job market has changed.

75% of resumes are rejected by ATS systems before a human ever sees them (Korn Ferry, Future of Work Trends, 2024).

Submitting applications alone is no longer a reliable strategy, and many highly qualified candidates are struggling to gain traction in today’s job market.

👉   Organizations are experiencing constant disruption.

Technology, tariffs, and generational workforce shifts are driving restructuring, layoffs, and growing instability in traditional career paths.

Leaders are being asked to navigate continuous change while keeping teams aligned and moving forward.

👉 Leadership expectations are rapidly evolving.

Nearly 70% of executives say the skills gap is a major barrier to transformation (Deloitte, Global Human Capital Trends, 2023). 

This highlights the growing gap between what organizations need from leaders and what many were originally trained to do.

👉AI is accelerating the shift.

23% of jobs are expected to change by 2027 due to technology and economic shifts (World Economic Forum, Future of Jobs Report, 2023).

Workers are being forced to learn new skills at an unprecedented pace.

My advice: disruption creates opportunity — but only if you know how to navigate it and position yourself.

Due to increasing demand from my community and LinkedIn network, I’m opening a limited number of spots for 60-minute Leadership & Career Coaching sessions.

These sessions are for professionals who want clarity and direction as they navigate career and leadership change.

This is a structured working session where we identify your biggest constraint, define a focused 90-day priority, and build a practical action plan.

I only have capacity to support a small number of professionals 1:1 this quarter.

If you’ve been considering getting support, now is the time to act — these sessions are filling quickly.

You can learn more and book your session using the link below.

https://lnkd.in/eYNt5mwm

What’s the biggest career or leadership challenge you’re navigating right now?

Drop it in the comments. 👇 

♻️ Reshare to help your network navigate career uncertainty.

➕ Follow Morgan Davis, PMP, PROSCI, MBA for more leadership and career insights.

Open article: 60-Minute Leadership and Career Coaching Session by nas.io

60-Minute Leadership and Career Coaching Session

Are you getting likes… but not leads?

Conversations… but not conversions?

That gap is usually positioning.

If buyers can’t immediately understand who you help, what problem you solve, and why you’re different — they hesitate.

And hesitation slows your growth.

Tomorrow, I’m going live with Bryan McDonald to unpack how to position your expertise so it consistently attracts the right customers.

We’ll explore:
↳ What creates value in the eyes of your ideal customers
↳ The common mistakes that weaken your brand signal
↳ How to promote yourself in a way that builds credibility
↳ How founders should think about personal brand vs. company brand to unlock growth

I’ll share what I’ve learned from building my own brand and advising growth-stage leaders navigating this exact challenge.

If you’re feeling stuck, unclear, or under-leveraged in the market, this session will help you sharpen your positioning and attract the right business.

The session is free. Details are below.

♻️ Reshare to help your network attract their ideal clients.

➕ Follow Morgan Davis, PMP, PROSCI, MBA for insights on scaling and growing businesses.

Consultants & Experts Needing to Sell Their Expertise & Themselves | Fractional Consulting Expert | I help you make MORE MONEY than you did in corporate America. Selling a problem? DM me to find out why it’s really not!

Morgan Davis, PMP, PROSCI, MBA, Monika Singh and 79 other attendees

 Position Your Expertise to Attract Your Ideal Customers

Sometimes the strongest resistance to change comes from the top.

Leaders anchor to past wins — even when conditions and assumptions have changed.

The perceived risk feels too high. Or the initiative gets dismissed as “not a priority.”

When that resistance isn’t addressed early, it can shut down your transformation or strategic initiative before it ever has a real chance to start.

So how do you turn resistance at the top into alignment?

Pilots.

Pilots are a strategic tool for reducing risk, defining guardrails, and aligning stakeholders before full-scale commitment.

When designed intentionally, they create psychological safety and meaningful engagement — the foundation of lasting change.

I’ll be sharing how leaders can use pilots as a low-risk, high-trust method to test, learn, and adapt initiatives before scaling.

You’ll walk away with:

 ↳ A clear way to de-risk innovation and build trust
 ↳ A simple framework to design an effective pilot
 ↳ Engagement strategies that turn resistance into learning
 ↳ A Pilot Terms of Reference template you can use immediately

If you’re implementing technology, leading large-scale transformation, or trying to move hesitant stakeholders to yes — this session will give you skills to help move your stakeholders forward.

Registration details are below. 

The session is hosted by the PMI Delaware Valley Chapter and open to the general public.

Where does change get stuck in your organization?

Drop your thoughts below. 👇 

♻️ Reshare to help your network learn the power of a pilot.

➕ Follow Morgan Davis, PMP, PROSCI, MBA for more insights on leading organizational change.

Morgan Davis, PMP, PROSCI, MBA and 11 other attendees

View event: The Power of the Pilot: Building Trust Before You Scale

The Power of the Pilot: Building Trust Before You Scale

What’s the secret sauce to attracting your ideal customers?

Positioning.

Positioning is the intentional way you show your expertise to the market so your ideal buyers understand exactly why they should choose you.

It’s how you frame your value so it resonates — and when it’s unclear, it may be the very thing holding you back.

Next week, I’ll be joining Bryan McDonald to unpack the core components of powerful brand positioning and the common pitfalls that dilute your impact.

We’ll explore:
↳ What creates value in the eyes of your ideal customers
↳ The common mistakes that weaken your brand signal
↳ How to promote yourself in a way that builds credibility
↳ How founders should think about personal brand vs. company brand to unlock growth

I’ll share what I’ve learned from building my own brand and advising growth-stage leaders navigating this exact challenge.

If you’re feeling stuck, unclear, or under-leveraged in the market, this session will help you sharpen your positioning and attract the right business. 

The session is free. Details are below.

♻️ Reshare to help your network attract their ideal clients.

➕ Follow Morgan Davis, PMP, PROSCI, MBA for insights on scaling and growing businesses.

taplio

The Battle

Dr Emmanuel Ogungbemi

Morgan Davis, PMP, PROSCI, MBA

Followers

Posts

Engagements

Likes

Performance ScoreDr Emmanuel Ogungbemi's LinkedIn Performance

Performance ScoreMorgan Davis, PMP, PROSCI, MBA's LinkedIn Performance

Ready to grow your LinkedIn presence?

taplio

The Battle

Dr Emmanuel Ogungbemi

Morgan Davis, PMP, PROSCI, MBA

Followers

Posts

Engagements

Likes

Performance Score.css-goub6s{font-size:var(--chakra-fontSizes-md);font-weight:500;color:var(--chakra-colors-gray-600);display:block;margin-top:var(--chakra-space-1);}Dr Emmanuel Ogungbemi's LinkedIn Performance

Performance ScoreMorgan Davis, PMP, PROSCI, MBA's LinkedIn Performance

Ready to grow your LinkedIn presence?

Performance ScoreDr Emmanuel Ogungbemi's LinkedIn Performance