1 Basic Principles of Data Wrangling 1 Akshay Singh, Surender Singh and Jyotsna Rathee 1.1 Introduction 2 1.2 Data Workflow Structure 4 1.3 Raw Data Stage 4 1.3.1 Data Input 5 1.3.2 Output Actions at Raw Data Stage 6 1.3.3 Structure 6 1.3.4 Granularity 7 1.3.5 Accuracy 7 1.3.6 Temporality 8 1.3.7 Scope 8 1.4 Refined Stage 9 1.4.1 Data Design and Preparation 9 1.4.2 Structure Issues 9 1.4.3 Granularity Issues 10 1.4.4 Accuracy Issues 10 1.4.5 Scope Issues 11 1.4.6 Output Actions at Refined Stage 11 1.5 Produced Stage 12 1.5.1 Data Optimization 13 1.5.2 Output Actions at Produced Stage 13 1.6 Steps of Data Wrangling 14 1.7 Do’s for Data Wrangling 16 1.8 Tools for Data Wrangling 16 References 17 2 Skills and Responsibilities of Data Wrangler 19 Prabhjot Kaur, Anupama Kaushik and Aditya Kapoor 2.1 Introduction 20 2.2 Role as an Administrator (Data and Database) 21 2.3 Skills Required 22 2.3.1 Technical Skills 22 2.3.1.1 Python 22 2.3.1.2 R Programming Language 25 2.3.1.3 Sql 26 2.3.1.4 MATLAB 27 2.3.1.5 Scala 27 2.3.1.6 Excel 28 2.3.1.7 Tableau 28 2.3.1.8 Power BI 29 2.3.2 Soft Skills 31 2.3.2.1 Presentation Skills 31 2.3.2.2 Storytelling 32 2.3.2.3 Business Insights 32 2.3.2.4 Writing/Publishing Skills 32 2.3.2.5 Listening 33 2.3.2.6 Stop and Think 33 2.3.2.7 Soft Issues 33 2.4 Responsibilities as Database Administrator 34 2.4.1 Software Installation and Maintenance 34 2.4.2 Data Extraction, Transformation, and Loading 34 2.4.3 Data Handling 35 2.4.4 Data Security 35 2.4.5 Data Authentication 35 2.4.6 Data Backup and Recovery 35 2.4.7 Security and Performance Monitoring 36 2.4.8 Effective Use of Human Resource 36 2.4.9 Capacity Planning 36 2.4.10 Troubleshooting 36 2.4.11 Database Tuning 36 2.5 Concerns for a DBA 37 2.6 Data Mishandling and Its Consequences 39 2.6.1 Phases of Data Breaching 40 2.6.2 Data Breach Laws 41 2.6.3 Best Practices For Enterprises 41 2.7 The Long-Term Consequences: Loss of Trust and Diminished Reputation 42 2.8 Solution to the Problem 42 2.9 Case Studies 42 2.9.1 UBER Case Study 42 2.9.1.1 Role of Analytics and Business Intelligence in Optimization 44 2.9.1.2 Mapping Applications for City Ops Teams 46 2.9.1.3 Marketplace Forecasting 47 2.9.1.4 Learnings from Data 48 2.9.2 PepsiCo Case Study 48 2.9.2.1 Searching for a Single Source of Truth 49 2.9.2.2 Finding the Right Solution for Better Data 49 2.9.2.3 Enabling Powerful Results with Self-Service Analytics 50 2.10 Conclusion 50 References 50 3 Data Wrangling Dynamics 53 Simarjit Kaur, Anju Bala and Anupam Garg 3.1 Introduction 53 3.2 Related Work 54 3.3 Challenges: Data Wrangling 55 3.4 Data Wrangling Architecture 56 3.4.1 Data Sources 57 3.4.2 Auxiliary Data 57 3.4.3 Data Extraction 58 3.4.4 Data Wrangling 58 3.4.4.1 Data Accessing 58 3.4.4.2 Data Structuring 58 3.4.4.3 Data Cleaning 58 3.4.4.4 Data Enriching 59 3.4.4.5 Data Validation 59 3.4.4.6 Data Publication 59 3.5 Data Wrangling Tools 59 3.5.1 Excel 59 3.5.2 Altair Monarch 60 3.5.3 Anzo 60 3.5.4 Tabula 61 3.5.5 Trifacta 61 3.5.6 Datameer 63 3.5.7 Paxata 63 3.5.8 Talend 65 3.6 Data Wrangling Application Areas 65 3.7 Future Directions and Conclusion 67 References 68 4 Essentials of Data Wrangling 71 Menal Dahiya, Nikita Malik and Sakshi Rana 4.1 Introduction 71 4.2 Holistic Workflow Framework for Data Projects 72 4.2.1 Raw Stage 73 4.2.2 Refined Stage 74 4.2.3 Production Stage 74 4.3 The Actions in Holistic Workflow Framework 74 4.3.1 Raw Data Stage Actions 74 4.3.1.1 Data Ingestion 75 4.3.1.2 Creating Metadata 75 4.3.2 Refined Data Stage Actions 76 4.3.3 Production Data Stage Actions 77 4.4 Transformation Tasks Involved in Data Wrangling 78 4.4.1 Structuring 78 4.4.2 Enriching 78 4.4.3 Cleansing 79 4.5 Description of Two Types of Core Profiling 79 4.5.1 Individual Values Profiling 80 4.5.1.1 Syntactic 80 4.5.1.2 Semantic 80 4.5.2 Set-Based Profiling 80 4.6 Case Study 80 4.6.1 Importing Required Libraries 81 4.6.2 Changing the Order of the Columns in the Dataset 82 4.6.3 To Display the DataFrame (Top 10 Rows) and Verify that the Columns are in Order 82 4.6.4 To Display the DataFrame (Bottom 10 rows) and Verify that the Columns Are in Order 83 4.6.5 Generate the Statistical Summary of the DataFrame for All the Columns 83 4.7 Quantitative Analysis 84 4.7.1 Maximum Number of Fires on Any Given Day 84 4.7.2 Total Number of Fires for the Entire Duration for Every State 85 4.7.3 Summary Statistics 86 4.8 Graphical Representation 86 4.8.1 Line Graph 86 4.8.2 Pie Chart 86 4.8.3 Bar Graph 87 4.9 Conclusion 89 References 90 5 Data Leakage and Data Wrangling in Machine Learning for Medical Treatment 91 P.T. Jamuna Devi and B.R. Kavitha 5.1 Introduction 91 5.2 Data Wrangling and Data Leakage 93 5.3 Data Wrangling Stages 94 5.3.1 Discovery 94 5.3.2 Structuring 95 5.3.3 Cleaning 95 5.3.4 Improving 95 5.3.5 Validating 95 5.3.6 Publishing 95 5.4 Significance of Data Wrangling 96 5.5 Data Wrangling Examples 96 5.6 Data Wrangling Tools for Python 96 5.7 Data Wrangling Tools and Methods 99 5.8 Use of Data Preprocessing 100 5.9 Use of Data Wrangling 101 5.10 Data Wrangling in Machine Learning 104 5.11 Enhancement of Express Analytics Using Data Wrangling Process 106 5.12 Conclusion 106 References 106 6 Importance of Data Wrangling in Industry 4.0 109 Rachna Jain, Geetika Dhand, Kavita Sheoran and Nisha Aggarwal 6.1 Introduction 110 6.1.1 Data Wrangling Entails 110 6.2 Steps in Data Wrangling 111 6.2.1 Obstacles Surrounding Data Wrangling 113 6.3 Data Wrangling Goals 114 6.4 Tools and Techniques of Data Wrangling 115 6.4.1 Basic Data Munging Tools 115 6.4.2 Data Wrangling in Python 115 6.4.3 Data Wrangling in R 116 6.5 Ways for Effective Data Wrangling 116 6.5.1 Ways to Enhance Data Wrangling Pace 117 6.6 Future Directions 119 References 120 7 Managing Data Structure in R 123 Mittal Desai and Chetan Dudhagara 7.1 Introduction to Data Structure 123 7.2 Homogeneous Data Structures 125 7.2.1 Vector 125 7.2.2 Factor 131 7.2.3 Matrix 132 7.2.4 Array 136 7.3 Heterogeneous Data Structures 138 7.3.1 List 139 7.3.2 Dataframe 144 References 146 8 Dimension Reduction Techniques in Distributional Semantics: An Application Specific Review 147 Pooja Kherwa, Jyoti Khurana, Rahul Budhraj, Sakshi Gill, Shreyansh Sharma and Sonia Rathee 8.1 Introduction 148 8.2 Application Based Literature Review 150 8.3 Dimensionality Reduction Techniques 158 8.3.1 Principal Component Analysis 158 8.3.2 Linear Discriminant Analysis 161 8.3.2.1 Two-Class LDA 162 8.3.2.2 Three-Class LDA 162 8.3.3 Kernel Principal Component Analysis 165 8.3.4 Locally Linear Embedding 169 8.3.5 Independent Component Analysis 171 8.3.6 Isometric Mapping (Isomap) 172 8.3.7 Self-Organising Maps 173 8.3.8 Singular Value Decomposition 174 8.3.9 Factor Analysis 175 8.3.10 Auto-Encoders 176 8.4 Experimental Analysis 178 8.4.1 Datasets Used 178 8.4.2 Techniques Used 178 8.4.3 Classifiers Used 179 8.4.4 Observations 179 8.4.5 Results Analysis Red-Wine Quality Dataset 179 8.5 Conclusion 182 References 182 9 Big Data Analytics in Real Time for Enterprise Applications to Produce Useful Intelligence 187 Prashant Vats and Siddhartha Sankar Biswas 9.1 Introduction 188 9.2 The Internet of Things and Big Data Correlation 190 9.3 Design, Structure, and Techniques for Big Data Technology 191 9.4 Aspiration for Meaningful Analyses and Big Data Visualization Tools 193 9.4.1 From Information to Guidance 194 9.4.2 The Transition from Information Management to Valuation Offerings 195 9.5 Big Data Applications in the Commercial Surroundings 196 9.5.1 IoT and Data Science Applications in the Production Industry 197 9.5.1.1 Devices that are Inter Linked 199 9.5.1.2 Data Transformation 199 9.5.2 Predictive Analysis for Corporate Enterprise Applications in the Industrial Sector 204 9.6 Big Data Insights’ Constraints 207 9.6.1 Technological Developments 207 9.6.2 Representation of Data 207 9.6.3 Data That Is Fragmented and Imprecise 208 9.6.4 Extensibility 208 9.6.5 Implementation in Real Time Scenarios 208 9.7 Conclusion 209 References 210 10 Generative Adversarial Networks: A Comprehensive Review 213 Jyoti Arora, Meena Tushir, Pooja Kherwa and Sonia Rathee List of Abbreviations 213 10.1 Introductıon 214 10.2 Background 215 10.2.1 Supervised vs Unsupervised Learning 215 10.2.2 Generative Modeling vs Discriminative Modeling 216 10.3 Anatomy of a GAN 217 10.4 Types of GANs 218 10.4.1 Conditional GAN (CGAN) 218 10.4.2 Deep Convolutional GAN (DCGAN) 220 10.4.3 Wasserstein GAN (WGAN) 221 10.4.4 Stack GAN 222 10.4.5 Least Square GAN (LSGANs) 222 10.4.6 Information Maximizing GAN (INFOGAN) 223 10.5 Shortcomings of GANs 224 10.6 Areas of Application 226 10.6.1 Image 226 10.6.2 Video 226 10.6.3 Artwork 227 10.6.4 Music 227 10.6.5 Medicine 227 10.6.6 Security 227 10.7 Conclusion 228 References 228 11 Analysis of Machine Learning Frameworks Used in Image Processing: A Review 235 Gurpreet Kaur and Kamaljit Singh Saini 11.1 Introduction 235 11.2 Types of ML Algorithms 236 11.2.1 Supervised Learning 236 11.2.2 Unsupervised Learning 237 11.2.3 Reinforcement Learning 238 11.3 Applications of Machine Learning Techniques 238 11.3.1 Personal Assistants 238 11.3.2 Predictions 238 11.3.3 Social Media 240 11.3.4 Fraud Detection 240 11.3.5 Google Translator 242 11.3.6 Product Recommendations 242 11.3.7 Videos Surveillance 243 11.4 Solution to a Problem Using ml 243 11.4.1 Classification Algorithms 243 11.4.2 Anomaly Detection Algorithm 244 11.4.3 Regression Algorithm 244 11.4.4 Clustering Algorithms 245 11.4.5 Reinforcement Algorithms 245 11.5 ml in Image Processing 246 11.5.1 Frameworks and Libraries Used for ML Image Processing 246 11.6 Conclusion 248 References 248 12 Use and Application of Artificial Intelligence in Accounting and Finance: Benefits and Challenges 251 Ram Singh, Rohit Bansal and Niranjanamurthy M. 12.1 Introduction 252 12.1.1 Artificial Intelligence in Accounting and Finance Sector 252 12.2 Uses of AI in Accounting & Finance Sector 254 12.2.1 Pay and Receive Processing 254 12.2.2 Supplier on Boarding and Procurement 255 12.2.3 Audits 255 12.2.4 Monthly, Quarterly Cash Flows, and Expense Management 255 12.2.5 AI Chatbots 255 12.3 Applications of AI in Accounting and Finance Sector 256 12.3.1 AI in Personal Finance 257 12.3.2 AI in Consumer Finance 257 12.3.3 AI in Corporate Finance 257 12.4 Benefits and Advantages of AI in Accounting and Finance 258 12.4.1 Changing the Human Mindset 259 12.4.2 Machines Imitate the Human Brain 260 12.4.3 Fighting Misrepresentation 260 12.4.4 AI Machines Make Accounting Tasks Easier 260 12.4.5 Invisible Accounting 261 12.4.6 Build Trust through Better Financial Protection and Control 261 12.4.7 Active Insights Help Drive Better Decisions 261 12.4.8 Fraud Protection, Auditing, and Compliance 262 12.4.9 Machines as Financial Guardians 263 12.4.10 Intelligent Investments 264 12.4.11 Consider the “Runaway Effect” 264 12.4.12 Artificial Control and Effective Fiduciaries 264 12.4.13 Accounting Automation Avenues and Investment Management 265 12.5 Challenges of AI Application in Accounting and Finance 265 12.5.1 Data Quality and Management 267 12.5.2 Cyber and Data Privacy 267 12.5.3 Legal Risks, Liability, and Culture Transformation 267 12.5.4 Practical Challenges 268 12.5.5 Limits of Machine Learning and AI 269 12.5.6 Roles and Skills 269 12.5.7 Institutional Issues 270 12.6 Suggestions and Recommendation 271 12.7 Conclusion and Future Scope of the Study 272 References 272 13 Obstacle Avoidance Simulation and Real-Time Lane Detection for AI-Based Self-Driving Car 275 B. Eshwar, Harshaditya Sheoran, Shivansh Pathak and Meena Rao 13.1 Introduction 275 13.1.1 Environment Overview 277 13.1.1.1 Simulation Overview 277 13.1.1.2 Agent Overview 278 13.1.1.3 Brain Overview 279 13.1.2 Algorithm Used 279 13.1.2.1 Markovs Decision Process (MDP) 279 13.1.2.2 Adding a Living Penalty 280 13.1.2.3 Implementing a Neural Network 280 13.2 Simulations and Results 281 13.2.1 Self-Driving Car Simulation 281 13.2.2 Real-Time Lane Detection and Obstacle Avoidance 283 13.2.3 About the Model 283 13.2.4 Preprocessing the Image/Frame 285 13.3 Conclusion 286 References 287 14 Impact of Suppliers Network on SCM of Indian Auto Industry: A Case of Maruti Suzuki India Limited 289 Ruchika Pharswan, Ashish Negi and Tridib Basak 14.1 Introduction 290 14.2 Literature Review 292 14.2.1 Prior Pandemic Automobile Industry/COVID- 19 Thump on the Automobile Sector 294 14.2.2 Maruti Suzuki India Limited (MSIL) During COVID-19 and Other Players in the Automobile Industry and How MSIL Prevailed 296 14.3 Methodology 297 14.4 Findings 298 14.4.1 Worldwide Economic Impact of the Epidemic 298 14.4.2 Effect on Global Automobile Industry 298 14.4.3 Effect on Indian Automobile Industry 301 14.4.4 Automobile Industry Scenario That Can Be Expected Post COVID-19 Recovery 306 14.5 Discussion 306 14.5.1 Competitive Dimensions 306 14.5.2 MSIL Strategies 307 14.5.3 MSIL Operations and Supply Chain Management 308 14.5.4 MSIL Suppliers Network 309 14.5.5 MSIL Manufacturing 310 14.5.5 MSIL Distributors Network 311 14.5.6 MSIL Logistics Management 312 14.6 Conclusion 312 References 312 About the Editors 315 Index 317

Written and edited by some of the world’s top experts in the field, this exciting new volume provides state-of-the-art research and latest technological breakthroughs in data wrangling, its theoretical concepts, practical applications, and tools for solving everyday problems. Data wrangling is the process of cleaning and unifying messy and complex data sets for easy access and analysis. This process typically includes manually converting and mapping data from one raw form into another format to allow for more convenient consumption and organization of the data. Data wrangling is increasingly ubiquitous at today’s top firms. Data cleaning focuses on removing inaccurate data from your data set whereas data wrangling focuses on transforming the data’s format, typically by converting “raw” data into another format more suitable for use. Data wrangling is a necessary component of any business. Data wrangling solutions are specifically designed and architected to handle diverse, complex data at any scale, including many applications, such as Datameer, Infogix, Paxata, Talend, Tamr, TMMData, and Trifacta. This book synthesizes the processes of data wrangling into a comprehensive overview, with a strong focus on recent and rapidly evolving agile analytic processes in data-driven enterprises, for businesses and other enterprises to use to find solutions for their everyday problems and practical applications. Whether for the veteran engineer, scientist, or other industry professional, this book is a must have for any library.