Global Healthcare Logistics Firm

The Challenge

The client is implementing an enterprise solution to overhaul their supply chain and logistics processes, which will be used by staff and clients worldwide. This solution replaces multiple legacy systems. One critical aspect of this implementation is migrating a substantial volume of data from the legacy systems to the new platform. The data is unstructured and presents challenges such as redundancies and merging information from various sources, including paper-based systems.

The Solution

We implemented a systematic approach to streamline the data management process, resulting in significant benefits:

1.Data Collection and Standardization:

• Collecting data from diverse sources, including digital databases and paper-based systems.
• Achieving standardization by identifying common traits across datasets. 

2.Data Cleansing and Advanced Data Processing:

• Rigorously cleansing data to rectify errors, such as city names, shipping methods, currency, and address details.
• Eliminating duplicate records, reducing manual effort.
• Identifying duplicate entries based on shipping methods and prices to ensure consistency in pricing and shipment conditions.
• Incorporating transit times and direct drive conditions to establish a unified pricing structure.

3.Machine Learning for Table Identification:

• Deploying machine learning techniques to identify table header data within data files.
• Enhancing data accuracy by precisely determining the starting point of information tables

4.Error Logging and Automation:

• Utilizing Python scripts to automate condition checks for all data within files.
• Meticulously logging error messages for each specific file, facilitating quick resolution.

5.Unstructured Data Handling:

• Employing Named Entity Recognition (NER) to extract critical information using key-value pairs for unstructured data.

The Outcome

By leveraging AI/ML and data analysis techniques, BTC successfully automated and streamlined its data analysis and migration processes for our client. This solution not only handled a significant volume of data but also ensured accuracy and consistency in handling diverse data formats. The automation of error tracking improved efficiency, making the client’s data analysis processes more robust and reliable. All this was achieved within record time to meet the organization-wide roll-out plan.

Tech Stack:

• Python for automation and scripting.
• Machine learning techniques for data table identification.
• Geopandas, Pandas, and NumPy for data manipulation.