Accurate & updated product specifications data for powersports superstore enhances conversions

Challenges.

Data source identification: Data and images to be extracted were spread across different manufacturers’ websites in different page layouts. Collating this diverse data was a complex process.

Unstructured data: Name of spare parts and aftermarket parts were displayed in different formats on different websites. Client was required to spend more time to manually inspect, identify and collect data and images for the correct product.

Photo retouching: Background watermarks on spare part and accessories would render photos of no use. Removing watermarks and redesigning spare part diagrams in a different layout required a team of expert auto ancillary photo editors.

Solution.

Ongoing product update on the client portal with product specifications and images was delivered backed by an automated workflow which accounted for seamless web extraction, validation and data enrichment activities. Adobe Photoshop, Adobe Illustrator, and Lightroom were leveraged to make images ready to be published with relevant product specifications.

Approach.

Designed a web scraping tool using bots, macros and scripts to collect data requested by client from manufacturer’s websites. Photos of accessories and spare parts were edited and retouched manually.

Data Acquisition

• Identification of relevant manufacturer websites as key data sources after detailed discussion with the client.
• Programmable bots were leveraged, to scrape relevant field information from manufacturer’s websites into specific data fields in Excel. Data thus collected was used to add new information, append and update existing product database.
• Similar automation tool was used to download product images. The photos then were edited and retouched by experts to make them look alike the one on manufacturer’s website.

Data Preparation

• Data was collected in an Excel template through an automated web scraping procedure.
• Validation rules were scripted and used to ensure only relevant data is scraped and keyed into the database.
• Nomenclature for product photos was defined to ensure relevant tagging of photos to product information.

Image Redrawing

• Photo editing and retouching experts removed the watermark and redrew each image to make it look entirely different from the one on manufacturer’s website. Adobe Photoshop was used.

Quality Check and Audit

• ML backed workflows identified and auto-created data extraction patterns to expedite the web scraping process.
• Bots were used to validate and verify extracted information which improved quality of product database on client’s portal.
• Multi-layered QC process ensured that product data, before being uploaded on client’s portal, was 100% accurate and according to client requirement.
• Reports/dashboards for product information gathered and dispatched were generated and shared with client regularly.

Dispatch

• Automated upload of product data files and redrawn product images on designated FTP servers.

Software and Technology: Custom web scraping tools, Macros, Bots, Scripts and Adobe Photoshop