Accidentally sending imperfect items to customers can result in unwanted returns, so Amazon is using technology to identify damaged products before shipping them to buyers, it announced this week. “Using a combination of generative AI and computer vision technologies, Project P.I. (“private investigator”) is able to uncover defects, like damaged products or issues like wrong color or size, before products reach customers.”
Project P.I. has been in place at several fulfillment centers in North America since May 2022 and Amazon will expand the technology to additional sites throughout 2024. Amazon explained how it worked:
“Before an item ships to a customer, it travels through an imaging tunnel, where Project P.I. uses computer vision to scan the product and evaluate the images to detect any defects, like a bent book cover. If a defect is found, Amazon isolates the product so it is not shipped to a customer, and investigates further to determine if there is a wider issue with similar items.
“Amazon associates—who review the items Project P.I. flags—then decide whether the item is eligible to be resold at a discounted price as part of Amazon’s Second Chance site, donate it, or find another use for it.”
Amazon is also trying to “learn” from the system to prevent problems from happening in the first place.
“In parallel, Amazon teams are leveraging a generative AI system that uses a Multi-Modal LLM (MLLM) to investigate the root cause of negative customer experiences. When we learn of a defect from the customer that we missed to identify, we use that to understand the cause and continuously improve the system. The system first reviews customer feedback and then analyzes images taken from Project P.I. in fulfillment centers and other data sources to confirm what led to the problem.”
Amazon said the new procedure would help its third-party sellers for whom it fulfills product through its FBA fulfillment service. “This same technology is poised to help Amazon’s selling partners by making data on defects more easily accessible. For example, if a selling partner accidentally put stickers with the wrong size on a product, Amazon would communicate the issue to help prevent the error from happening again,” it said.
Amazon acknowledged shipping carriers could also be the cause of defects. It delved into the technology behind the new practice in an accompanying blog post, showing how it can check for expiration dates before sending products to buyers:
“For example, optical character recognition (OCR) — the process that converts an image of text into a machine-readable text format — checks expiration dates on product packaging to ensure expired items are not sent to customers. Computer vision (CV) models — trained with reference images from the product catalog and actual images of products sent to customers — pore over color and monochrome images for signs of product damage such as bent book covers.”
Amazon explained it would also use customer feedback to identify product quality issues: in instances where a buyer notes an issue when requesting a return, the company will track down the originating batch to find remaining products in its fulfillment centers, verify the issue, remove the items, issue refunds, and inform the seller. It did not elaborate on how it would compensate sellers for damage done to FBA products after it received them at its fulfillment centers.
Amazon is its own worst enemy. I’ve been shipping books for over 40 years. There is a right way to do it and a wrong way. If I ordered a book on Amazon 15 or 20 years ago, they would place the book on an oversize piece of cardboard and shrink-wrap it. Nowadays they just throw the book into a box, regardless if it is a $20 book or a $95 book. About 25% of the time the book arrives with a corner bumped. And it gets returned. Amazon should have come up with a better way to protect book corners. It’s not rocket science.