By KI: Amazon app is looking for and ordered in foreign online shops

If a product searched for is not available on Amazon, customers should not have to buy somewhere else in the future: A AI should search for you in foreign online shops and, if desired, also order there on your name. “Buy for me”, in German “Buy for me”, is the name of the new AI function, which is now available as a beta version for some US customers from Amazon.

If you are looking for products in the Amazon shopping app, you will also receive offers that have foreign online shops in your range. They can be called up and viewed exactly in the Amazon app, such as products that are offered directly on Amazon. A “AI shopping agent” orders on behalf of the Amazon users from the external provider. Customers have to do nothing more than give up their order in the Amazon app as usual. In one Blog entry shows Amazon the course of such an order on the smartphone.

Amazon-Ki serves web browser

Behind the scenes, the purchasing agent visits the external website, selects the desired product, gives the name, the delivery address and the user’s payment data in encrypted form and completes the purchase, explains Amazon. The agent therefore runs with Amazon’s in-house Nova Kis and the Claude-KIS from Anthropic. Nova Act should be behind the new function, a offshoot that Amazon presented this week. He can control web browser and has already mastered simple work steps on external websites, reports the tech magazine techcrunch. Nova Act is already for developers Accessible to Amazon.

However, “Buy for Me” should also cause some concerns. Because in the end it remains a AI that gives up orders on behalf of Amazon users and processes sensitive data such as bank details, home addresses, etc. And as is well known, KIS tend to hallucinate and are susceptible to mistakes. For example, the former leads to apparently correct results of the AI, which can be objectively wrong and cannot be plausibly attributed to the training data used. In the case of Amazon’s purchasing agents, the AI ​​could suddenly turn a product from ten ordered pieces of a product. Techcrunch also reportsthat AI shopping agent has often taken for a very long time to process orders, and that often fails. Openai and Google also offer similar “purchasing agents” in which the users still have to enter their data themselves.

And some customers should also worry about the safety of their data. Again and again jailbreak methods (German: prison outbreak methods) are known again and again. This can be overridden from the safety guidelines of the KIS, so that they suddenly reveal sensitive data. Heise online recently made the Claude-KI to talk about construction instructions for Molotov cocktails and other things. The answers were only partially hallucinated. Later, Claude also gave a list of identities from his developers, which were also not hallucinated. In normal condition, Claude refuses such inquiries. Chatbots are susceptible to jailbreaks behind which Large Language Models (LLMS) are. So far, it is of course questionable whether data from Amazon’s AI shopping agents would be read out in this or similar way. The question is also exciting whether Amazon excludes such foreign shops from his shopping agent from the outset, and in turn processed the customer data that he secures securely – it would not be excluded in the theoretically, the order would be abandoned via Nova Act via a (virtual) web browser. It is shown whether such possible risks for Amazon users provide reservations and whether they are entitled.