Walmart: ChatGPT checkout converted 3x worse than website

(searchengineland.com)

90 points | by speckx 3 days ago

17 comments

  • janalsncm 1 hour ago
    This is from one of the links in the article

    > Why this is happening. Two forces are slowing agentic commerce, according to Leigh McKenzie, director of online visibility at Semrush: infrastructure and trust. Real-time catalog normalization across tens of millions of SKUs is a decade-scale problem Google already solved with Merchant Center, and consumers still default to checkout flows they trust — Apple Pay, Google Wallet, and Amazon one-click.

    It turns out when you step outside of “hard tech” problems like building GPT6 there are all of these details others have solved already. E-commerce has been optimized to the last decimal point for the last 30 years.

    OpenAI is new to it, and if I had to guess, not that interested in getting good at it.

    • petcat 1 hour ago
      > not that interested in getting good at it

      I think they're interested in getting good at it. They just don't want to put in the human time and effort to do so. They expect their many failures and short-comings to be shored up by continuous model training.

      But that, of course, means that in the meantime it will suck and nobody will use it.

      • TeMPOraL 48 minutes ago
        I think they're operating beyond their current (human) capacity, trying to test out too many things at a time.

        But a dreamer in me entertains another idea: perhaps they're just holding back, because they realize that actually succeeding at this will instantly kill (or at least mortally wound) e-commerce as we know it.

        (This is a more narrow version of my belief that general AI tools like LLMs fundamentally don't fit as additions to products, but rather subsume products, and this makes them an existential threat to the software industry. Not to software or computing, just to all the software vendors, whose job is to slice off pieces of computational universe, put them in boxes to prevent interoperability, and give each a name so it's a "product" that can be sold or rented).

        • DrewADesign 13 minutes ago
          Why do you foresee OpenAI’s involvement in the software business mitigating the resistance to interoperability and companies making money through productization? If they were actually interested in solving those problems instead of trying to secure themselves the biggest slice of economic pie, wouldn’t they have been happy about Chinese companies distilling their models? Are they engagement-juicing inn their heavily subsidized service à la Uber because they’re interested in promoting a better future for humanity? I’m skeptical.
        • latexr 33 minutes ago
          > But a dreamer in me entertains another idea: perhaps they're just holding back, because they realize that actually succeeding at this will instantly kill (or at least mortally wound) e-commerce as we know it.

          Sam Altman doesn’t give a shit about anyone but himself and has time and again shown he has no restraint for trampling over others to further his own goals. Why would e-commerce be where he draws the line?

          • TeMPOraL 22 minutes ago
            I don't think there is any line drawn here. I think if they executed well (and by they I mean any one of the three SOTA LLM vendors), they could already mortally wound the entire software industry today.

            Whether or not they want, or will want, to do it at some point, is unknown; the reasons to not do it now are obvious:

            1) it's more profitable to keep renting intelligence per token to everyone, preserving the status quo and milking it indefinitely (i.e. while the models aren't yet good enough to reliably single-shot complex software products from half-baked prompts, because once they get there, disruption will happen organically)

            2) trying to compete with ~every other software product today is not likely to succeed in the end; a serious attempt would still burn down the software industry, but the major players don't have the capacity to handle it all at once, and doing it gradually will give enough time for regulatory agencies to try and stop it; either way, no one wins

            • mrbungie 9 minutes ago
              How would they mortally wound the software industry as of today?

              I find their software to be of subpar quality and resilience anyways.

        • NicoJuicy 33 minutes ago
          > perhaps they're just holding back

          Considering the money they need, they over promise and under deliver.

    • rvz 54 minutes ago
      Also having to wait for ChatGPT for a "thinking" response to search for information that is slower than a Google search loses them lots of money.

      I believe that it can still work and I won't claim about being unsurprised about this failure. But this is a great opportunity to execute this problem really well if OpenAI and others are not interested in getting good at this.

      Perplexity also attempted this, got sued by Amazon and it appears semi-abandoned.

      The only problem is that it must be quicker or just as quick as a Google search, and also compatible with the existing checkout flows.

      • TeMPOraL 34 minutes ago
        > Perplexity also attempted this, got sued by Amazon and it appears semi-abandoned.

        Any details on that? I feel the answer is more likely there than in "friction".

        Hardly any purchase of consequence is so sensitive to friction that the difference between Google Search and an LLM response matters (especially that in reality, we're talking 20+ manual searches per one LLM response). I.e. I'm not going to use LLMs advise on some random 0-100$ purchase anyway, and losing #$ on a ##$ purchase due to suboptimal choice is not that big of a deal - but I absolutely am going to consult it (and have it compile tables and verify sources) on a $500+ purchase and for those I can afford spending few more minutes on research (or rather few hours less, compared of doing it the usual way).

  • cmiles8 8 minutes ago
    The idea that AI will suddenly solve e-commerce demonstrates a lack of understanding on everything that has happened in this space over the last 25 years.

    There’s a lot of this going on in AI at the moment. New folks come in thinking they have a magic solution and then produce a total train wreck as it turns out domain expertise is still a thing.

  • __alexs 2 hours ago
    (Good) E-commerce has been ruthlessly optimised to get shoppers to products they'll actually buy and then remove all distractions from buying.

    A chat interface is just fundamentally incompatible with this. The agent makes it too easy to ask questions and comparison shop.

    • itopaloglu83 1 hour ago
      They practically want to funnel users like cattle and not let them think about it or compare things.

      It’s like corporations are angry that they need to go through us to get our money.

      • Elfener 12 minutes ago
        > It’s like corporations are angry that they need to go through us to get our money.

        This is why I think the "you're the product" saying is wrong. You're just some annoyance to managers (whether they're trying to use you just for user numbers and ad views or they're trying to get your money), whose product is the company (shares or just outright selling the company).

      • amelius 48 minutes ago
        Users already use the internet to compare things. It makes no sense to bet on them not doing that.
      • ares623 57 minutes ago
        You can sit on your couch all day for 30 days and corporations will still be able to take your money. The marvels of frictionless payments.
    • bko 1 hour ago
      Why is this good? I want an impartial consistent system for shopping. If I can find it at a different site for a lower price, I should be able to do so. I should also be able to have it give me non-bot reviews and ask relevant questions about the product.

      The same way I think shopping at Amazon is better than a place like Nike due to objectivity and comparison, I think a chat interface has the potential to take this to another level since places like Amazon have degraded considerably in terms of things like fake third party products and fake reviews.

      • __alexs 45 minutes ago
        The buyer of this technology is not shoppers, it's retailers. The measurement of quality is "does it make us more money?" not "does it help me make better buying choices."

        Retailers do not want you to make better choices. They want you to buy the widget.

        A lot of evidence suggests that also shoppers aren't that interested in making the best choice either. They want to make a tolerable choice with as little effort as possible. There is no basically no consumer market for "power shopping" outside of weird niches like pcpartpicker.com etc.

      • throwaway290 1 hour ago
        > impartial consistent system for shopping

        > for a lower price

        Catalog is impartial, chatbot is ads pretending as advice.

        • danlitt 39 minutes ago
          I do agree with your conclusion, but the catalog in most online shops is certainly not impartial. Amazon sells the entire first page of search placement, for example.
        • TeMPOraL 46 minutes ago
          Catalog is an ad, the SKU database behind the catalog is impartial (at least as much as it gets), but no one is giving you access to that.
    • TeMPOraL 1 hour ago
      > (Good) E-commerce has been ruthlessly optimised to get shoppers to products they'll actually buy and then remove all distractions from buying.

      The only e-commerce site that fits this standard is that old one for buying (IIRC) nuts and bolts or such, that pops up on HN every other year, and whose name sadly escapes me now. Everyone else is ruthlessly optimizing their experience to fuck shoppers over and get them to products the vendor wants them to buy, not the products the shoppers actually want (or need).

      > A chat interface is just fundamentally incompatible with this. The agent makes it too easy to ask questions and comparison shop.

      That is precisely the point.

      Chats may suck as an interface, but majority of the value and promise of end-user automation (and more than half the point of the term "User Agent" (as in, e.g., a web browser)) is in enabling comparison shopping in spite of the merchants, and more generally, helping people reduce information asymmetry that's intertwined with wealth and power asymmetry.

      But it's not something you can generally sell to the vendors, who benefit from that asymmetry relative to their clients (in fact, I was dumbfounded to see so much interest on the sales/vendor side for such ideas, but I blame it on general AI hype).

      Adversarial interoperability is the name of the game.

      • busymom0 1 hour ago
        • bandrami 35 minutes ago
          Just made an order from them. It's weirdly comforting to know there's a company that knows I need clevis bolts and is willing to sell them to me for a transparent price.
        • jaapz 35 minutes ago
          Wow, that's amazing
        • TeMPOraL 45 minutes ago
          It's the one, thank you!
      • froggit 37 minutes ago
        Not sure you're aware but you initially sound like you disagree with the post you replied to, only to follow up by enthusiastically reiterating that author's words as if in agreement.

        You realize what shoppers and vendors each consider to be "good" e-commerce sites are fundamentally opposed concepts?

        • TeMPOraL 31 minutes ago
          Maybe? I'm not sure which way the OP is arguing, in particular because of that "(Good)". So perhaps I misread the comment as arguing the opposite of what it is.
    • exegete 1 hour ago
      The article says Walmart is not abandoning ChatGPT but is going to use their own app in Chat’s ecosystem
    • locknitpicker 1 hour ago
      > (Good) E-commerce has been ruthlessly optimised to get shoppers to products they'll actually buy and then remove all distractions from buying.

      I don't think so. I know for a fact that search terms are a minefield of gotchas and hacks caused by product decisions that reflect ad-hoc negotiations with partners and sellers. It's an unstable equilibrium of partners trying to shift attention to their products in a certain way. I think that calling this fragile equilibrium optimized has no bearing with reality.

    • TZubiri 1 hour ago
      It depends on the product, if we are talking commodities or mass produced products like groceries, sure.

      If we are talking custom products or complex appliances that need a lot of guidance, then maybe chat interface is appropriate.

      • bashkiddie 1 hour ago
        When I shop for special hardware (e.g. bicycle shift gear) it is usually underspecified. If the information does not exist in the text block, a chat bot is of no use.
        • TeMPOraL 41 minutes ago
          Chat bots don't belong to an e-commerce site; chat bots belong on the outside, specifically to comparison-shop and pull in some external information to de-bullshitify offers, correct "mistakes" and "accidental omissions" in the listings, resolve the borderline-fraudlent crap companies play these days with store-specific and season/promotion-specific SKUs with different parameters all resolving to same model/make name (think Black Friday/Cyber Monday deals that are not actually deals, just inferior hardware with dedicated SKU).
        • TZubiri 45 minutes ago
          The use case for chat interfaces would be as follows:

          Grandma wants to buy a good bike, but doesn't know about types of wheels or how many gears they need, or what type of frame is appropriate for their body type.

          • TeMPOraL 40 minutes ago
            Reliable information on this does not exist on vendor sites, though. It exists on Reddit and in books and in med/physio papers and bunch of other places a SOTA model has read in training or can (for now) access via web search.

            LLMs are already very good for shopping, but only as long as they sit on the outside.

      • pjc50 1 hour ago
        The problem is that the chat transcript is legally binding. If the chatbot makes incorrect statements which the customer relies on for their complex purchase, then you're going to have to refund them.

        https://www.bbc.co.uk/travel/article/20240222-air-canada-cha...

  • Lerc 2 hours ago
    Is their issue that ChatGPT served their customers more than it served them?
    • Vespasian 1 hour ago
      To early to say that but it's certainly a part of the equation all vendors are currently looking at.

      And given the past few decades there is no reason to not try to do that.

    • d--b 1 hour ago
      That's definitely it.

      This probably means that OpenAI et al are fine-tuning salesman-like LLMs to "fix" that problem.

      Can't wait for the future.

  • asimpletune 21 minutes ago
    Isn’t there already a much older rule that predicts this?

    Your product has to be a 10x improvement over the incumbent to be competitive.

    In AI speak it would be the “extra-bitter” lesson I guess?

    You need to add 10x resources to beat a product that’s already solved with mature tech.

  • qoez 48 minutes ago
    It's probably stuff like this along with investor pressure that will make AI companies slowly make their AIs more profit maximizing (and the long term reason ilya etc was so against even going down that path)
  • hexasquid 2 hours ago
    Last year they couldn't draw fingers on hands properly, this year they can't convert at checkouts, I wonder what they'll be failing to do a year from now.
  • keiferski 1 hour ago
    I don’t trust AI bots to access my wallet. Not sure I ever will.

    I sort of trust them to make product recommendations, but at best I will only open a link they suggest and buy the product there.

    • falcor84 46 minutes ago
      > my wallet

      Does it actually need direct access to your wallet? I haven't tried it yet, but assumed it would work with a separate wallet, fed through by top-ups.

      • keiferski 42 minutes ago
        Sounds like they are an intermediary between the user and the business. So it’s not a top-up thing. That would be needlessly complex from ChatGPT’s end, too.

        https://chatgpt.com/merchants/

    • everdrive 1 hour ago
      >I sort of trust them to make product recommendations

      Never, ever.

      • raincole 1 hour ago
        Your argument is "they're designed to influence us" right?

        Amazon reviews are paid influence. Reddit posts are paid influence. Everything everywhere you read online is paid influence. I'd rank LLMs between "people I personally trust" and "random people online."

        • vulcan01 41 minutes ago
          Since LLMs are trained on "random people online", why are they not of equal rank?
          • TeMPOraL 20 minutes ago
            For me, for now, they are. And being "many random people" and not "random person", they average out into something much more trustworthy than even recommendations from most individuals I know personally.

            Operative word is "for now" - LLMs caught entrepreneurs unprepared, but they'll catch up and poison this too, same thing that happened with search giving rise to SEO.

      • keiferski 1 hour ago
        I trust them as much as any other online source, which is to say, sort of, but only as a starting point for research.

        Do you have a better alternative?

      • crooked-v 57 minutes ago
        Even if they're (somehow) bias-free, you're still stuck with "the state of the internet circa 20XX" from the training data.
  • firefoxd 2 hours ago
    The experience is a lot like when you are talking with a friend, then they decide to ask siri or google a question using voice. The result is always imprecise. Meaning they either have to repeat their query, or end up typing it anyway.

    If you want to buy a Walmart product, the easiest way is to go to Walmart. Why add an imprecise middle man in between?

  • kvisner 3 days ago
    That doesn't seem terribly surprising, a human can quickly look through a grid of shirts to find one they like. ChatGPT would be guessing what they might want and the human would probably get a bad experience there with some regularity.
  • zmmmmm 1 hour ago
    > Walmart will embed its own chatbot, Sparky, inside ChatGPT. Users will log into Walmart, sync carts across platforms, and complete purchases within Walmart’s system.

    The enshittification is upon us.

    • netsharc 49 minutes ago
      Hah, Clippy's cousin Sparky: every once in a while after ChatGPT answers a question it'll say "Looks like you still have stuff in your WalMart cart. Would you like me to complete that checkout for you? Also, WalMart-brand diapers is on offer this week, shall I add that to your cart?"
  • brador 35 minutes ago
    Original AI was sourced from university level text books, stack, wiki. Average iq around 140.

    The latest AI is trained on the average citizens social media output. Iq 90.

    That’s why AI seemed smart. The bar will not be raised again. We’re cooked.

  • fennecfoxy 1 hour ago
    Not really many details...

    Perhaps clickthrough is worse because there are fewer dark patterns involved and people are mostly just browsing and occasionally buying only what they need.

    They didn't really seem to specify the "why" of it with any research. And weird that OAI wasn't supporting them to see wha the issue was.

  • casey2 2 hours ago
    Because most people can't read. Wait for agents generating personalized websites/self checkout apps.
  • charcircuit 2 hours ago
    What if they made instant checkout actually instant? You ask ChatGPT to setup a website and it instantly purchases web hosting and sets up the website there. You can't beat a 100% conversion rate by actually checking out instantly. If you didn't like that host you can ask it to find it alternative and ChatGPT would automatically attempt a refund and then purchase from someone else.
    • LoganDark 2 hours ago
      I'd become afraid to ask that bot anything, not knowing what it would automatically try to purchase for me. Paying for the bot itself is already a lot: $20/mo to $200/mo or more.
      • lukan 1 hour ago
        20$ for a personal assistant is not much, but no, I surely don't trust those assistants to access my money.
    • TZubiri 1 hour ago
      What are we talking about here? ChatGPT as an interface to buy groceries? Or ChatGPT as an interface to build a website. I fail to see how these would be related other than using a specific technology
      • wrqvrwvq 34 minutes ago
        > Because most people can't read. Wait for agents generating personalized websites/self checkout apps.

        > You ask ChatGPT to setup a website and it instantly purchases web hosting and sets up the website

        Multiple comments deflecting from the original shopping conversion failure to recommend ... building a whole new website (with hosting for some reason?). W/o bothering to look through commenter history, one has to assume there are a lot of chatbots on this site or else the people using this stuff have been lobotomized.

        I'm sure it'll start happening too, and when that fails, the bots will, i don't know, invent a new macarena. We are definitely headed for an irredeemably stupid future.

  • FlowPagesVael 7 minutes ago
    [dead]