In a recent interview, Steve Huffman, Reddit CEO, called out Microsoft’s Bing, Anthropic, and Perplexity for raking Reddit’s data without permission and asked them to pay if they wanted to continue scraping data.
He said, “Without these agreements, we don’t have any say or knowledge of how our data is displayed and what it’s used for, which has put us in a position now of blocking folks who haven’t been willing to come to terms with how we’d like our data to be used or not used.”
Huffman added that it’s been “a real pain in the a** to block these companies.”
At the beginning of July, Reddit updated its robots.txt file to block web crawlers not in agreement with it. However, due to this, the Reddit results were only visible on Google results, where it pays for data visibility.
Huffman revealed that Microsoft has been using Reddit’s data to train its AI, summarize content in Bing results, and sell that content to other search engines without permission.
“We’ve had Microsoft, Anthropic, and Perplexity act as though all of the content on the internet is free for them to use.” He added, “That’s their real position.”
In reply, Microsoft’s head of search, Jordi Ribas, said on X, “Reddit has blocked Bing from crawling their site for search, favoring another search engine and impacting competition from Bing and Bing-powered engines.”
He revealed that OpenAI’s SearchGPT will show Reddit results thanks to the deal that happened earlier this year.
Huffman commented on the licensing deals, saying, “I think the traditional value exchange from search engines has changed. Search and summarization and training are merging, and the value exchange of crawling in exchange for traffic back is becoming muddied.”
Following this report, Anthropic has given a reply, while Microsoft and Propensity refused to comment.
