Reddit says Microsoft’s Bing, Anthropic, and Perplexity have scraped its data without permission. “It has been a real pain in the ass to block these companies.”
Reddit says Microsoft’s Bing, Anthropic, and Perplexity have scraped its data without permission. “It has been a real pain in the ass to block these companies.”
Beyond that, if you are serving webpages with data on them, you don’t get to decide what people do with those pages. They can’t stop search engines from scraping
Just to nitpick, they can stop scraping, anyone can. However, doing so would require implementing barriers that tend to also negatively effect sites that are dependent on being discovered and browsed.
you can decide who you serve pages to