New York is set to become the first state to impose guardrails on “stealth crawlers,” or unauthorized software that trawls news sources to scrape content.
I hope those tools are exempt because, just like a browser, they respond only to specific commands issued by a human user. They don’t “crawl” pages in the way we describe bots that jump from page to page.
I hope those tools are exempt because, just like a browser, they respond only to specific commands issued by a human user. They don’t “crawl” pages in the way we describe bots that jump from page to page.