*But, have Yahoo or Bing or DuckDuckGo made the transition to be able to crawl t... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

workhere-io on May 27, 2014 | parent | context | favorite | on: Googlebot's recent improvements might revolutioniz...

But, have Yahoo or Bing or DuckDuckGo made the transition to be able to crawl the web with a full JS & DOM rendering engine?

They can just use PhantomJS (http://phantomjs.org/), which is free and open source.

voltagex_ on May 28, 2014 [–]

They could, but I wonder what it'd take to scale it to crawling that number of pages.

I think only Bing would have the cash and resources to build that.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact