Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Is this new? Several years ago I wrote an Adsense-esque ad service for use by a group of entrepreneurs that wanted to promote each other's sites. I found that Google was crawling those urls even then. The text of the ads was in an HTML file, but the actual ads were served through JavaScript.


Alot of people seem to think Google only crawls content found via ANCHOR elements, but for a long time they've been able to extract the path from EMBED, SRC, and other markup elements that indicate a remote resource is being included; but that's a far cry from being able to process and execute scripting languages and understand the DOM transformations happening from AJAX requests.

In your case, I'd suspect they were simply following the src of your: <script src="path here"></script> markup... though if you read the articles cited, we suspect they've been crawling and understanding JavaScript for a pretty long time now.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: