Scraping HTML and javascript websites in Python

I’m trying to scrap text data in a list of URLs. In most cases, the required text is under p tags of HTML, and I can extract them. But in some sites, the required text is under javascript code. How can I identify the URLs that have their text generated from javascript to handle them separately?

One of the sites I’m having trouble with has the code written like this.

How to identify the URLs whose text is generated from javascript?

20 thoughts on “Scraping HTML and javascript websites in Python”

Leave a Comment