Open a link in a new tab, scrape, go to previous page

I’m using puppeteer for the following:

I switched await link.click(".ExCategory-results > .ExResult-row:nth-child(${i}) > .ExResult-cell > .ExHeading > a",); for await new.page('...') but it says that it can’t find the a.

This is the page that I’m scraping but notice the Load More button at the bottom of the page.

https://www.bodybuilding.com/exercises/finder

To prevent resetting the Load more button I want to open each new in a new tab, scrape, close tab and go to the next name.

How can I open each link in a new tab, close, and go to the previous tab?

My code:

var buttonExists = true;
let allData = [];
while (buttonExists == true) {
// const loadMore = true;
const rowsCounts = await page.$$eval(
    '.ExCategory-results > .ExResult-row',
    (rows) => rows.length
);
console.log(`row counts = ${rowsCounts}`);

for (let i = 2; i < rowsCounts + 1; i++) {
    const exerciseName = await page.$eval(
        `.ExCategory-results > .ExResult-row:nth-child(${i}) > .ExResult-cell > .ExHeading > a`,
        (el) => el.innerText
    );
    console.log(`Exercise = ${exerciseName}`);

    await link.click(`.ExCategory-results > .ExResult-row:nth-child(${i}) > .ExResult-cell > .ExHeading > a`,);
    await page.waitForSelector('#js-ex-content');

      ... fancy code here

    await page.goBack();

    let obj = {
        exercise: exerciseName,
    };

    allData.push(obj);

}
// clicking load more button and waiting 1sec
try {
    await page.click(LoadMoreButton);
}
catch (err) {
    buttonExists = false;
}
await page.waitForTimeout(1000);
}

69 thoughts on “Open a link in a new tab, scrape, go to previous page”

Leave a Comment