Unable to make my script keep clicking on a button

asmitu

New Member
#1
I've created a script using node.js in association with puppeteer to click on more button located at the bottom of a webpage to dig out all the headlines from it's landing page.
The thing is when I execute my script, It only clicks once and then quits. How can I keep clicking on that button until there is no more button to be clicked, meaning all the links are revealed?
My attempt so far:
Mã:
const puppeteer = require("puppeteer");

(async function main() {
    const browser = await puppeteer.launch({headless:false});
    const page = await browser.newPage();
    await page.goto("https://www.newsnow.co.uk/h/Sport/Football/Championship/Transfer+News", {waitUntil: 'networkidle2'});
    await page.waitForSelector("[class^='hl_more']");
    await page.click("[class^='hl_more']");
    await browser.close();
})();
As I'm new to node.js and puppeteer, I can't understand as to how I can define a loop to accomplish the task.
 

Admin

Administrator
Thành viên BQT
#2
What I often do in situations like this is use a try catch block to check for an elements visibility using waitForSelector and using a very short timeout. You need the try catch block because waitForSelector will eventually timeout when that more button is no longer visible. This is also why you will need to use a short, specific timeout as you don't want your code to pause for 30 seconds (the default timeout) while trying to find the component.

So what I would do is this:

Mã:
// This method checks if an element is visible and times out cleanly after 2 seconds if it is not displayed
const isElementVisible = async (page, cssSelector) => {
  try {
    await page.waitForSelector(cssSelector, { timeout: 2000 });
  } catch () {
    return false;
  }
  return true;
};

(async function main() {
  const browser = await puppeteer.launch({headless:false});
  const page = await browser.newPage();
  await page.goto("https://www.newsnow.co.uk/h/Sport/Football/Championship/Transfer+News", {waitUntil: 'networkidle2'});
  let moreDisplayed = await isElementVisible(page, '[class^="hl_more"]');
  while (moreDisplayed) {
    await page.click('[class^="hl_more"]');
    moreDisplayed = await isElementVisible(page, '[class^="hl_more"]');
  }
  await browser.close();
})();
 
Top