I know is a noob question, but I want to know when I should use page.evaluate
I also know the documentation exists, but I still do not understand
Can anybody give me an explanation about how and when to use this function when creating a scraper with puppeteer?
First, it is important to understand that there are two main environments:
You should use page.evaluate() when you are seeking to interact with the page directly in the page DOM environment by passing a function and returning a <Promise<Serializable>> which resolves to the return value of the passed function.
Otherwise, if you do not use page.evaluate(), you will be dealing with elements as an ElementHandle object in the Node.js (Puppeteer) environment.
Example Usage:
const example = await page.evaluate(() => {
const elements = document.getElementsByClassName('example');
const result = [];
document.title = 'New Title';
for (let i = 0; i < elements.length; i++) {
result.push(elements[i].textContent);
}
return JSON.stringify(result);
});
See the simplified diagram below:
page.evaluate is evaluated inside the Page DOM environment, essentially as if you'd opened a developer tools console and run the code from there. There, document is available - but if you tried to reach outside and use something else from the rest of the Puppeteer script, it wouldn't be there.