Read data from url in javascript

Question

i'm trying to read data stored in google drive, on drive or other cloud storage using javascript `

let btn = document.getElementById('action-btn');



btn.addEventListener('click', () => {
    // let baseurl = document.getElementById('inputurl').value;
    // let guidDcoument = baseurl.split('#guid=')[1];
    // const query = encodeURIComponent('Select *')
    // fetch('', {
    //     mode: "no-cors",
    //     method: "get",
    // }).then(Response => console.log(Response))
    fetch('https://docs.google.com/spreadsheets/d/1kwfv6L2lBrPw8OjHGyhO7YHOXFNwHYyPI_noM5TUMLw/edit?pli=1#gid=1097732605',{
        mode:"no-cors"
    })
    .then((response) => { console.log(Response.error); })
    .catch(console.error())
})

` What I need is that given a url, I can read the data from the file, and then show them on my website.

I think that when I try to access any other cloud storage I will have the same problem. That it will be necessary to access the account to be able to read the data that would be a csv document.

What is Response.error and why do you use it in the handler? — Konrad
– Konrad, Commented Oct 31, 2022 at 18:45
Yes, you will need to authenticate yourself before accessing a Google Drive file, and you can't get the file content using fetch. You will need to use the dedicated API for that. — Karthik Sivasubramaniam
– Karthik Sivasubramaniam, Commented Oct 31, 2022 at 18:47

Franco Torres · Accepted Answer · 2022-10-31 18:55:20Z

1

First of all, what you're trying to achieve is called 'web scraping' and you can't use fetch for that, instead you should use puppeteer (in the server side), which is the most popular library for web scraping.

Run npm init -y to initialize a npm project and install puppeteer npm i puppeteer and also install express and cors npm i express cors in order to create a server that scraps data and sets it back to your client. So, instead of trying to scrap the information directly from the client you do it from the server with puppeteer.

Try the following .js server code:

const express = require('express')
const cors = require('cors')
const puppeteer = require('puppeteer')
const app = express()
app.use(express.json())
app.use(cors())

;(async () => {
  const browser = await puppeteer.launch({headless: false})
  const page = await browser.newPage()

  app.use('/', async (req, res) => {
    await page.goto('url-to-the-website')
    const data = {}
    return res.json(data)
  })
})()

app.listen(3000)

And learn more about puppeteer: https://pptr.dev/.

And your client code should connect to this server to send scrape requests to it like this:

(async () => {
  const res = await fetch('http://localhost:3000')
  const rawjson = await res.json()
  const json = JSON.parse(rawjson)
  console.log(json)
})

Note: we wrap the code in anonymous functions with the reserved word async in order to use async and await syntax. More information: https://javascript.info/async-await.

answered Oct 31, 2022 at 18:55

Franco Torres

1,1184 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

carlos eduardo grageda flores Over a year ago

Thank you very much for taking the time to answer your question, your explanation has helped me a lot, I will try it.

Collectives™ on Stack Overflow

Read data from url in javascript

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related