Getting API Data and writing to MongoDB using NodeJS

Question

Im learning NodeJS and trying to put up a mock website locally. Basically, its a website that uses the WOW AuctionHouse API. I take the data from the request, parse it to JSON and attempt to write it to the DB. The problem is, if i put the auctions.save inside the request it doesnt recognize the .save function as a function (assuming im out of the scope for it) and if i place it outside the problem comes from the async nature and i get an empty entry in my DB. Here is the code:

var express = require('express');
var hbs = require('express-hbs');
var mongoose = require('mongoose');
//var blizzard = require('blizzard.js').initialize({ apikey: "xxx" });

mongoose.connect('mongodb://127.0.0.1/blood_of_sargeras')

var post = mongoose.model(
  'post',
  {
    name: String,
    type: String,
    price: String
  });

var app = express();

app.engine('hbs', hbs.express4({
  partialsDir:__dirname+'/views/partials'
}));

app.set('view engine', 'hbs');
app.set('views', __dirname + '/views');

app.use('/static', express.static('static'));

var request = require("request");
var url = "http://auction-api-eu.worldofwarcraft.com/auction-data/xxx-xxx/auctions.json"
var auctions = new post;

request({
    url: url,
    json: true
}, function (error, response, body) {

    if (!error && response.statusCode === 200) {
        auctions = JSON.stringify(body);
        post.save(function (err) {
          if(err)
            console.log("Error with database!");
          else
            console.log("Data written to DB!");

        })
    }
});

app.get('/', function (req, res){

  post.find({}, function (err, o){
    if (err)
      console.log('Database  error!');
    else{
      res.render('index', o);
    }
  });
});

app.listen(80, function(){
  console.log('Listening on port 80')
});

Is this even the right way to do it? Is there a better way? I've been stuck on this for quite a while and its gotten extremely frustrating. Other things i tried are:

Timeout function outside the request so it waits for the whole thing to be copied, but that doesnt work i get the same issue as trying it inside, it says .save is not a function
Tried calling a function defined on the outtermost scope, while inside the request but that also tells me .save is not a function.

Try this: var postSchema = mongoose.Schema({name: String,mType: String,price: String}); var post = mongoose.model('Post',postSchema); var newPost = new Kitten(); newPost.name = newPost.mType = //do not use type it is reserved newPost.price = newPost.save(function(err, mPost) { if (err) console.error(err); console.log(mPost); }); — Matt Kuhns
– Matt Kuhns, Commented Mar 7, 2017 at 20:26
Im not sure what this is suposed to do, but i get the exact same error: newPost.price = newPost.save(function(err, mPost){ ^ TypeError: newPost.save is not a function Also, as far as ive figured it out its a problem with the scope of the mongoose(where i can use it) — Dragan
– Dragan, Commented Mar 7, 2017 at 20:41

Simon Mo · Accepted Answer · 2017-03-07 20:24:16Z

1

I think you might need to wrap your request in an event listener. This will ensure the request runs once the server has finished starting up rather than before. Here is a quick example.

app.on('listening', function () {
    // Run your request to the WoW auction house  once the express server has started up
    request({
        url: url,
        json: true
    }, function (error, response, body) {

        if (!error && response.statusCode === 200) {
            auctions = JSON.stringify(body);
            post.save(function (err) {
                if(err)
                    console.log("Error with database!");
                else
                    console.log("Data written to DB!");

            })
        }
    });
});

answered Mar 7, 2017 at 20:24

Simon Mo

7733 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Dragan Over a year ago

Same error as the whole time, auctions.save is not a function (it works if i try it outside of any function) Edit: Sorry, actually nothing happens. None of the code gets executed(no console logs, nothing)

Simon Mo Over a year ago

I've edited your code and this should work. You will need to decide how you want to save the data to the database as I can see that the auctions.json file is quite big so you might want to loop through it. I have created a gist for you here gist.github.com/Fried-Chicken/7955f671b8ac9da08328752335d13b3c

Dragan Over a year ago

Thats great and it works, but another problem pops up. auctions.insert inserts a useless value, not the value i get in the body variable. If i try body.save it says its not a function and ive tried with json.parse and json.stringify and its the same problem, i think it stops treating auctions as a 'post' object.

Simon Mo Over a year ago

That is actually the expected behavior because I did not attempt to save your object to the database as I am not sure how you want to split the data up since the autions.json object is really big. You need to either pass the JSON.stringify(body); object into the new post when you initiate it, or use mongoose's create method.

Dragan Over a year ago

// auction = new post (JSON.parse(body));           //auction = new post ( "'" )           //console.log(auction);           // auction = JSON.stringify(body);           // console.log(body);           // auction = JSON.parse('"' + body + '"');           //auction.insertMany(body);           // auction.body = body;           // auction = new post (body.auctions);           //auction = new post (JSON.stringify(body));

These are all the methods ive tried. None of them work

Matt Kuhns · Accepted Answer · 2017-03-07 20:29:01Z

0

Try this:

var postSchema = mongoose.Schema({name: String,mType: String,price: String});
var post = mongoose.model('Post',postSchema);

var newPost = new Kitten();

Then later in request:

newPost.name = auctions.name;
newPost.mType = auctions.type;    //do not use type it is reserved
newPost.price = auctions.price

newPost.save(function(err, mPost) {
    if (err) console.error(err);
    console.log(mPost);
});

answered Mar 7, 2017 at 20:29

Matt Kuhns

1,3782 gold badges13 silver badges30 bronze badges

Collectives™ on Stack Overflow

Getting API Data and writing to MongoDB using NodeJS

2 Answers 2

5 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related