How can I dynamically generate test cases in javascript/node?

Question

The nose testing framework (for python) supports dynamically generating test cases at run-time (the following, from the documentation, results in five distinct test cases):

def test_evens():
    for i in range(0, 5):
        yield check_even, i, i*3

def check_even(n, nn):
    assert n % 2 == 0 or nn % 2 == 0

How can I achieve this result using javascript frameworks such as mocha or qunit? (I am not attached to any particular framework at this point.)

My use-case is writing a test runner to monitor several items on an external server. I would provide a list of resource URLs. Each test attempts to poll that resource and returns success or failure depending on what it finds. I have a prototype built in python (using nose) but would like to implement in node.js if I can. Eventually, this would be included in a CI setup.

node as in node.js? Maybe you should tag it, since just javascript will be interpreted as meaning javascript in browser. — developerwjk
– developerwjk, Commented Mar 17, 2014 at 23:04

ZachB · Accepted Answer · 2017-11-08 22:48:10Z

42

Yes you can dynamically created test suites with cases using Mocha. I have installed mocha globally npm install -g mocha and I use should.

var should = require('should');

var foo = 'bar';

['nl', 'fr', 'de'].forEach(function(arrElement) {
  describe(arrElement + ' suite', function() {
    it('This thing should behave like this', function(done) {
      foo.should.be.a.String();
      done();
    });
    it('That thing should behave like that', function(done) {
      foo.should.have.length(3);
      done();
    });
  });
});

edited Nov 8, 2017 at 22:48

ZachB

15.8k5 gold badges67 silver badges94 bronze badges

answered Jun 2, 2014 at 7:17

Christiaan Westerbeek

11.2k13 gold badges69 silver badges94 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Alexander Mills Over a year ago

good work, this worked for me. One quick point - you probably don't need to use aysnc. You can probably just use ['n1','fr','de'].forEach(). If you tests are synchronous, they will run in order, and no more slowly than otherwise. If your tests are asynchronous, then they will run asynchronously, but the forEach loop will keep loop, although it shouldn't matter that much either way, async.each or [].forEach should be about the same. Unless I am missing something and it() is blocking somehow?

Christiaan Westerbeek Over a year ago

@AlexMills you're right. I updated the code and removed the use of async.

polkovnikov.ph Over a year ago

NB! This works only for synchronously dynamically created test cases. In general case Mocha doesn't support that.

rob3c Over a year ago

@polkovnikov.ph You're right that this accepted answer only works synchronously, which doesn't actually answer the OP's question about generating them from server requests (which are always async in nodejs). However, you can dynamically generate tests asynchronously using the hack in my answer below: stackoverflow.com/a/35793665/871990

polkovnikov.ph Over a year ago

@rob3c I have seen your answer, but it was a bit more complex than I was ready to do to put a library into my project. I just switched to node-tap, which works properly out of the box.

Community · Accepted Answer · 2017-05-23 11:47:09Z

33

+50

If you want to dynamically create It() tests using data obtained asynchronously, you can (ab)use the before() hook with a placeholder It() test to ensure mocha waits until before() is run. Here's the example from my answer to a related question, for convenience:

before(function () {
    console.log('Let the abuse begin...');
    return promiseFn().
        then(function (testSuite) {
            describe('here are some dynamic It() tests', function () {
                testSuite.specs.forEach(function (spec) {
                    it(spec.description, function () {
                        var actualResult = runMyTest(spec);
                        assert.equal(actualResult, spec.expectedResult);
                    });
                });
            });
        });
});

it('This is a required placeholder to allow before() to work', function () {
    console.log('Mocha should not require this hack IMHO');
});

edited May 23, 2017 at 11:47

CommunityBot

11 silver badge

answered Mar 4, 2016 at 10:28

rob3c

2,1161 gold badge22 silver badges22 bronze badges

6 Comments

mags Over a year ago

I tried all methods to accomplish this including trying to use the TDD framework for Mocha which has next to NO documentation. This method above works, but I just want to add that the outer "describe" is really just a placeholder. The inner test block is what counts. You will need to return a new promise once the dynamic tests are built and hook on to that promise with a new "then" method that will create an "after" block. So, there are two promises involved.

Harry Over a year ago

@rob3c If you add a after() block then you would notice, that it is not waiting for the before() to finish. The loop in the before() will surely create all those describe() blocks but it won't wait for them to finish. Any way to achieve that?

rob3c Over a year ago

@Harry The before() call does complete before invoking any declared after() calls. However, any promises invoked within its callback may not have completed by then, which is just the nature of async javascript. It seems that your question is really a more general one about how javascript promises work and are coordinated, rather than how to generate mocha tests dynamically as asked in this question. It would be off-topic to discuss general javascript features further here, but I encourage you to create a new question if you don't find a satisfactory answer from your searching.

rex Over a year ago

I'm trying to use this in cypress - any thoughts on why it wont work to load my expectation file and run the it() tests on it dynamically?

rob3c Over a year ago

@rex I haven't used cypress.io before, but the FAQ says it's built on top of mocha and runs javascript in the browser. This hack works with regular mocha.js in the browser, so I don't know what's different about how cypress is wrapping it. Perhaps that's best asked as a separate question, since this question is about using mocha directly?

|

Quanlong · Accepted Answer · 2014-09-11 14:28:36Z

14

With Mocha 1.21.4, you can create suite/test at runtime in following way.

require('chai').should()

Mocha = require 'mocha'
Test = Mocha.Test
Suite = Mocha.Suite


mocha = new Mocha
suite = Suite.create mocha.suite, 'I am a dynamic suite'
suite.addTest new Test 'I am a dynamic test', ->
  true.should.equal true

mocha.run () ->
  console.log("done")

See https://gist.github.com/cybertk/fff8992e12a7655157ed for more details

answered Sep 11, 2014 at 14:28

Quanlong

25.7k17 gold badges73 silver badges80 bronze badges

Comments

Tom Spencer · Accepted Answer · 2016-01-15 14:23:52Z

13

It's worth noting that in addition to the accepted answer above, mocha's docs now include an example of how to achieve this. I've reproduced it below for posterity.

var assert = require('assert');

function add() {
  return Array.prototype.slice.call(arguments).reduce(function(prev, curr) {
    return prev + curr;
  }, 0);
}

describe('add()', function() {
  var tests = [
    {args: [1, 2],       expected: 3},
    {args: [1, 2, 3],    expected: 6},
    {args: [1, 2, 3, 4], expected: 10}
  ];

  tests.forEach(function(test) {
    it('correctly adds ' + test.args.length + ' args', function() {
      var res = add.apply(null, test.args);
      assert.equal(res, test.expected);
    });
  });
});

answered Jan 15, 2016 at 14:23

Tom Spencer

8,0974 gold badges58 silver badges51 bronze badges

1 Comment

Hendy Irawan Over a year ago

This is only for synchronous data.

Michael Yurin · Accepted Answer · 2019-05-24 07:50:39Z

6

Yep! Brilliant advice from Quanlong!

Here is my example of dynamic test generation with Node's readline module:

const Mocha = require('mocha');
var Test = Mocha.Test;
var Suite = Mocha.Suite;

var mocha = new Mocha();
var suite = Suite.create(mocha.suite, 'My test suite with dynamic test cases');

lineReader
    .on('line', function (line) {
        suite.addTest(new Test(line, function () {
            return true;
        }));
    })
    .on('close', function () {
        mocha.run();
    });

edited May 24, 2019 at 7:50

answered Mar 15, 2017 at 8:37

Michael Yurin

1,07014 silver badges19 bronze badges

Comments

Jeff Lowery · Accepted Answer · 2019-02-14 00:46:32Z

4

I like @rob3c's answer, but tried to simplify it a bit:

describe("Master test suite", function () {
  before(async function () {
    const rows = await mySQLQuery();

    describe(`Testing ${rows.length} rows`, function () {
      rows.forEach(function (row, index) {
        it(`Test row ${index}`, async function() {
          console.log("your row assertions go here")
        });
      });
    });
  });


  it("stub", async function(){})  // this is important!
});

answered Feb 14, 2019 at 0:46

Jeff Lowery

2,6192 gold badges33 silver badges46 bronze badges

2 Comments

vincent Over a year ago

This is awesome as it plays nicely with node-tdd when mocking requests with the "useNock" flag <3

StormFoo Over a year ago

Ensuring you have at least one test already defined within the describe block when your dynamic test relies on an asynchronous process (in my case reading files) was the bit I was missing, this answer makes that clearer. Thanks!

Shannon Hochkins · Accepted Answer · 2019-09-18 00:14:30Z

0

You can accomplish this by updating the tests property manually after the response is returned from the async method:

describe(`sometest`, function() {
  let results = null
  before(async () => {
    results = await someAsyncMethod();
    results.forEach((result, index) => {
      // to hold on to the new dynamic tests
      const newTest = it(result.name || `test ${index}`, () => {
        // do something here in test
      });
      // update the test objects before the main tests run
      this.tests.push(newTest);
    });
  });

  it(`sometest`, () => {
    expect(results.length).toBeGreaterThan(2);
  });

});

This doesn't use dynamic describes etc, just updates the current describe block before the main tests run!

answered Sep 18, 2019 at 0:14

Shannon Hochkins

12.3k15 gold badges69 silver badges101 bronze badges

2 Comments

magixx Over a year ago

Thanks, this solution worked for me and seems just a bit cleaner than the others. One thing to try in real world scenarios would be to separate the test as a separarte function but I did have problems with this.

Shannon Hochkins Over a year ago

Glad to have helped :)

Collectives™ on Stack Overflow

How can I dynamically generate test cases in javascript/node?

7 Answers 7

5 Comments

6 Comments

Comments

1 Comment

Comments

2 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

7 Answers 7

5 Comments

6 Comments

Comments

1 Comment

Comments

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related