Generating, compiling and executing JUnit on the fly

Question

I have a class that generates Restassured JUnit tests in a specific subfolder of my build/tests/ structure, based on some postman test collections.

I have written a test to take all the *.postman_collection.json files in my src/test/resources/testCollections folder. If I want to verify that the generated files work I have to run the test again. I'd like to be able to run everything with mvn clean test.

Furthermore, I clear out the generated folder every time I run mvn clean, which I would have to undo if I had to run the tests twice.

Code is on github (I had issues picking out a minimalistic snippet to understand it from).

Update: switch to gradle

I tried to follow the suggestions from @chubbsondubs but something is not working as I expect it to.

build.gradle

plugins {
    id 'java-library'
    id 'jacoco'
}

repositories {
    mavenLocal()
    maven {
        url = uri('https://repo.maven.apache.org/maven2/')
    }
}

dependencies {
    api 'com.fasterxml.jackson.core:jackson-databind:2.18.3'
    api 'org.slf4j:slf4j-api:2.0.17'
    api 'ch.qos.logback:logback-classic:1.5.13'

    testImplementation 'org.junit.jupiter:junit-jupiter:5.12.1'
    testImplementation 'org.junit.jupiter:junit-jupiter-engine:5.12.1'
    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.12.1'
    testImplementation 'org.junit.jupiter:junit-jupiter-params:5.12.1'

    testRuntimeOnly 'org.junit.platform:junit-platform-launcher'
    
    testImplementation 'io.rest-assured:rest-assured:5.5.1'
    testImplementation 'io.rest-assured:rest-assured-common:5.5.1'
    
    testImplementation 'org.jacoco:org.jacoco.core:0.8.11'
    testImplementation 'org.jacoco:org.jacoco.agent:0.8.11'
    testImplementation 'org.jacoco:org.jacoco.report:0.8.11'

}

group = 'dk.kodeninjaer.testing'
version = '1.0-SNAPSHOT'
description = 'postman-to-restassured'
java {
   sourceCompatibility = '21'
}

test {
    finalizedBy jacocoTestReport // report is always generated after tests run
}

tasks.withType(JavaCompile) {
    options.encoding = 'UTF-8'
}

tasks.withType(Javadoc) {
    options.encoding = 'UTF-8'
}

tasks.named('test', Test) {
    useJUnitPlatform()
}

jacocoTestReport {
    dependsOn test // tests are required to run before generating the report
    // exclude Main.class from the coverage report
    afterEvaluate {
        classDirectories.setFrom(files(classDirectories.files.collect {
            fileTree(dir: it, exclude: [
                "dk/kodeninjaer/testing/converter/Main.class"
            ])
        }))
    }
}

jacoco {
    toolVersion = "0.8.11"
    //reportsDirectory = layout.buildDirectory.dir('customJacocoReportDir')
}


/* generate and compile */

// register a new directory for source files
sourceSets {
    generatedTests {
        java.srcDir 'build/generated/gradle'
        compileClasspath += sourceSets.main.output + sourceSets.main.compileClasspath + sourceSets.test.compileClasspath
        runtimeClasspath += sourceSets.main.output + sourceSets.main.compileClasspath + sourceSets.test.compileClasspath
    }
}

// a task to execute the generatedTests source set
tasks.register("runGeneratedTests",Test) {
    description = 'Runs the generated tests.'
    group = 'verification'
    testClassesDirs = sourceSets.generatedTests.output.classesDirs
    classpath = sourceSets.main.runtimeClasspath + sourceSets.generatedTests.runtimeClasspath
    useJUnitPlatform()

    // run this after you run the test task
    shouldRunAfter test
}

tasks.register("generateUnitTests", JavaExec) {
    description = 'Generates dynamic unit tests from Postman collections'
    group = 'verification'
    classpath = sourceSets.main.runtimeClasspath
    mainClass = "dk.kodeninjaer.testing.converter.Main"
    args = [
            // your args to generate the program
            "src/test/resources/TestCollection.postman_collection.json",
            "dk.kodeninjaer.gradle.testing",
            "PostmanToRestAssuredGenerator"
        ]
    //shouldRunBefore generatedTests
}



// when check task is run we add our generatedTests that way if test task 
// is run it will also run our generatedTests
tasks.named('check') {
   dependsOn(tasks.runGeneratedTests)
}

Error

 Could not create task ':generateUnitTests'.
    > No signature of method: org.gradle.api.tasks.JavaExec.mustRunBefore() is applicable for argument types: (org.gradle.api.tasks.compile.JavaCompile_Decorated) values: [task ':compileTestJava']
      Possible solutions: mustRunAfter([Ljava.lang.Object;), mustRunAfter([Ljava.lang.Object;)

Update 2

If I run ./gradlew generateUnitTests and ./gradlew runGeneratedTests or ./gradlew check it executes, but no testreport is generated...

First generated code should not be generated in src/ directories` ... the correct location would be the target directory.. It might be the case that you take a deep look at dynamic tests of JUnit Jupiter junit.org/junit5/docs/current/user-guide/… If you like to generate those tests and execute them later with Maven you should write a plugin which should be bound to the generate-test-sources .... — khmarbaise
– khmarbaise, Commented Mar 15 at 8:30

chubbsondubs · Accepted Answer · 2025-03-23 21:56:48Z

1

Since the modality of your tool is to have a developer generate unit tests then run those generated files later. I think you should mirror that way of working in your projects build. So instead of executing your tool within a unit test goal. You should execute it outside of the unit test phase/goal. That would generate the unit tests based on your test Postman collections. Then when the test goal runs the files will already be there ready to run as normal unit tests. Then there is no problem of execute and test from within tests.

As far as I know having another separate build folder with Maven is not possible. But, I'm not much of a maven kinda guy. This is possible with Gradle though.

// register a new directory for source files
sourceSets {
    generatedTests {
        java.srcDir 'src/test/gen'
        compileClasspath += sourceSets.main.output + sourceSets.main.compileClasspath + sourceSets.test.compileClasspath
        runtimeClasspath += sourceSets.main.output + sourceSets.main.compileClasspath + sourceSets.test.compileClasspath
    }
}

// a task to execute the generatedTests source set
tasks.register("generatedTests",Test) {
    description = 'Runs the generated tests.'
    group = 'verification'
    testClassesDirs = sourceSets.generatedTests.output.classesDirs
    classpath = sourceSets.main.runtimeClasspath + sourceSets.generatedTests.runtimeClasspath
    useJUnitPlatform()

    // run this after you run the test task
    shouldRunAfter test
}

tasks.register("generateUnitTests", JavaExec) {
    description = 'Generates dynamic unit tests from Postman collections'
    group = 'verification'
    classpath = sourceSets.main.runtimeClasspath
    main = "dk.kodeninjaer.testing.converter.Main"
    args = [
        // your args to generate the program
    ]
}

// when check task is run we add our generatedTests that way if test task 
// is run it will also run our generatedTests
tasks.named('check') {
   dependsOn(tasks.generatedTests)
}

// run this before we compile all tests source sets
tasks.name('compileTestJava') {
   dependsOn(tasks.generateUnitTests)
}

edited Mar 23 at 21:56

answered Mar 16 at 14:06

chubbsondubs

39.4k25 gold badges110 silver badges143 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

JoSSte Mar 16 at 21:52

Thank you. My primary goal is to make my project convert a postman test suite to restassured, with the basic assertions taken care of. I agree that it is wrong to add it top the original project. If Gradle can do that, I'll have to learn gradle.Maybe this is the excuse to learn Gradle. Compiling and running will indicate the general state of the conversion, as opposed to making a set of .java files and letting the user fix the errors by themselves.

chubbsondubs Mar 17 at 1:15

So you could add these generated tests to the buildDir directory which would allow them to be cleaned automatically when executing build clean. I didn't solve that problem with the above code. I was torn as to whether to add them to build or add them under src. Some people generate code into src others into build. There is debate for either approach and IDEs treat that code in either directory. All you need to adjust is src/test/gen to whichever you prefer.

JoSSte Mar 22 at 12:15

I spent some time converting my builds to Gradle and learning the basics. when I try to run it it fails...

chubbsondubs Mar 23 at 1:42

What part fails? Any error messages?

JoSSte Mar 23 at 17:11

I updated the question: No signature of method: org.gradle.api.tasks.JavaExec.mustRunBefore

|

Collectives™ on Stack Overflow

Generating, compiling and executing JUnit on the fly

Update: switch to gradle

build.gradle

Error

Update 2

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

Update: switch to gradle

build.gradle

Error

Update 2

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related