Apache Flink Rest-Client Jar-Upload not working

Question

I am struggling to automatically deploy new Flink jobs within our CI/CD workflows by using the Flink rest-api (which may be found here in the flink Github repository).

Documentation only says that that jar upload may be achieved by using /jars/upload, but not how exactly a valid rest request has to be build (which Headers, which Body type, which Authorization, which Method and so on).

So I took a look at the Flink dashboard code of flink/flink-runtime-web project on Github and searched for the implementation they used to upload a jar and - Yippie! Its implemented by calling the rest-api I am trying to use (using POST as method). After that I tried to figure out with Postman which is correct way to send requests using different Content-Type headers and Body types, but none of them worked for me now.

I would have filed a ticket directly to the flink project, but could not find any reference to their ticket system.

So the basic Question here is:

How do I have to call the rest endpoint /jars/upload to successfully upload a file?

Helder Pereira · Accepted Answer · 2017-04-26 12:09:34Z

10

Those who are more partial to a command line can use curl:

curl -X POST -H "Expect:" -F "jarfile=@path/to/flink-job.jar" http://hostname:8081/jars/upload

edited Apr 26, 2017 at 12:09

answered Apr 24, 2017 at 9:49

Helder Pereira

5,7713 gold badges39 silver badges55 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Kulasangar Over a year ago

Thanks, this worked. I actually missed the @path and struggled for some time.

Nicolas Esteves · Accepted Answer · 2017-04-15 14:18:59Z

8

I've run into the same issue and solved it by looking at the network request in chrome when uploading a jar with the web UI.

The request must

Use multipart upload
The field name must be jarfile
The multi part content must include the file Content-Type as well (otherwise you'll get a 500 from Flink complaining about the missing header)

Here is a python script using requests that does the upload

upload = requests.post(                                                                                               
    base_url + "/jars/upload",                                                                                        
    files={
        "jarfile": (
            os.path.basename(path), 
            open(path, "rb"), 
            "application/x-java-archive"
         )
    }
)

answered Apr 15, 2017 at 14:18

Nicolas Esteves

1861 gold badge3 silver badges7 bronze badges

2 Comments

Jake W Over a year ago

Thanks. Item 3 'application/x-java-archive' is the key, which resolved for me as well.

Vikrant Goel Over a year ago

What about deleting old jar files, any idea?

sky · Accepted Answer · 2019-03-11 10:41:20Z

For those who want a java solution can use:

CloseableHttpClient client = HttpClients.createDefault();
        HttpPost httpPost = new HttpPost("YOUR API URL");
        MultipartEntityBuilder builder = MultipartEntityBuilder.create();
        builder.addBinaryBody("jarfile", new File("jarr.jar"));
        HttpEntity multipart = builder.build();
        httpPost.setEntity(multipart);

        try{
            CloseableHttpResponse response = client.execute(httpPost);
            System.out.println(response);
            BufferedReader rd = new BufferedReader(new InputStreamReader(
                    response.getEntity().getContent()));

            String line = "";
            while ((line = rd.readLine()) != null) {
                System.out.println(line);
                }
        }
        catch (Exception e){
            e.printStackTrace();
        }

Collectives™ on Stack Overflow

Apache Flink Rest-Client Jar-Upload not working

3 Answers 3

1 Comment

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related