0

In my SpringBoot application, I have used Spring Integration for uploading files to a remote SFTP server.
Below is the code snippet I have used to transfer file to SFTP server.

protected String transfer(String downloadDirectory) {
        File[] files = new File(downloadDirectory).listFiles();
        byte[] payload = null;
        String path = null;
        try {

            path = getSftpPath(downloadDirectory);
            for (int i = 0; i < files.length; i++) {
                if (files[i].isFile()) {
                    payload = Files.readAllBytes(Paths.get(files[i].toURI()));
                    gateway.upload(payload, files[i].getName(), path);
                }
            }
        } catch (Exception e) {

            throw new Exception(e);
        }
        return path;
    }

The upload method declaration and MessageHandler is as follows

@MessagingGateway(asyncExecutor = "sftpAsync")
    public interface UploadGateway {
        @Gateway(requestChannel = "toSftpChannel")
        void upload(@Payload byte[] file, @Header("filename") String filename, @Header("path") String path);
    }

 @Bean
    @ServiceActivator(inputChannel = "toSftpChannel")
    public MessageHandler handler() {
        SftpMessageHandler handler = new SftpMessageHandler(sftpSessionFactory());
        ExpressionParser expressionParser = new SpelExpressionParser();
        handler.setRemoteDirectoryExpression(expressionParser.parseExpression("headers['path']"));
        handler.setFileNameGenerator(msg -> (String) msg.getHeaders().get("filename"));
        handler.setAutoCreateDirectory(true);
        return handler;
    }

It works fine for uploading files to a remote SFTP server. But after couple of execution, it starts showing

java.lang.OutOfMemoryError: Direct buffer memory
    at java.nio.Bits.reserveMemory(Unknown Source)
    at java.nio.DirectByteBuffer.<init>(Unknown Source)
    at java.nio.ByteBuffer.allocateDirect(Unknown Source)
    at sun.nio.ch.Util.getTemporaryDirectBuffer(Unknown Source)
    at sun.nio.ch.IOUtil.read(Unknown Source)
    at sun.nio.ch.FileChannelImpl.read(Unknown Source)
    at sun.nio.ch.ChannelInputStream.read(Unknown Source)
    at sun.nio.ch.ChannelInputStream.read(Unknown Source)
    at sun.nio.ch.ChannelInputStream.read(Unknown Source)
    at java.nio.file.Files.read(Unknown Source)
    at java.nio.file.Files.readAllBytes(Unknown Source)

payload = Files.readAllBytes(Paths.get(files[i].toURI())); is the line where it throws the above error message.

As per my understanding, I am reading entire file into byte array, hence I am getting this error. Since I need entire file to send to upload method ,I have followed byte[] approach.

I already checked below mentioned threads.

Byte[] and java.lang.OutOfMemoryError reading file by bits

java.lang.OutOfMemoryError: Direct buffer memory when invoking Files.readAllBytes

but the upload method need entire file instead of stream of bytes or chunks of data (correct me if i am wrong).

So,any suggestions to avoid above issue is great help.

Thanks.

2
  • Replace this: payload = Files.readAllBytes(Paths.get(files[i].toURI())); gateway.upload(payload, files[i].getName(), path); For this: Path tempFile = Paths.get(files[i].toURI()); gateway.upload(tempFile.toFile()); Commented Nov 21, 2018 at 13:34
  • You need to work with an InputStream instead of array or entire blob that will sit in memory. It can be ByteArrayInputStream, FileInputStream, etc. Study the sftp adapter java docs to see which interfaces are available. Commented Nov 21, 2018 at 13:51

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.