This is a follow-up question to two previous posts on the most efficient way to convert an OpenCV Mat object into a JavaFX Image (post 1, post 2).

@James_D, @Slaw, pointed to two possible approaches, in which @Slaw elaborated in his answer:

MemorySegment(of the Mat) -> ByteBuffer -> PixelBuffer -> return the image using WritableImage

and

Byte [] -> ByteBuffer -> read the buffer using WritableImage and return the image

For the first one, I am getting a fatal error. Meanwhile, with the second approach, I am getting an unusual result where the image becomes repetitive on the X-axis, displaying horizontal stripes.

About the app

This part of the application is responsible for maintaining only a specific color, based on the user's input of the maximum and minimum HSV values.

For this purpose, the class FilterByHSV has the filterImage(int minHue, int maxHue, int minSaturation, int maxSaturation, int minValue, int maxValue) method that returns a matrix, where all pixels within the specified HSV range, using sliders, are shown in color, and the rest are in gray.

Whenever the user changes one of the slider , filterImage is invoked, and the resulting Mat object is then converted into an Image that will be displayed in the application.

For converting the mat into an image, the class MatToFXImage extends Task<Image> implements the necessary logic in the call method.

The code

1. ProcessImage is to create the UI and its elements.

import javafx.application.Application;
import javafx.geometry.Insets;
import javafx.scene.Scene;
import javafx.scene.control.Label;
import javafx.scene.control.Slider;
import javafx.scene.image.Image;
import javafx.scene.image.ImageView;
import javafx.scene.layout.ColumnConstraints;
import javafx.scene.layout.GridPane;
import javafx.scene.layout.HBox;
import javafx.scene.layout.RowConstraints;
import javafx.scene.text.Font;
import javafx.stage.Stage;

import java.util.Objects;

public class ProcessImage extends Application {

    private GridPane createFilterWindow() {
        // Create main GridPane
        GridPane mainGridPane = new GridPane();
        mainGridPane.setPrefHeight(600.0);
        mainGridPane.setPrefWidth(800.0);
        mainGridPane.setStyle("-fx-background-color: #5b5b5b");
    
        // Create column constraints
        ColumnConstraints mainColumn = new ColumnConstraints();
        mainColumn.setHgrow(javafx.scene.layout.Priority.ALWAYS);
        mainGridPane.getColumnConstraints().add(mainColumn);
    
        // Create row constraints
        RowConstraints row1 = new RowConstraints();
        row1.setVgrow(javafx.scene.layout.Priority.NEVER);
        RowConstraints row2 = new RowConstraints();
        row2.setVgrow(javafx.scene.layout.Priority.ALWAYS);
        mainGridPane.getRowConstraints().addAll(row1, row2);
    
        // Create first HBox (controls section)
        HBox controlsHBox = new HBox();
        controlsHBox.setAlignment(javafx.geometry.Pos.CENTER);
    
        // Create inner GridPane for controls
        GridPane controlsGridPane = new GridPane();
        controlsGridPane.setAlignment(javafx.geometry.Pos.CENTER);
        controlsGridPane.setVgap(20);
        controlsGridPane.setStyle("-fx-background-color: #00ff93");
        HBox.setHgrow(controlsGridPane, javafx.scene.layout.Priority.ALWAYS);
    
        // Create column constraint for inner GridPane
        ColumnConstraints innerColumn = new ColumnConstraints();
        innerColumn.setHgrow(javafx.scene.layout.Priority.ALWAYS);
        controlsGridPane.getColumnConstraints().add(innerColumn);
    
        // Create slider controls
        createSliderControl(controlsGridPane, 0, "Min H", "minHueSlider", "minHueLabel");
        createSliderControl(controlsGridPane, 1, "Max H", "maxHueSlider", "maxHueLabel");
        createSliderControl(controlsGridPane, 2, "Min S", "minSaturationSlider", "minSaturationLabel");
        createSliderControl(controlsGridPane, 3, "Max S", "maxSaturationSlider", "maxSaturationLabel");
        createSliderControl(controlsGridPane, 4, "Min V", "minValueSlider", "minValueLabel");
        createSliderControl(controlsGridPane, 5, "Max V", "maxValueSlider", "maxValueLabel");
    
        controlsHBox.getChildren().add(controlsGridPane);
        mainGridPane.add(controlsHBox, 0, 0);
    
        // Create second HBox (image section)
        HBox imageHBox = new HBox();
        imageHBox.setAlignment(javafx.geometry.Pos.CENTER);
    
        // Create ImageView
        ImageView inputImage = new ImageView();
        inputImage.setId("inputImage");
        inputImage.setPickOnBounds(true);
        inputImage.setPreserveRatio(true);
        inputImage.setFitHeight(500);
        inputImage.setFitWidth(500);
    
        // Load image
        try {
            Image image = new Image(getClass().getResource("path_to_your_image").toString(), true);
            inputImage.setImage(image);
        } catch (Exception e) {
            System.err.println("Could not load image: " + e.getMessage());
        }
    
        imageHBox.getChildren().add(inputImage);
        mainGridPane.add(imageHBox, 0, 1);
    
        return mainGridPane;
    }
    
    private void createSliderControl(GridPane gridPane, int rowIndex, String labelText, String sliderId, String labelId) {
        HBox hbox = new HBox();
        hbox.setAlignment(javafx.geometry.Pos.CENTER);
        hbox.setPrefHeight(60);
    
        // Create label
        Label nameLabel = new Label(labelText);
        nameLabel.setTextFill(javafx.scene.paint.Color.BLACK);
        nameLabel.setFont(Font.font("Times New Roman Bold", 15.0));
        HBox.setMargin(nameLabel, new Insets(0, 20, 0, 20));
    
        // Create slider
        Slider slider = new Slider();
        slider.setId(sliderId);
        slider.setBlockIncrement(0.1);
        slider.setMajorTickUnit(0.5);
        slider.setMax(255.0);
        slider.getStyleClass().add("slider");
        HBox.setHgrow(slider, javafx.scene.layout.Priority.ALWAYS);
    
        // Create value label
        Label valueLabel = new Label("0");
        valueLabel.setId(labelId);
        valueLabel.setTextFill(javafx.scene.paint.Color.BLACK);
        valueLabel.setFont(Font.font("Times New Roman Bold", 15.0));
        HBox.setMargin(valueLabel, new Insets(0, 20, 0, 20));
    
        hbox.getChildren().addAll(nameLabel, slider, valueLabel);
        gridPane.add(hbox, 0, rowIndex);
    }
    
    @Override
    public void start(Stage stage) throws Exception {
        GridPane parentNode = createFilterWindow();
        Scene scene = new Scene(parentNode, 900, 700);
        stage.setTitle("Test window");
    
        ProcessImageController controller = new ProcessImageController();
        controller.mainGridPane = parentNode;
        controller.inputImage = (ImageView) parentNode.lookup("#inputImage");
        controller.setInputImagePath("C:\\Users\\PC\\Desktop\\bird-9445431.jpg");
        controller.minHueLabel = (Label) parentNode.lookup("#minHueLabel");
        controller.maxHueLabel = (Label) scene.lookup("#maxHueLabel");
        controller.minSaturationLabel = (Label) parentNode.lookup("#minSaturationLabel");
        controller.maxSaturationLabel = (Label) parentNode.lookup("#maxSaturationLabel");
        controller.minValueLabel = (Label) parentNode.lookup("#minValueLabel");
        controller.maxValueLabel = (Label) parentNode.lookup("#maxValueLabel");
        controller.minHueSlider = (Slider) scene.lookup("#minHueSlider");
        controller.maxHueSlider = (Slider) scene.lookup("#maxHueSlider");
        controller.minSaturationSlider = (Slider) parentNode.lookup("#minSaturationSlider");
        controller.maxSaturationSlider = (Slider) parentNode.lookup("#maxSaturationSlider");
        controller.minValueSlider = (Slider) parentNode.lookup("#minValueSlider");
        controller.maxValueSlider = (Slider) parentNode.lookup("#maxValueSlider");
        controller.initialize();
    
        try {
            stage.getIcons().add(new Image(Objects.requireNonNull(getClass().getResourceAsStream("resources/window-icon.png"))));
        } catch (Exception e) {
            System.out.println(e.getMessage());
        }
    
        stage.setScene(scene);
        stage.show();
    
        stage.show();
    }

2. ProcessImageController Is the controller of the view and its controls.

import javafx.application.Platform;
import javafx.fxml.FXML;
import javafx.scene.control.Label;
import javafx.scene.control.Slider;
import javafx.scene.image.Image;
import javafx.scene.image.ImageView;
import javafx.scene.layout.GridPane;
import org.opencv.core.Mat;
import tasks.MatToFXImage;
import utils.FilterByHSV;
import utils.HSVAdaptor;

import java.util.Objects;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicReference;

public class ProcessImageController {
    // UI Components
    @FXML
    public GridPane mainGridPane;
    @FXML
    public ImageView inputImage;

    // Labels
    @FXML
    public Label minHueLabel;
    @FXML
    public Label maxHueLabel;
    @FXML
    public Label minSaturationLabel;
    @FXML
    public Label maxSaturationLabel;
    @FXML
    public Label minValueLabel;
    @FXML
    public Label maxValueLabel;

    // Sliders
    @FXML
    public Slider minHueSlider;
    @FXML
    public Slider maxHueSlider;
    @FXML
    public Slider minSaturationSlider;
    @FXML
    public Slider maxSaturationSlider;
    @FXML
    public Slider minValueSlider;
    @FXML
    public Slider maxValueSlider;

    private FilterByHSV filterByHSV;

    // Single-thread executor ensures processing tasks don't overlap and preserves order.
    private final ExecutorService imageProcessingExecutor = Executors.newSingleThreadExecutor(r -> {
        Thread t = new Thread(r, "image-processing");
        t.setDaemon(true);
        return t;
    });

    // Scheduled executor used only for debounce timing.
    private final ScheduledExecutorService debounceExecutor = Executors.newSingleThreadScheduledExecutor(r -> {
        Thread t = new Thread(r, "debounce-timer");
        t.setDaemon(true);
        return t;
    });

    // Holds the currently scheduled debounce task so it can be cancelled/rescheduled.
    private final AtomicReference<ScheduledFuture<?>> pendingDebounce = new AtomicReference<>();

    // Debounce interval in milliseconds.
    private static final long DEBOUNCE_MS = 150L;

    @FXML
    public void initialize() {
        setupSliderListeners();
    }

    private void setupSliderListeners() {
        addSliderPair(minHueSlider, minHueLabel, maxHueSlider, maxHueLabel);
        addSliderPair(minSaturationSlider, minSaturationLabel, maxSaturationSlider, maxSaturationLabel);
        addSliderPair(minValueSlider, minValueLabel, maxValueSlider, maxValueLabel);
    }

    private void addSliderPair(Slider minSlider, Label minLabel, Slider maxSlider, Label maxLabel) {
        // Min slider
        minSlider.valueProperty().addListener((_, _, newV) -> {
            double minVal = newV.doubleValue();
            // Clamp to maintain the gap
            double maxAllowed = maxSlider.getValue() - 10.0;
            if (minVal > maxAllowed) {
                minVal = maxAllowed;
                minSlider.setValue(minVal);
            }
            minLabel.setText(String.format("%.02f", minVal));
            scheduleDebouncedUpdate();
        });

        // Max slider
        maxSlider.valueProperty().addListener((_, _, newV) -> {
            double maxVal = newV.doubleValue();
            // Clamp to maintain the gap
            double minAllowed = minSlider.getValue() + 10.0;
            if (maxVal < minAllowed) {
                maxVal = minAllowed;
                maxSlider.setValue(maxVal);
            }
            maxLabel.setText(String.format("%.02f", maxVal));
            scheduleDebouncedUpdate();
        });
    }

    private void scheduleDebouncedUpdate() {
        ScheduledFuture<?> prev = pendingDebounce.getAndSet(debounceExecutor.schedule(() ->
                Platform.runLater(this::updateImageFilter), DEBOUNCE_MS, TimeUnit.MILLISECONDS));
        if (prev != null)
            prev.cancel(false);
    }

    private Image matToImage(Mat mat) {
        try {
            MatToFXImage task = new MatToFXImage(mat);
//            task.exceptionProperty().subscribe(Throwable::printStackTrace);
            new Thread(task).start();
            return task.get();
        } catch (InterruptedException | ExecutionException e) {
            throw new RuntimeException("Failed to convert Mat to Image", e);
        }
    }

    private static String stripFilePrefix(String filePath) {
        if (filePath != null && filePath.startsWith("file:")) {
            return filePath.substring("file:".length());
        }
        return filePath;
    }

    private void updateImageFilter() {
        if (filterByHSV == null) return;

        // Read current slider values once
        int minHue = (int) HSVAdaptor.adaptHue(minHueSlider.getValue(), 255, 179);
        int maxHue = (int) HSVAdaptor.adaptHue(maxHueSlider.getValue(), 255, 179);

        int minSaturation = (int) minSaturationSlider.getValue();
        int maxSaturation = (int) maxSaturationSlider.getValue();

        int minValue = (int) minValueSlider.getValue();
        int maxValue = (int) maxValueSlider.getValue();

        imageProcessingExecutor.submit(() -> {
            try {
                Mat outputMat = filterByHSV.filterImage(minHue, maxHue, minSaturation, maxSaturation, minValue, maxValue);
                Image outputImage = matToImage(outputMat);
                Platform.runLater(() -> inputImage.setImage(outputImage));
            } catch (Exception e) {
                // Consider logging to a logger instead of throwing if UI stability is preferred
                throw new RuntimeException("Image processing failed", e);
            }
        });
    }

    @FXML
    public void setInputImagePath(String path) {
        Objects.requireNonNull(path, "path");
        Image image = new Image("file:" + path);
        inputImage.setImage(image);
        String filePath = stripFilePrefix(image.getUrl());
        filterByHSV = new FilterByHSV(filePath);
    }

    @FXML
    public void setSlidersValues(double minHue, double maxHue, double minSaturation, double maxSaturation, double minValue, double maxValue) {
        minHueSlider.setValue(minHue);
        maxHueSlider.setValue(maxHue);
        minSaturationSlider.setValue(minSaturation);
        maxSaturationSlider.setValue(maxSaturation);
        minValueSlider.setValue(minValue);
        maxValueSlider.setValue(maxValue);
        scheduleDebouncedUpdate();
    }

    public void dispose() {
        ScheduledFuture<?> pending = pendingDebounce.getAndSet(null);
        if (pending != null) pending.cancel(false);

        debounceExecutor.shutdown();
        imageProcessingExecutor.shutdown();

        try {
            if (!debounceExecutor.awaitTermination(500, TimeUnit.MILLISECONDS)) {
                debounceExecutor.shutdownNow();
            }
            if (!imageProcessingExecutor.awaitTermination(1, TimeUnit.SECONDS)) {
                imageProcessingExecutor.shutdownNow();
            }
        } catch (InterruptedException e) {
            debounceExecutor.shutdownNow();
            imageProcessingExecutor.shutdownNow();
            Thread.currentThread().interrupt();
        }
    }
}

3. MatToFXImage It is a Task where the mat will be converted into a JavaFX image.

   import java.awt.image.BufferedImage;
   import java.awt.image.DataBufferByte;
   import java.awt.image.WritableRaster;
   import java.io.ByteArrayInputStream;
   import java.lang.foreign.Arena;
   import java.lang.foreign.MemorySegment;
   import java.nio.Buffer;
   import java.nio.ByteBuffer;
   import java.util.Arrays;
   
   import javafx.concurrent.Task;
   import javafx.embed.swing.SwingFXUtils;
   import javafx.scene.image.Image;
   import javafx.scene.image.PixelBuffer;
   import javafx.scene.image.PixelFormat;
   import javafx.scene.image.WritableImage;
   import org.opencv.core.CvType;
   import org.opencv.core.Mat;
   import org.opencv.core.MatOfByte;
   import org.opencv.imgcodecs.Imgcodecs;
   
   public class MatToFXImage extends Task<Image> {
       private final Mat mat;
   
       public MatToFXImage(Mat mat) {
           if (mat == null)
               throw new IllegalArgumentException("Mat object can't be null");
   
           this.mat = mat;
       }
   
       private int determineImageType(int matType) {
           if (matType == CvType.CV_8UC1)
               return BufferedImage.TYPE_BYTE_GRAY;
           else if (matType == CvType.CV_8UC3)
               return BufferedImage.TYPE_3BYTE_BGR;
           else
               throw new IllegalArgumentException("Unsupported Mat type: " + matType);
       }
   
       public Image fromAddress(long address, int width, int height) {
           try {
               int length = width * height * 6; // may want to guard against overflow
               var segment = MemorySegment.ofAddress(address).reinterpret(length);
   
               var buffer = segment.asByteBuffer();
               var format = PixelFormat.getByteBgraPreInstance();
               var pixels = new PixelBuffer<>(width, height, buffer, format);
   
               return new WritableImage(pixels);
           } catch (Exception e) {
               e.printStackTrace();
               return null;
           }
       }
   
       public <T extends Buffer> Image fromBuffer(T buffer, int width, int height, PixelFormat<T> format) {
           var image = new WritableImage(width, height);
           var writer = image.getPixelWriter();
           writer.setPixels(0, 0, width, height, format, buffer, width);
   
           return image;
       }
   
       @Override
       protected Image call() {
   // Works fine
           MatOfByte byteMat = new MatOfByte();
           Imgcodecs.imencode(".bmp", mat, byteMat);
           return new Image(new ByteArrayInputStream(byteMat.toArray()));
   
   // Fatal Error
           // try {
           //     System.out.println(mat.dataAddr());
           //     return fromAddress(mat.dataAddr(), mat.width(), mat.height());
           // }catch (Exception e) {
           //         e.printStackTrace();
           //         return null;
           // }
   
   // Strange result 1
           // try {
           //     int size = (int) (mat.total() * mat.channels());
           //     byte[] byteArray = new byte[size * 5];
           //     mat.get(0, 0, byteArray);
   
           //     ByteBuffer buffer = ByteBuffer.wrap(byteArray);
           //     PixelFormat<ByteBuffer> pixelFormat = PixelFormat.getByteBgraPreInstance();
   
           //     return fromBuffer(buffer, mat.width(), mat.height(), pixelFormat);
           // } catch (Exception e) {
           //     e.printStackTrace();
           //     return null;
           // }
   
   
   // Strange result 2
           // try {
           //     int size = (int) (mat.total() * mat.channels());
           //     System.out.println(mat.channels());
           //     byte[] byteArray = new byte[size * 5];
   
           //     mat.get(0, 0, byteArray);
   
           //     ByteBuffer buffer = ByteBuffer.wrap(byteArray);
           //     PixelFormat<ByteBuffer> pixelFormat = PixelFormat.getByteBgraPreInstance();
           //     PixelBuffer<ByteBuffer> pixelBuffer = new PixelBuffer<>(mat.width(), mat.height(), buffer, pixelFormat);
   
           //     return new WritableImage(pixelBuffer);
           // } catch (Exception e) {
           //     e.printStackTrace();
           //     return null;
           // }
   
   
   // Works fine
           // int type;
           // type = determineImageType(mat.type());
   
           // BufferedImage image = new BufferedImage(mat.width(), mat.height(), type);
           // WritableRaster raster = image.getRaster();
           // DataBufferByte dataBuffer = (DataBufferByte) raster.getDataBuffer();
   
           // // byte[] data = dataBuffer.getData();
   
           // mat.get(0, 0, dataBuffer.getData());
           // return SwingFXUtils.toFXImage(image, null);
       }
   }
   
   
   

4. FilterByHSV keeps only the pixels within the specified range and turns the rest into grey.

   package utils;
   
   import org.opencv.core.Core;
   import org.opencv.core.Mat;
   import org.opencv.core.Scalar;
   import org.opencv.imgcodecs.Imgcodecs;
   import org.opencv.imgproc.Imgproc;
   
   public class FilterByHSV {
       private static final int COLOR_CONVERSION_BGR_TO_HSV = Imgproc.COLOR_BGR2HSV;
       private static final int COLOR_CONVERSION_BGR_TO_GRAY = Imgproc.COLOR_BGR2GRAY;
       private static final int COLOR_CONVERSION_GRAY_TO_BGR = Imgproc.COLOR_GRAY2BGR;
   
       static {
           System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
       }
   
       Mat image;
       Mat imageInHSV;
       Mat grey;
       Mat mask;
       Mat result;
   
       public FilterByHSV(String imageToFilterPath) {
           imageInHSV = new Mat();
           grey = new Mat();
           mask = new Mat();
           result = new Mat();
   
           image = Imgcodecs.imread(imageToFilterPath);
           Imgproc.cvtColor(image, imageInHSV, COLOR_CONVERSION_BGR_TO_HSV);
           Imgproc.cvtColor(image, grey, COLOR_CONVERSION_BGR_TO_GRAY);
       }
   
       public Mat filterImage(int minHue, int maxHue, int minSaturation, int maxSaturation, int minValue, int maxValue) {
           Core.inRange(imageInHSV, new Scalar(minHue, minSaturation, minValue), new Scalar(maxHue, maxSaturation, maxValue), mask);
           Imgproc.cvtColor(grey, result, COLOR_CONVERSION_GRAY_TO_BGR);
           image.copyTo(result, mask);
   
           return result;
       }
   }
   
   
   

5. HSVAdaptor convert the HSV values into their real values.

   public class HSVAdaptor {
       public static double adaptHue(double hueValue, double from, double to) {
           return (hueValue / from) * to;
       }
   
       public static double adaptSaturation(double saturation) {
           return saturation * 255;
       }
   
       public static double adaptValue(double value) {
           return value * 255;
       }
   }
   

Now this is what the result should look like (the working parts are commented as // work fine)

For the // fatal error par this the error code I get:

# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffa1a92059b, pid=49852, tid=13600
#
# JRE version: Java(TM) SE Runtime Environment (25.0+37) (build 25+37-LTS-3491)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25+37-LTS-3491, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, windows-amd64)
# Problematic frame:
# C  [VCRUNTIME140.dll+0x1059b]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\Users\PC\IdeaProjects\Color Isolator 2\hs_err_pid49852.log
[17.144s][warning][os] Loading hsdis library failed
#
# If you would like to submit a bug report, please visit:
#   https://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.

For // strange result 1 this is what i get

For // strange result 2 this is what i get

In order to use MemorySegment I added the following configiration, so native code can be run

--enable-native-access=javafx.graphics
--enable-native-access=ALL-UNNAMED

I would like to know why both approaches are not working, and what is causing the fatal error and the unusual results when using the byte buffer