Introducing Opencv Library in a Java Application

At the heart of many machine vision solutions is a software package named OpenCV. Since I'm a Java developer, I've wondered if I can take advantage of this Python library inside a Java application. And so I built this project.

In this project, I created a backend application that can be tested using CURL or Postman that will integrate a simple OpenCV application. The API endpoint or endpoints handle all the inputs required by the application. Link to the OpenCV application can be found here: https://www.geeksforgeeks.org/detect-an-object-with-opencv-python/

This exercise is done using Quarkus, a serverless framework for JavaEE. To use the OpenCV, it is compiled to generate a jar and so library that is used by the Java application.

--

An alternative would be to use GraalVM, but sadly it doesn't support the OpenCV library yet.

In case you wanted to test running a Python code using Java running on GraalVM, I have commented on endpoints in the PolyglotResource class. To deploy, go to the GraalVM section.

Currently, GraalVM supports the following language:

• language:nfi
• language:js
• language:python
• language:regex
• language:llvm

1. Requirements

If you want to run the project as standalone:

• Ubuntu OS
• OpenJDK 11
• GraalVM 20.3.0
• Python
• PIP
• CMake

If you just want to run the project, docker is all you need.

2. Running the Application with Docker

The application can be packaged using:

./mvnw package

It produces the opencv-haar-java-1.0.0-SNAPSHOT-runner.jar file in the /target directory. Be aware that it’s not an über-jar as the dependencies are copied into the target/lib directory.

You can then execute your native executable with: ./target/opencv-haar-java-1.0.0-SNAPSHOT-runner

3. Build the dockerfile

Here's part of the dockerfile that instantiates the container and builds the OpenCV library.

# Install java and the run-java script
# Also set up permissions for user `1001`
RUN apt-get update \
    && apt-get -y install curl ca-certificates cmake git ant ${JAVA_PACKAGE} python3-pip python3-opencv \
    && pip3 install opencv-python \
    && export JAVA_HOME \
    && mkdir /deployments \
    && chown 1001 /deployments \
    && chmod "g+rwX" /deployments \
    && chown 1001:root /deployments \
    && curl https://repo1.maven.org/maven2/io/fabric8/run-java-sh/${RUN_JAVA_VERSION}/run-java-sh-${RUN_JAVA_VERSION}-sh.sh -o /deployments/run-java.sh \
    && chown 1001 /deployments/run-java.sh \
    && chmod 540 /deployments/run-java.sh

# Build the OpenCV libraries
RUN git clone https://github.com/opencv/opencv.git \
    && cd opencv \
    && git checkout tags/4.5.1 -b 4.5.1 \
    && mkdir build \
    && cd build \
    && cmake -D BUILD_SHARED_LIBS=OFF .. \
    && make -j8
      

After building the project, the docker image can now be created.

It creates a container from Ubuntu 20.04 with the following components:

• OpenJDK 11
• Python3 (OpenCV)
• PIP
• Cmake

It checks out the OpenCV code 4.5.1 and builds the library that will be used by the Java app.

docker build -f src/main/docker/Dockerfile.jvm -t quarkus/opencv-haar-java-jvm .

To run the application execute

docker run -i --rm -p 8080:8080 quarkus/opencv-haar-java-jvm

4. API

Open API documentation is accessible at http://localhost:8080/q/openapi.

Swagger UI is accessible at http://localhost:8080/q/swagger-ui.

5. Testing

To test this application Open the "OpenCV Haar in Java" Postman collection inside the test folder.

• Detect object - process a hardcoded image in code
• Detect object - accepts an image parameter and a filename use to cache the processed image. Make sure to change the attached image as it is lost during transfer.

6. GraalVM

As a bonus, I'll also share my findings on how you can run a Python code on a Java application. Take note that so far, only selected Python libraries are available.

Note: Ubuntu OS is needed.

Before you start building, make sure that:

1. you set GRAALVM_HOME to where you extracted GraalVM
2. Set JAVA_HOME to GRAALVM_HOME
3. Add JAVA_HOME\bin to the path

6.1 Follow these instructions to install GraalVM https://www.graalvm.org/docs/getting-started/#install-graalvm.

6.2 After GraalVM is installed, make sure that it is the default JVM, by using update-alternatives:

In case, GraalVM is not in the update-alternatives, you can install it by using these commands:

  sudo update-alternatives --install /usr/bin/javac javac /opt/graalvm-ce-java11-21.0.0/bin/javac 1
  sudo update-alternatives --install /usr/bin/java java /opt/graalvm-ce-java11-21.0.0/bin/java 1
  

6.3 Install the following, the last item is to test the installation:

		sudo apt-get install python3-opencv libopencv-dev
		pip3 install opencv-python
		python3 -c "import cv2; print(cv2.__version__)"
	

6.3 Navigate to GraalVM's bin folder.

6.4 Install Python:

sudo  ./gu install python

To get help, simply execute with -h parameter.

6.5 To check what libraries are available, execute:

 ./graalpython -m ginstall install -h

*As you can see, there is no opencv lib.

6.6 To install numpy:

./graalpython -m ginstall install numpy

6.7 Let's try to install opencv

./graalpython -m ginstall install opencv

It should throw an unknown package as of the date of this article's publishing.

6.8 Check out the graalvm branch of the repository.

6.9 Inspecting our project.

The relevant codes are in com.czetsuyatech.resource.PolyglotResource class. Make sure to uncomment the following endpoints:

@Path("/js")
@GET
@Produces(MediaType.TEXT_PLAIN)
public String helloJs() {
    context.eval(Source.create("js", "console.log('hello js')"));
    return "Running JS script console.log('hello js'). Check the logs.";
}

@Path("/py")
@GET
@Produces(MediaType.TEXT_PLAIN)
public String helloPy() {
    final URL pyTestPy = getClass().getClassLoader().getResource("pytest.py");
    System.out.println("" + pyTestPy);
    Value effectValue = context.eval(Source.newBuilder("python", pyTestPy).build());
    return "Result read from pytest.py=" + effectValue.toString();
}

6.10 Once you're ready you can build and run the Quarkus project:

    ./mvnw package -Dquarkus.package.type=uber-jar --this command will take some time
    java -jar target/opencv-haar-java-1.0.0-SNAPSHOT-runner.jar .
    

And access the resources that execute the Python script from file or Javascript in code.

7. Bonus: Build OpenCV Locally

7.1 Clone from https://github.com/opencv/opencv.git.

7.2 Install ant: sudo apt-get install ant

7.3 Go inside the cloned repository and execute the following commands:

        
mkdir build
cd build
cmake -D BUILD_SHARED_LIBS=OFF ..
make -j8

8. References

• https://github.com/czetsuya/opencv-haar-java
• https://advancedweb.hu/getting-started-with-opencv-for-java-on-ubuntu/