Question for FireFly Developers: HDMI input and openCV

hanban

Hello, I need to know if it is possible to grab frames coming from the Firefly RK3288 reload HDMI input , to use in OpenCV.

I know HDMI input does not work under linux, but I am developing in android studio to use on the FireFly android OS.

I need to grab 1080p HDMI from a camera going into the HDMI input  port and display it with openCV.

is it possible? any SDK to do it?
in openCV , cap = cv2.VideoCapture('device index') , i need to know how to make it read from the HDMI input.

Please help!

thanks!

isle

Hi ,yes It is possible,
In Android, It is a device of camera, so, see the class of android.hardware.camera2

hanban

isle Posted at 5/22/2017 09:43
Hi ,yes It is possible,
In Android, It is a device of camera, so, see the class of android.hardware ...

Hello isle, good to know that it works! So I import android.hardware classes to android studio ?
Is there any sample code I might look at to understand how to capture the frames?
If it is recognized as a camera device, will this work with no modifications?
int main(int, char**)
{
    VideoCapture cap(0); // open the default camera
    if(!cap.isOpened())  // check if we succeeded
        return -1;

isle

there is a demo in github : https://github.com/pinguo-yuyidong/Camera2

hanban

isle Posted at 5/23/2017 09:35
there is a demo in github : https://github.com/pinguo-yuyidong/Camera2

I need help, i got it working but the frame rate is very slow (10-20 fps). I need atleast constant 30 fps and above. can you look at this and help me out? thanks!!
i ad 2 posts witrh my code ands third posat with a pictue of the result!


package com.example.david.opencv_test;

import android.os.Bundle;
import android.support.v7.app.AppCompatActivity;
import android.util.Log;
import android.view.MenuItem;
import android.view.SurfaceView;
import android.view.WindowManager;

import org.opencv.android.JavaCameraView;
import org.opencv.android.BaseLoaderCallback;
import org.opencv.android.CameraBridgeViewBase;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewFrame;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewListener2;
import org.opencv.android.LoaderCallbackInterface;
import org.opencv.android.OpenCVLoader;
import org.opencv.core.Core;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
import org.opencv.imgproc.Imgproc;
import org.opencv.videoio.VideoCapture;

// OpenCV Classes

public class MainActivity_show_camera extends AppCompatActivity implements CvCameraViewListener2 {
    VideoCapture vcap   = new VideoCapture(0);


    // Used for logging success or failure messages
    private static final String TAG = "OCVSample::Activity";

    // Loads camera view of OpenCV for us to use. This lets us see using OpenCV
    private CameraBridgeViewBase mOpenCvCameraView;

    // Used in Camera selection from menu (when implemented)
    private boolean              mIsJavaCamera = true;
    private MenuItem             mItemSwitchCamera = null;

    // These variables are used (at the moment) to fix camera orientation from 270degree to 0degree
    Mat mRgba;
    Mat mRgbaF;
    Mat mRgbaT;

    private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
        @Override
        public void onManagerConnected(int status) {
            switch (status) {
                case LoaderCallbackInterface.SUCCESS:
                {
                    Log.i(TAG, "OpenCV loaded successfully");
                    mOpenCvCameraView.enableView();
                } break;
                default:
                {
                    super.onManagerConnected(status);
                } break;
            }
        }
    };

    public MainActivity_show_camera() {
        Log.i(TAG, "Instantiated new " + this.getClass());
    }



    @Override
    protected void onCreate(Bundle savedInstanceState) {
        Log.i(TAG, "called onCreate");
        super.onCreate(savedInstanceState);
        getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);

        setContentView(R.layout.show_camera_activity_java_surface_view);

        mOpenCvCameraView = (JavaCameraView) findViewById(R.id.show_camera_activity_java_surface_view);

        mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);

        mOpenCvCameraView.setCvCameraViewListener(this);
    }

    @Override
    public void onPause()
    {
        super.onPause();
        if (mOpenCvCameraView != null)
            mOpenCvCameraView.disableView();
    }

    @Override
    public void onResume()
    {
        super.onResume();
        if (!OpenCVLoader.initDebug()) {
            Log.d(TAG, "Internal OpenCV library not found. Using OpenCV Manager for initialization");
            OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_0_0, this, mLoaderCallback);
        } else {
            Log.d(TAG, "OpenCV library found inside package. Using it!");
            mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);
        }
    }

    public void onDestroy() {
        super.onDestroy();
        if (mOpenCvCameraView != null)
            mOpenCvCameraView.disableView();
    }

    public void onCameraViewStarted(int width, int height) {

        mRgba = new Mat(height, width, CvType.CV_8UC4);
        mRgbaF = new Mat(height, width, CvType.CV_8UC4);
        mRgbaT = new Mat(width, width, CvType.CV_8UC4);
    }

    public void onCameraViewStopped() {
        mRgba.release();
    }

    public Mat onCameraFrame(CvCameraViewFrame inputFrame) {

        // TODO Auto-generated method stub
        mRgba = inputFrame.rgba();
        // Rotate mRgba 90 degrees




        return mRgba; // This function must return
    }
}

hanban

and manifest file:


<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.example.david.opencv_test">

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:supportsRtl="true"
        android:theme="@style/AppTheme"
        >
        <activity
            android:name=".MainActivity_show_camera"
            android:label="@string/app_name"

            android:screenOrientation="portrait">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>

    <supports-screens   android:resizeable="true"
        android:smallScreens="true"
        android:normalScreens="true"
        android:largeScreens="true"
        android:anyDensity="true" />

    <uses-permission android:name="android.permission.CAMERA"/>
    <uses-feature android:name="android.hardware.camera" android:required="false"/>
    <uses-feature android:name="android.hardware.camera.autofocus" android:required="false"/>
    <uses-feature android:name="android.hardware.camera.front" android:required="false"/>
    <uses-feature android:name="android.hardware.camera.front.autofocus" android:required="false"/>

</manifest>

hanban

this is the result: HDMI input in firefly RK 3288 reload. very low FPS and bad rtesolution (this is windows desktop  from my PC going into the HDMI input and displayed on the TV). any ideas?

[img]%5Burl=https://ibb.co/fYPkSa%5D[/url] high quality image hosting[/img]

yyangss@163.com

查看源码：

    is_video = (((preview_w == 1920) && (preview_h == 1080)) ||
                ((preview_w == 1280) && (preview_h == 720)));

if (is_video) {
            resReq.request_fps = 20;
        } else if (is_capture) {
            resReq.request_fps = 0;
            resReq.request_exp_t = curExp*curGain;
        } else {
            resReq.request_fps = 10;
            resReq.request_exp_t = curExp;
        }

所以，HAL层已经限定死了，不明白为什么这么写？而且为什么限定1920x1080和1280x720时才能达到fps20？是硬件有限制吗？

mike-dabkick

Hi Hanban, were you able to boost your framerate or make any improvement?  I'm just getting started on something similar and on the look-out for any tips or gotchas that folks with experience can provide.

Thanks.