Firefly Open Source Community

Title: Question for FireFly Developers: HDMI input and openCV [Print This Page]

Author: hanban Time: 5/22/2017 01:48
Title: Question for FireFly Developers: HDMI input and openCV
Last edited by hanban In 5/22/2017 01:51 Editor

Hello, I need to know if it is possible to grab frames coming from the Firefly RK3288 reload HDMI input , to use in OpenCV.

I know HDMI input does not work under linux, but I am developing in android studio to use on the FireFly android OS.

I need to grab 1080p HDMI from a camera going into the HDMI input port and display it with openCV.

is it possible? any SDK to do it?
in openCV , cap = cv2.VideoCapture('device index') , i need to know how to make it read from the HDMI input.

Please help!

thanks!

Author: isle Time: 5/22/2017 09:43
Hi ,yes It is possible,
In Android, It is a device of camera, so, see the class of android.hardware.camera2

Author: hanban Time: 5/22/2017 15:41
Last edited by hanban In 5/22/2017 15:46 Editor

isle Posted at 5/22/2017 09:43
Hi ,yes It is possible,
In Android, It is a device of camera, so, see the class of android.hardware ...

Hello isle, good to know that it works! So I import android.hardware classes to android studio ?
Is there any sample code I might look at to understand how to capture the frames?
If it is recognized as a camera device, will this work with no modifications?
int main(int, char**)
{
VideoCapture cap(0); // open the default camera
if(!cap.isOpened()) // check if we succeeded
return -1;

Author: isle Time: 5/23/2017 09:35
there is a demo in github : https://github.com/pinguo-yuyidong/Camera2

Author: hanban Time: 5/23/2017 17:14
Last edited by hanban In 5/23/2017 17:19 Editor

isle Posted at 5/23/2017 09:35
there is a demo in github : https://github.com/pinguo-yuyidong/Camera2

I need help, i got it working but the frame rate is very slow (10-20 fps). I need atleast constant 30 fps and above. can you look at this and help me out? thanks!!
i ad 2 posts witrh my code ands third posat with a pictue of the result!

package com.example.david.opencv_test;

import android.os.Bundle;
import android.support.v7.app.AppCompatActivity;
import android.util.Log;
import android.view.MenuItem;
import android.view.SurfaceView;
import android.view.WindowManager;

import org.opencv.android.JavaCameraView;
import org.opencv.android.BaseLoaderCallback;
import org.opencv.android.CameraBridgeViewBase;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewFrame;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewListener2;
import org.opencv.android.LoaderCallbackInterface;
import org.opencv.android.OpenCVLoader;
import org.opencv.core.Core;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
import org.opencv.imgproc.Imgproc;
import org.opencv.videoio.VideoCapture;

// OpenCV Classes

public class MainActivity_show_camera extends AppCompatActivity implements CvCameraViewListener2 {
VideoCapture vcap = new VideoCapture(0);

// Used for logging success or failure messages
private static final String TAG = "OCVSample::Activity";

// Loads camera view of OpenCV for us to use. This lets us see using OpenCV
private CameraBridgeViewBase mOpenCvCameraView;

// Used in Camera selection from menu (when implemented)
private boolean             mIsJavaCamera = true;
private MenuItem          mItemSwitchCamera = null;

// These variables are used (at the moment) to fix camera orientation from 270degree to 0degree
Mat mRgba;
Mat mRgbaF;
Mat mRgbaT;

private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
      @Override
      public void onManagerConnected(int status) {
         switch (status) {
            case LoaderCallbackInterface.SUCCESS:
            {
                  Log.i(TAG, "OpenCV loaded successfully");
                  mOpenCvCameraView.enableView();
            } break;
            default:
            {
                  super.onManagerConnected(status);
            } break;
         }
      }
};

public MainActivity_show_camera() {
      Log.i(TAG, "Instantiated new " + this.getClass());
}

@Override
protected void onCreate(Bundle savedInstanceState) {
      Log.i(TAG, "called onCreate");
      super.onCreate(savedInstanceState);
      getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);

      setContentView(R.layout.show_camera_activity_java_surface_view);

      mOpenCvCameraView = (JavaCameraView) findViewById(R.id.show_camera_activity_java_surface_view);

      mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);

      mOpenCvCameraView.setCvCameraViewListener(this);
}

@Override
public void onPause()
{
      super.onPause();
      if (mOpenCvCameraView != null)
         mOpenCvCameraView.disableView();
}

@Override
public void onResume()
{
      super.onResume();
      if (!OpenCVLoader.initDebug()) {
         Log.d(TAG, "Internal OpenCV library not found. Using OpenCV Manager for initialization");
         OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_0_0, this, mLoaderCallback);
      } else {
         Log.d(TAG, "OpenCV library found inside package. Using it!");
         mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);
      }
}

public void onDestroy() {
      super.onDestroy();
      if (mOpenCvCameraView != null)
         mOpenCvCameraView.disableView();
}

public void onCameraViewStarted(int width, int height) {

      mRgba = new Mat(height, width, CvType.CV_8UC4);
      mRgbaF = new Mat(height, width, CvType.CV_8UC4);
      mRgbaT = new Mat(width, width, CvType.CV_8UC4);
}

public void onCameraViewStopped() {
      mRgba.release();
}

public Mat onCameraFrame(CvCameraViewFrame inputFrame) {

      // TODO Auto-generated method stub
      mRgba = inputFrame.rgba();
      // Rotate mRgba 90 degrees

      return mRgba; // This function must return
}
}

Author: hanban Time: 5/23/2017 17:15
and manifest file:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.example.david.opencv_test">

<application
      android:allowBackup="true"
      android:icon="@mipmap/ic_launcher"
      android:label="@string/app_name"
      android:supportsRtl="true"
      android:theme="@style/AppTheme"
      >
      <activity
         android:name=".MainActivity_show_camera"
         android:label="@string/app_name"

         android:screenOrientation="portrait">
         <intent-filter>
            <action android:name="android.intent.action.MAIN" />

            <category android:name="android.intent.category.LAUNCHER" />
         </intent-filter>
      </activity>
</application>

<supports-screens android:resizeable="true"
      android:smallScreens="true"
      android:normalScreens="true"
      android:largeScreens="true"
      android:anyDensity="true" />

<uses-permission android:name="android.permission.CAMERA"/>
<uses-feature android:name="android.hardware.camera" android:required="false"/>
<uses-feature android:name="android.hardware.camera.autofocus" android:required="false"/>
<uses-feature android:name="android.hardware.camera.front" android:required="false"/>
<uses-feature android:name="android.hardware.camera.front.autofocus" android:required="false"/>

</manifest>

Author: hanban Time: 5/23/2017 17:18
Last edited by hanban In 5/23/2017 17:22 Editor

this is the result: HDMI input in firefly RK 3288 reload. very low FPS and bad rtesolution (this is windows desktop from my PC going into the HDMI input and displayed on the TV). any ideas?

[img]%5Burl=https://ibb.co/fYPkSa%5D

[/url] high quality image hosting[/img]

Author: yyangss@163.com Time: 7/6/2017 15:59
查看源码：

is_video = (((preview_w == 1920) && (preview_h == 1080)) ||
            ((preview_w == 1280) && (preview_h == 720)));

if (is_video) {
         resReq.request_fps = 20;
      } else if (is_capture) {
         resReq.request_fps = 0;
         resReq.request_exp_t = curExp*curGain;
      } else {
         resReq.request_fps = 10;
         resReq.request_exp_t = curExp;
      }

所以，HAL层已经限定死了，不明白为什么这么写？而且为什么限定1920x1080和1280x720时才能达到fps20？是硬件有限制吗？

Author: mike-dabkick Time: 8/6/2017 10:48
Hi Hanban, were you able to boost your framerate or make any improvement? I'm just getting started on something similar and on the look-out for any tips or gotchas that folks with experience can provide.

Thanks.

Welcome Firefly Open Source Community (https://bbs.t-firefly.com/)