Firefly Open Source Community

Title: Hardware decoder on linux for firefly-rk3288(source opened) [Print This Page]
Author: isle    Time: 12/30/2015 10:37
Title: Hardware decoder on linux for firefly-rk3288(source opened)
Last edited by isle In 12/30/2015 10:40 Editor

This is how to add hardware decoder to ffmpeg on linux for firefly-rk3288.

First, choose the vpu driver librarys.
We use the hybris library made by mac_l1,also you can choose rockchip's vpu driver library ,which they has put it on a websites.
For convenience ,I put a mac_l1's hybris lib's copy to "baidu pan". here is the link:
http://pan.baidu.com/s/1pKdqUQR


Install the driver library:
  1. sudo dpkg  -i machybris-0.1.0_armhf.deb
Copy the code
if you get a mistake. please run this:
  1. sudo apt-get -f install
Copy the code

and try again.

After install done ,reboot your board .

Install dependencies library :
  1. sudo apt-get install libsdl-dev
Copy the code

this source code need compiled on the development board. if you want to compiled on PC ,you need modify the source code.

Install compile environment:
  1. sudo apt-get install build-essential
Copy the code
Get the source code:
  1. git clone https://github.com/isle0118/firefly-vpu-linux.git
Copy the code
Compile and Install:
  1. ./configure  --enable-shared --prefix=/usr --disable-ffmpeg --disable-ffprobe --disable-ffserver  --disable-static
Copy the code
  1. make && sudo make install
Copy the code

Now, you can test the hardware decoder with ffplay .

the source file is here :
  1. libavcodec/rkdec_decoder.c
Copy the code

if you want use other driver library or compiled on PC ,just need to modify this file.


Have fun !



Author: mac_l1    Time: 12/30/2015 13:11
Wow! I am impressed!

This opens up the way to get hw decoding in chromium, mplayer and vlc, as they all use ffmpeg/ avcodec as base.
Maybe we can even play netflix on chromium!

Also a new version of machybris (0.1.1) can be downloaded here: http://www.mac-l1.com/v0.1.1/mac ... 4.2-0.1.1_armhf.deb

Great work, isle!
Author: isle    Time: 12/31/2015 09:43
your machybris is AmazingĦ£we can do many things on it.

we really appreciate all your work!
Author: 346409941    Time: 1/3/2016 11:05
Hi mac_l1:
can you share the method that how to use GPU for encoding?
Author: mac_l1    Time: 1/3/2016 14:11
346409941 Posted at 1/3/2016 11:05
Hi mac_l1:
can you share the method that how to use GPU for encoding?

Hi,
See: https://github.com/mac-l1/machyb ... 64recorder.cpp#L158.
I used this code to capture the screen into a h264-movie using vpu.
This can probably quite easily be added into the shared ffmpeg / avcodec code above from isle.
Author: fd101283    Time: 1/5/2016 22:40
Hi,

I get an error : librk_on2.so and libvpu.so are missing !

I think a package is missing....
Author: fd101283    Time: 1/5/2016 23:11
OK it's solved with just a reboot .....
Do you know why it's not working with 4K video ?
Author: isle    Time: 1/6/2016 09:34
the vpu can decode 4K video smoothly. But the display was terrible.
so we didn't add 4k support here.
Author: fd101283    Time: 1/6/2016 16:09
Oh bad news.....
It's not possible to add 4K support (maybe with display improvement) ?

I think that the main advantage of this chip is 4K support.

Author: 346409941    Time: 1/6/2016 19:16
Hi mac_l1:
can you give me screenrecoderh264 for testing HW encoder
Author: 346409941    Time: 1/6/2016 19:19
mac_l1 Posted at 1/3/2016 14:11
Hi,
See: https://github.com/mac-l1/machybris_examples/blob/master/recorder/h264recorder.cpp#L158. ...

Hi mac_l1:
can you give me screenrecoderh264 for testing HW encoder
Author: mac_l1    Time: 1/6/2016 23:24
Attached zip file with two files in it (one binary, one shell script).
Use the script to capture the screen, the recording is stopped by pressing a key:
  1. ./screenrecord_h264.sh test.mp4
Copy the code
You should have installed machybris before, and also avconv (sudo apt-get install libav-tools).

Every frame is captured from framebuffer device /dev/fb0. Using shared memory this RGB frame is converted to YUV420SP / NV21 format using the RGA hardware module. This YUV frame is then given to the encoder using the raw vpu api that generates an h264 bitstream. After key-press this h264 bitstream file is then encapsulated into a movie-container (mp4 in this case) by avconv utility.

You can see some android logging by:
  1. sudo /system/bin/logcat
Copy the code
You can see some kernel logging by:
  1. sudo tail -f /var/log/syslog
Copy the code
Hope this helps.


Cheers -- mac_l1


h264recorder.zip

45.93 KB, Down times: 26
Author: fd101283    Time: 1/6/2016 23:40
@mac_l1, @isle,

Do you think that it's possible to add 4K support (maybe with display improvement) ?
I can't see in sourcecode where 4K is forbidden ! Can you show me where i can find such thing ?

Author: isle    Time: 1/7/2016 09:49
Last edited by isle In 1/7/2016 09:51 Editor

Hi fd101283.
the 4k video  hasn't be  forbbidened on this sourcecode.  
but when playing 4k video ,the ffplay sometimes get rong video width&height .like "avctx->width=0 & avctx->height=0";in the initializtion function "rkdec_prepare".
so you need give them the width&height  value before "rkdec_ctx->ctx->init".
after init is done ,the decoder works. Now your can get the real width&height "pframe->DisplayWidth" by "rkdec_ctx->vpu_mem_link(&pframe->vpumem);".
this is what I get. when i playing 4k videos.

If someone can make the display woking well on 4k.that must be excellent.
But if you just need deocode with 4k , display with HD or FHD. you can use rga to do downscale.thay is a way to get 4k woks.
haha!



Author: 346409941    Time: 1/7/2016 15:54
mac_l1 Posted at 1/6/2016 23:24
Attached zip file with two files in it (one binary, one shell script).
Use the script to capture th ...

thank you for your help!
screenrecord_h264.sh called bin file h264recorder, Where can I get the source code of h264recorder?

thanks
Author: mac_l1    Time: 1/7/2016 15:59
346409941 Posted at 1/7/2016 15:54
thank you for your help!
screenrecord_h264.sh called bin file h264recorder, Where can I get the s ...

Here is the source code:
  1. git clone https://github.com/mac-l1/machybris_examples.git
Copy the code

Look in the recorder folder for the screenrecorder.
Cheers! mac_l1
Author: fd101283    Time: 1/7/2016 18:31
Hi Isle,

Thanks alot for the explanation...It's very great job that you've done.

I try to modify source code in order to decode 4K. As a result i always get errors before decoding. This is the log :

  1. Firefly@firefly:~/Downloads/firefly-vpu-linux$ ./ffplay Astra_Canal_UHD.ts
  2. ffplay version 2.6.3 Copyright (c) 2003-2015 the FFmpeg developers
  3.   built with gcc 4.8 (Ubuntu/Linaro 4.8.4-2ubuntu1~14.04)
  4.   configuration: --enable-shared --prefix=/usr --disable-ffmpeg --disable-ffprobe --disable-ffserver --disable-static
  5.   libavutil      54. 20.100 / 54. 20.100
  6.   libavcodec     56. 26.100 / 56. 26.100
  7.   libavformat    56. 25.101 / 56. 25.101
  8.   libavdevice    56.  4.100 / 56.  4.100
  9.   libavfilter     5. 11.102 /  5. 11.102
  10.   libswscale      3.  1.101 /  3.  1.101
  11.   libswresample   1.  1.100 /  1.  1.100
  12. [hevc @ 0xb3922dc0] nb threads_frame = 1, nb threads_slice 1, thread_type = null
  13. [hevc @ 0xb3908220] nb threads_frame = 1, nb threads_slice 1, thread_type = null
  14. [hevc @ 0xb3922880] nb threads_frame = 1, nb threads_slice 1, thread_type = null
  15. [hevc_rkvpu @ 0xb39044a0] PPS id out of range: 0
  16. [hevc @ 0xb3908220] PPS id out of range: 0
  17. [hevc @ 0xb3908220] Error parsing NAL unit #2.
  18. Error while decoding frame
  19. get frame failed ret -1004
  20. Something wrong during decode!!!
  21. get frame failed ret -1004
  22. Something wrong during decode!!!
  23. get frame failed ret -1004
  24. Something wrong during decode!!!
  25. get frame failed ret -1004
  26. Something wrong during decode!!!
  27.     Last message repeated 2 times
  28. get frame failed ret -10040 aq=    0KB vq=    0KB sq=    0B f=0/0   
  29. Something wrong during decode!!!
  30. get frame failed ret -1004
  31. Something wrong during decode!!!
  32. get frame failed ret -1004
  33. Something wrong during decode!!!
  34. get frame failed ret -1004
  35. Something wrong during decode!!!
  36. get frame failed ret -1004
  37. Something wrong during decode!!!
  38. get frame failed ret -1004
  39. Something wrong during decode!!!
  40. get frame failed ret -1004
  41. Something wrong during decode!!!
  42. get frame failed ret -1004
  43. Something wrong during decode!!!
  44. get frame failed ret -1004
  45. Something wrong during decode!!!
  46. get frame failed ret -1004
  47. Something wrong during decode!!!
Copy the code

Author: isle    Time: 1/8/2016 09:25
Oh,
"[hevc_rkvpu @ 0xb39044a0] PPS id out of range: 0"
This is very strange.
have you try another 4k video ?and get the some mistake?  
Author: fd101283    Time: 1/8/2016 16:20
Yes, the same stream works well with android player...
And i also try with some other stream and it give the same problem...
Author: fd101283    Time: 1/8/2016 20:55
This is the stream which gives problems :

http://www.filedropper.com/astracanaluhd
Author: fd101283    Time: 1/11/2016 21:47
Hi @Isle, @Mac_L1 and Everybody,

I try to debug ffmpeg but without success

I can't debug in libs like livavcodec. Can you explain me step by step what i should do to eable this debug please ?

Thanks in advance for your help.
Author: fd101283    Time: 1/12/2016 21:32
OK I can debug libs ....

But it's very hard.... and I saw nothing
Author: fd101283    Time: 1/14/2016 20:52
i can't find why this PPS id is bad. I try to download last ffmpeg sources... and same problem.

On top of that if i try to play other HEVC file, i get a screen with bad colours like in this screenshot :

Author: mac_l1    Time: 1/15/2016 17:13
fd101283 Posted at 1/14/2016 20:52
i can't find why this PPS id is bad. I try to download last ffmpeg sources... and same problem.
...
just to check: does kodi (installed by mackodi) play the movie correctly?
about the colors: i guess it has to do with some special alignment for rockchip hevc vpu frames (stride and slice height). see my post on freaktab: http://freaktab.com/forum/tv-player-support/rk3288-devices/530835-how-to-realize-rk3288-hevc-h265-hw-decoder-under-linux?p=536311#post536311



Author: fd101283    Time: 1/18/2016 17:50
Hi mac_l1,

Really happy to read you.....

In fact mackodi gives the same colours problem.

Author: fd101283    Time: 1/19/2016 00:29
Maybe it's a clue: Preview in KODI (little screen) is OK. (colours are good)
Author: mac_l1    Time: 1/19/2016 05:16
Ok... Have you installed dual boot  so you can also check with android kodi or other player how the output is?
Or maybe you can send me a link to the movie file so i can check for myself whats  wrong?
Author: fd101283    Time: 1/19/2016 16:39
I have single boot with Linux on  eMMC. But when i try with Android (on SD card), it's working perfectly!
The best is to download the file on the original website :

http://demo-uhd3d.com/fiche.php?cat=uhd&id=125

You can find other sample on this website.....
Thanks in advance for your help ....
Author: mac_l1    Time: 1/19/2016 17:07
fd101283 Posted at 1/19/2016 16:39
I have single boot with Linux on  eMMC. But when i try with Android (on SD card), it's working perfe ...

When playing on Android: is that on lollipop or kitkat (as they have different video libs)? and is that using kodi? what kodi version?
Author: fd101283    Time: 1/19/2016 18:25
Android 4.4.2.
Using original player (not KODI)
Author: fd101283    Time: 1/19/2016 23:28
isle Posted at 1/7/2016 09:49
Hi fd101283.
the 4k video  hasn't be  forbbidened on this sourcecode.  
but when playing 4k video  ...

There is the same problem with HD resolution....
avctx->width & height are wrong before init.


Author: fd101283    Time: 1/25/2016 16:46
Hi mac_l1,

Have you find something wrong with my file.
From my hand, i found nothing...
Author: fd101283    Time: 2/15/2016 17:16
UP !!



Author: newold    Time: 3/16/2016 19:04
Hi isle.
can this source be used on fireprime board(RK3128 chip)?
Author: isle    Time: 4/22/2016 17:42
newold Posted at 3/16/2016 19:04
Hi isle.
can this source be used on fireprime board(RK3128 chip)?

yes ,I think it can be used on fireprime .what you need to do is just get mac's hybris lib for rk3128.
Author: newold    Time: 4/25/2016 19:44
Last edited by newold In 4/25/2016 21:36 Editor
root@firefly:/home/firefly/firefly-vpu-linux# ./ffplay
./ffplay: relocation error: /usr/lib/libavfilter.so.5: symbol avpriv_vga16_font, version LIBAVUTIL_54 not defined in file libavutiime reference

Hi, isle

Thank you for your reply.
I have applied the mac's hybris lib for rk3128.
However, when i run command(./ffplay) occurs error.

For reference, my firmware was used for the image received from the firefly website.
(FirePrime_Ubuntu15.04_201603030939.img)
Author: Bokrenok    Time: 6/25/2016 14:54
Hi, isle.
As newold I've tried to run ffplay on fireprime, everything looks good except one thing:
it can't find "librk_on2.so" and then says "Failed to open: RK_DEC_LIB, init declib failed"

What can I do with it? Where can I get librk_on2.so?
Author: Bokrenok    Time: 6/25/2016 15:01
Last edited by Bokrenok In 6/25/2016 15:03 Editor

Hi, newold.
I've just solved the same problem.

The metter is that there are already some libs for ffmpeg in the firmware located in:
/usr/lib/arm-linux-gnueabihf/
/usr/lib/arm-linux-gnueabihf/vfp
/usr/lib/arm-linux-gnueabihf/neon

And compiled ffplay installed libs in /usr/lib/, but tries to load libs from "/usr/lib/arm-linux-gnueabihf/neon" and "/usr/lib/arm-linux-gnueabihf"

You should remove (or move anywhere) default libs which present in those directories.

Here is the list (for each mentioned directory):

libavcodec.so.56.1.0
libavdevice.so.55
libavdevice.so.55.0.0
libavfilter.so.5
libavfilter.so.5.0.0
libavformat.so.56
libavformat.so.56.1.0
libavresample.so.2
libavresample.so.2.1.0
libavutil.so.54
libavutil.so.54.3.0
libswscale.so.3
libswscale.so.3.0.0

But there is one more problem: ffplay can't find "librk_on2.so" and I didn't find a solution yet.

Author: Bokrenok    Time: 6/26/2016 02:51
well, eventually I managed to install and run kodi on fireprime-rk3128
instructions are here: http://freaktab.com/forum/tv-pla ... or-rockchip-devices

many thanks to mac_l1 for his great job!



Welcome Firefly Open Source Community (https://bbs.t-firefly.com/) Powered by Discuz! X3.1