Firefly Open Source Community

Title: Using imx219 camera with thunder boot [Print This Page]
Author: Fuzzy1    Time: 4/28/2022 09:10
Title: Using imx219 camera with thunder boot
Last edited by Fuzzy1 In 5/8/2022 20:19 Editor

I have a camera with a imx219 sensor that is connected to the mother board (JD4-
RV11091126) with Core-1126-JD4 and I have managed to capture frames in normal mode (build file aio-rv1126-jd4-BE-45.mk).

When I enable thunder boot with aio-rv1126-jd4-BE-45-tb-v11.mk no frames are captured.
How to fix this?


  1. grep '' /sys/class/video4linux/video*/name
  2. /sys/class/video4linux/video0/name:rkisp_rawrd0_m
  3. /sys/class/video4linux/video1/name:rkisp_rawrd1_l
  4. /sys/class/video4linux/video10/name:rkisp_rawrd2_s
  5. /sys/class/video4linux/video11/name:rkisp-mipi-luma
  6. /sys/class/video4linux/video12/name:stream_cif_mipi_id0
  7. /sys/class/video4linux/video13/name:stream_cif_mipi_id1
  8. /sys/class/video4linux/video14/name:stream_cif_mipi_id2
  9. /sys/class/video4linux/video15/name:stream_cif_mipi_id3
  10. /sys/class/video4linux/video16/name:rkcif-mipi-luma
  11. /sys/class/video4linux/video2/name:rkisp-statistics
  12. /sys/class/video4linux/video3/name:rkisp_mainpath
  13. /sys/class/video4linux/video4/name:rkisp_selfpath
  14. /sys/class/video4linux/video5/name:rkisp_rawwr0
  15. /sys/class/video4linux/video6/name:rkisp_rawwr1
  16. /sys/class/video4linux/video7/name:rkisp-input-params
  17. /sys/class/video4linux/video8/name:rkisp_rawwr2
  18. /sys/class/video4linux/video9/name:rkisp_rawwr3
Copy the code


  1. v4l2-ctl -d /dev/video4 --set-fmt-video=width=640,height=480,pixelformat=NV12
  2. --stream-mmap=3 --stream-skip=3 --stream-to=/tmp/nv12.bin --stream-count=100 --s
  3. tream-poll
  4. select timeout
Copy the code



  1. [   41.913678] rkisp0: rkisp_isp_sd_s_power on:1
  2. [   41.914059] videodev: v4l2_open: video6: open (0)
  3. [   41.914137] video6: VIDIOC_QUERYCAP: driver=rkisp_v4, card=rkisp_selfpath, bus=platform:rkisp-vir0, version=0x0004136f, capabilities=0x84201000, device_caps=0x04201000
  4. [   41.914167] video6: VIDIOC_QUERY_EXT_CTRL: id=0x980001, type=6, name=User Controls, min/max=0/0, step=0, default=0, flags=0x00000044, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
  5. [   41.914185] video6: VIDIOC_TRY_EXT_CTRLS: which=0x0, count=0, error_idx=0
  6. [   41.914198] video6: VIDIOC_QUERYCTRL: id=0x980001, type=6, name=User Controls, min/max=0/0, step=0, default=0, flags=0x00000044
  7. [   41.914211] video6: VIDIOC_G_SELECTION: type=vid-cap, target=0, flags=0x0, wxh=1920x1080, x,y=0,0
  8. [   41.914224] video6: VIDIOC_QUERYCAP: driver=rkisp_v4, card=rkisp_selfpath, bus=platform:rkisp-vir0, version=0x0004136f, capabilities=0x84201000, device_caps=0x04201000
  9. [   41.914465] video6: VIDIOC_QUERY_EXT_CTRL: id=0x980001, type=6, name=User Controls, min/max=0/0, step=0, default=0, flags=0x00000044, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
  10. [   41.914486] video6: VIDIOC_QUERY_EXT_CTRL: id=0x980001, type=6, name=User Controls, min/max=0/0, step=0, default=0, flags=0x00000044, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
  11. [   41.914502] video6: VIDIOC_QUERY_EXT_CTRL: id=0x980911, type=1, name=Exposure, min/max=0/4095, step=1, default=1575, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
  12. [   41.914563] video6: VIDIOC_QUERY_EXT_CTRL: id=0x980913, type=1, name=Gain, min/max=256/43663, step=1, default=256, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
  13. [   41.914589] video6: VIDIOC_QUERY_EXT_CTRL: id=0x980914, type=2, name=Horizontal Flip, min/max=0/1, step=1, default=0, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
  14. [   41.914618] video6: VIDIOC_QUERY_EXT_CTRL: id=0x980915, type=2, name=Vertical Flip, min/max=0/1, step=1, default=0, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
  15. [   41.914642] video6: VIDIOC_QUERY_EXT_CTRL: id=0x9e0001, type=6, name=Image Source Controls, min/max=0/0, step=0, default=0, flags=0x00000044, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
  16. [   41.914658] video6: VIDIOC_QUERY_EXT_CTRL: id=0x9e0901, type=1, name=Vertical Blanking, min/max=686/686, step=1, default=686, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
  17. [   41.914723] video6: VIDIOC_QUERY_EXT_CTRL: id=0x9e0902, type=1, name=Horizontal Blanking, min/max=1524/1524, step=1, default=1524, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
  18. [   41.914752] video6: VIDIOC_QUERY_EXT_CTRL: id=0x9e0903, type=1, name=Analogue Gain, min/max=256/2816, step=1, default=512, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
  19. [   41.914776] video6: VIDIOC_QUERY_EXT_CTRL: id=0x9f0001, type=6, name=Image Processing Controls, min/max=0/0, step=0, default=0, flags=0x00000044, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
  20. [   41.914792] video6: VIDIOC_QUERY_EXT_CTRL: id=0x9f0901, type=9, name=Link Frequency, min/max=0/0, step=1, default=0, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
  21. [   41.914816] video6: VIDIOC_QUERY_EXT_CTRL: id=0x9f0902, type=5, name=Pixel Rate, min/max=0/182463120, step=1, default=182463120, flags=0x00000004, elem_size=8, elems=1, nr_of_dims=0, dims=0,0,0,0
  22. [   41.914839] video6: VIDIOC_QUERY_EXT_CTRL: id=0x9f0903, type=3, name=Test Pattern, min/max=0/13, step=1, default=0, flags=0x00000000, elem_size=4, elems=1, nr_of_dims=0, dims=0,0,0,0
  23. [   41.914862] video6: VIDIOC_QUERY_EXT_CTRL: error -22: id=0xc09f0903, type=0, name=, min/max=0/0, step=0, default=0, flags=0x00000000, elem_size=0, elems=0, nr_of_dims=0, dims=0,0,0,0
  24. [   41.914884] video6: VIDIOC_G_FMT: type=vid-cap-mplane, width=1920, height=1080, format=NV12, field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=1, xfer_func=0
  25. [   41.914962] plane 0: bytesperline=1920 sizeimage=3110400
  26. [   41.914982] rkisp0: rkisp_set_fmt: stream: 1 req(640, 480) out(640, 480)
  27. [   41.914984] video6: VIDIOC_S_FMT: type=vid-cap-mplane, width=640, height=480, format=NV12, field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=1, xfer_func=0
  28. [   41.914996] plane 0: bytesperline=640 sizeimage=460800
  29. [   41.915072] video6: VIDIOC_SUBSCRIBE_EVENT: error -25: type=0x2, id=0x0, flags=0x0
  30. [   41.915090] video6: VIDIOC_SUBSCRIBE_EVENT: error -25: type=0x5, id=0x0, flags=0x0
  31. [   41.915101] video6: VIDIOC_G_INPUT: error -25: value=0
  32. [   41.915252] rkisp0: vid-cap-mplane count 3, size 460800
  33. [   41.915308] rkisp0: rkisp_alloc_buffer buf:0xc804000~0xcaa7000 size:2764800
  34. [   41.915316] rkisp0: rkisp_alloc_common_dummy_buf buf:0xc804000 size:2764800
  35. [   41.915336] videobuf2_common: __setup_offsets: buffer 0, plane 0 offset 0x00000000
  36. [   41.915349] videobuf2_common: __setup_offsets: buffer 1, plane 0 offset 0x00071000
  37. [   41.915360] videobuf2_common: __setup_offsets: buffer 2, plane 0 offset 0x000e2000
  38. [   41.915364] videobuf2_common: __vb2_queue_alloc: allocated 3 buffers, 1 plane(s) each
  39. [   41.915369] video6: VIDIOC_REQBUFS: count=3, type=vid-cap-mplane, memory=mmap
  40. [   41.915400] video6: VIDIOC_QUERYBUF: 00:00:00.00000000 index=0, type=vid-cap-mplane, flags=0x00002000, field=any, sequence=0, memory=mmap
  41. [   41.915417] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x0, length=460800
  42. [   41.915422] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
  43. [   41.915435] video6: VIDIOC_QUERYBUF: 00:00:00.00000000 index=1, type=vid-cap-mplane, flags=0x00002000, field=any, sequence=0, memory=mmap
  44. [   41.915449] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x71000, length=460800
  45. [   41.915453] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
  46. [   41.915460] video6: VIDIOC_QUERYBUF: 00:00:00.00000000 index=2, type=vid-cap-mplane, flags=0x00002000, field=any, sequence=0, memory=mmap
  47. [   41.915473] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0xe2000, length=460800
  48. [   41.915478] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
  49. [   41.915593] videobuf2_common: vb2_mmap: buffer 0, plane 0 successfully mapped
  50. [   41.915597] videodev: v4l2_mmap: video6: mmap (0)
  51. [   41.915674] videobuf2_common: vb2_mmap: buffer 1, plane 0 successfully mapped
  52. [   41.915677] videodev: v4l2_mmap: video6: mmap (0)
  53. [   41.915747] videobuf2_common: vb2_mmap: buffer 2, plane 0 successfully mapped
  54. [   41.915750] videodev: v4l2_mmap: video6: mmap (0)
  55. [   41.915819] videobuf2_common: vb2_core_qbuf: qbuf of buffer 0 succeeded
  56. [   41.915821] video6: VIDIOC_QBUF: 00:00:00.00000000 index=0, type=vid-cap-mplane, flags=0x00002003, field=any, sequence=0, memory=mmap
  57. [   41.915834] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x0, length=460800
  58. [   41.915839] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
  59. [   41.915897] videobuf2_common: vb2_core_qbuf: qbuf of buffer 1 succeeded
  60. [   41.915900] video6: VIDIOC_QBUF: 00:00:00.00000000 index=1, type=vid-cap-mplane, flags=0x00002003, field=any, sequence=0, memory=mmap
  61. [   41.915912] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0x71000, length=460800
  62. [   41.915917] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
  63. [   41.915976] videobuf2_common: vb2_core_qbuf: qbuf of buffer 2 succeeded
  64. [   41.915978] video6: VIDIOC_QBUF: 00:00:00.00000000 index=2, type=vid-cap-mplane, flags=0x00002003, field=any, sequence=0, memory=mmap
  65. [   41.915991] plane 0: bytesused=0, data_offset=0x00000000, offset/userptr=0xe2000, length=460800
  66. [   41.915995] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
  67. [   41.916003] video6: VIDIOC_G_FMT: type=vid-cap-mplane, width=640, height=480, format=NV12, field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=1, xfer_func=0
  68. [   41.916017] plane 0: bytesperline=640 sizeimage=460800
  69. [   41.916057] rkisp0: rkisp_start_streaming 1
  70. [   41.916076] rkisp0: CSI ch0 vc:0 dt:0x2b 1920x1080
  71. [   41.916081] rkisp0: CSI ch1 vc:0 dt:0x2b 1920x1080
  72. [   41.916085] rkisp0: CSI ch2 vc:0 dt:0x2b 1920x1080
  73. [   41.916089] rkisp0: CSI ch3 vc:0 dt:0x2b 1920x1080
  74. [   41.916094] rkisp0: CSI ch4 vc:0 dt:0x2b 1920x1080
  75. [   41.916192] rkisp0: CSI2RX_IDS 0x2b2b2b2b 0x0000002b
  76. [   41.916201] rkisp0: stream 1 rsz/scale: 1920x1080 -> 640x480
  77. [   41.916204] rkisp0: chroma scaling 960x1080 -> 320x240
  78. [   41.916245] RSZ_CTRL 0x0000000f/0x0000000f
  79. [   41.916245] RSZ_SCALE_HY 21823/21823
  80. [   41.916245] RSZ_SCALE_HCB 21800/21800
  81. [   41.916245] RSZ_SCALE_HCR 21800/21800
  82. [   41.916245] RSZ_SCALE_VY 29094/29094
  83. [   41.916245] RSZ_SCALE_VC 14517/14517
  84. [   41.916245] RSZ_PHASE_HY 0/0
  85. [   41.916245] RSZ_PHASE_HC 0/0
  86. [   41.916245] RSZ_PHASE_VY 0/0
  87. [   41.916245] RSZ_PHASE_VC 0/0
  88. [   41.916249] rkisp0: stream 1 crop disabled
  89. [   41.916275] rkisp0: CSI2RX CTRL0:0x20820000 CTRL1:0x1
  90. [   41.916275] WR CTRL RAW0:0x2e0200 RAW1:0x200 RAW2:0x100
  91. [   41.916275] RD CTRL:0x0
  92. [   41.916286] rkisp0: SP streaming = 1, MP streaming = 0
  93. [   41.916289] rkisp0: CIF_ID 0x02001909
  94. [   41.916307] rkisp rkisp-vir0: can not get first iq setting in stream on
  95. [   41.916320] rkisp0: SP streaming = 1, MP streaming = 0
  96. [   41.917675] rkisp0: SP streaming = 1, MP streaming = 0 MI_CTRL 0x193a4002
  97. [   41.917675]   ISP_CTRL 0x80446117 MIPI_CTRL 0x20820001
  98. [   41.917693] rockchip-mipi-dphy-rx ff4b8000.csi-dphy: stream on:1
  99. [   41.917705] rockchip-mipi-dphy-rx: data_rate_mbps 912
  100. [   41.921106] rockchip-mipi-dphy-rx ff4b8000.csi-dphy: stream on:0
  101. [   41.922155] rkisp0: SP streaming = 1, MP streaming = 0
  102. [   41.922167] rkisp0: streaming(MP:1, SP:0), MI_CTRL:193a4002, ISP_CTRL:80446106, MIPI_CTRL:20820001
  103. [   41.922363] rkisp0: rkisp_free_buffer buf:0xc804000~0xcaa7000
  104. [   41.922379] videobuf2_common: vb2_buffer_done: done processing on buffer 2, state: 3
  105. [   41.922384] videobuf2_common: vb2_buffer_done: done processing on buffer 0, state: 3
  106. [   41.922387] videobuf2_common: vb2_buffer_done: done processing on buffer 1, state: 3
  107. [   41.922391] videobuf2_common: vb2_start_streaming: driver refused to start streaming
  108. [   41.922395] video6: VIDIOC_STREAMON: error -5: type=vid-cap-mplane
  109. [   41.923565] videobuf2_common: __vb2_buf_mem_free: freed plane 0 of buffer 0
  110. [   41.923574] videobuf2_common: __vb2_buf_mem_free: freed plane 0 of buffer 1
  111. [   41.923583] videobuf2_common: __vb2_buf_mem_free: freed plane 0 of buffer 2
  112. [   41.923650] rkisp0: rkisp_isp_sd_s_power on:0
  113. [   41.923678] videodev: v4l2_release: video6: release
Copy the code



graph.png (62.53 KB, Down times: 101)

graph.png
Author: 799959745    Time: 4/28/2022 14:58
Does the camera work normally when it starts up?
test mk£º
  1. ./build.sh device/rockchip/rv1126_rv1109/aio-rv1126-jd4-BE-45.mk
Copy the code
Author: Fuzzy1    Time: 4/28/2022 16:15
799959745 Posted at 4/28/2022 14:58
Does the camera work normally when it starts up?
test mk£º

Yes, the camera is detected.

  1. [    0.454200] imx219 2-0010: Model ID 0x0219, Lot ID 0x3b2454, Chip ID 0x0633
  2. [    0.454286] rockchip-mipi-dphy-rx ff4b8000.csi-dphy: match m01_f_imx219 2-0010:bus type 4
Copy the code
Author: Fuzzy1    Time: 5/1/2022 20:48
Some more info:

  1. # gst-launch-1.0 v4l2src device=/dev/video1 num-buffers=100 ! video/x-raw,format
  2. =NV12,width=640,height=480,framerate=30/1 ! filesink location=/tmp/h264.bin
  3. Setting pipeline to PAUSED ...
  4. Using mplane plugin for capture
  5. Pipeline is live and does not need PREROLL ...
  6. Setting pipeline to PLAYING ...
  7. New clock: GstSystemClock
  8. libv4l2: error turning on stream: Input/output error
  9. ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
  10. Additional debug info:
  11. gstv4l2src.c(658): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
  12. Buffer pool activation failed
  13. Execution ended after 0:00:00.009998917
  14. Setting pipeline to PAUSED ...
  15. Setting pipeline to READY ...
  16. Setting pipeline to NULL ...
  17. Freeing pipeline ...
Copy the code

Author: Fuzzy1    Time: 5/2/2022 16:01
Last edited by Fuzzy1 In 5/2/2022 18:36 Editor

Can someone verify that  MIPI CSI cameras can be used with Thunder Boot and non Quick Snapshot mode?
Author: 799959745    Time: 5/5/2022 11:05
Fuzzy1 Posted at 5/1/2022 20:48
Some more info:

Please do not use gstreamer to take pictures.
Please use v4l2-ctl to take pictures.
Author: Fuzzy1    Time: 5/5/2022 15:09
799959745 Posted at 5/5/2022 11:05
Please do not use gstreamer to take pictures.
Please use v4l2-ctl to take pictures.

Yes, but in my previous post I use v4l2-ctl:
  1.     v4l2-ctl -d /dev/video4 --set-fmt-video=width=640,height=480,pixelformat=NV12
  2.     --stream-mmap=3 --stream-skip=3 --stream-to=/tmp/nv12.bin --stream-count=100 --s
  3.     tream-poll
  4.     select timeout
Copy the code

Author: 799959745    Time: 5/10/2022 09:37
Last edited by 799959745 In 5/10/2022 09:38 Editor
Fuzzy1 Posted at 5/5/2022 15:09
Yes, but in my previous post I use v4l2-ctl:

Maybe the configuration is wrong. It is not within our support.
Author: Fuzzy1    Time: 5/11/2022 03:16
799959745 Posted at 5/10/2022 09:37
Maybe the configuration is wrong. It is not within our support.

Ok, I understand.

But if I buy the 4M Camera Module OS04A10 then I can use it it with Thunder Boot and Quick Snapshot mode, right?
Author: 799959745    Time: 5/11/2022 09:20
Fuzzy1 Posted at 5/11/2022 03:16
Ok, I understand.

But if I buy the 4M Camera Module OS04A10 then I can use it it with Thunder B ...

A Thunder Boot is fine. But the Quick Snapshot mode has not been tried, it should also be possible.
Author: boarsystem    Time: 5/11/2022 18:27
799959745 Posted at 5/11/2022 09:20
A Thunder Boot is fine. But the Quick Snapshot mode has not been tried, it should also be possible ...

I have ordered the OS04A10 camera, so hopefully I can use it with Thunder Boot with or without Snapshot Mode.
Author: Fuzzy1    Time: 5/13/2022 05:13
Last edited by Fuzzy1 In 5/13/2022 08:27 Editor

Some more questions:


1.

I noticed that there are Quick Snapshot MCU firmware for imx415 (rv1126_riscv_imx415_3864_2192_SGBRG10_compact_v1.00.bin),  os04c10 (rv1126_riscv_os04c10_2688_1520_SBGGR10_compact_v1.10.bin) and sc210iot (rv1126_riscv_sc210iot_1920_1080_SBGGR10_compact_v1.00.bin) but not for OS04A10.

Is there Quick Snapshot firmware available for OS04A10?

2.

Has the document Rockchip_Developer_Guide_RV1126_RV1109_Battery_Product_EN.pdf been updated recently by Rockchip? Current version in the Firefly SDK is v1.6.1.

3.

Does additional information exist about how to program the MCU? The documentation mentions for example the directory bsp/rockchip/rv1126-riscv but I can't find a similar path in the Firefly SDK.

Author: 799959745    Time: 5/14/2022 16:39
Fuzzy1 Posted at 5/13/2022 05:13
Some more questions:

There is no firmware for taking quick pictures. But it supports quick photo shooting. You can try it out by following the documentation first.
Author: Fuzzy1    Time: 5/14/2022 19:53
I read the information provided in Rockchip_Developer_Guide_RV1126_RV1109_Battery_Product_EN.pdf, but there is really no information about how to write code for the MCU.

Is there additional documentation somewhere?

Is there source files for the MCU that can be studied and compiled?
Author: 799959745    Time: 5/17/2022 14:44
Fuzzy1 Posted at 5/14/2022 19:53
I read the information provided in Rockchip_Developer_Guide_RV1126_RV1109_Battery_Product_EN.pdf, bu ...

This part of rockchip has not released relevant information.
Author: Fuzzy1    Time: 8/12/2022 04:00
I found out that if I use I2C1 instead of I2C2 it works fine. So there's probably some kind of bug which prevents using I2C2 with Thunder Boot enabled.



Welcome Firefly Open Source Community (https://bbs.t-firefly.com/) Powered by Discuz! X3.1