User Tools

Site Tools


hardwarerelated:camlink_hdmi_converter

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
hardwarerelated:camlink_hdmi_converter [2020/06/15 01:33]
chris created
hardwarerelated:camlink_hdmi_converter [2020/07/10 01:18] (current)
chris [What's here?]
Line 1: Line 1:
 +===== What's here? =====
 +My attempts to grab HDMI with [[|an addon board for the Raspi]] worked, but did not give me audio. For this, I ordered this device here, usb id 534d:2109. Works nicely, low latency and also audio is grabbed. My preferred software: mplayer. Details below.
   * https://twitter.com/Ascii211/status/1268631069051453448   * https://twitter.com/Ascii211/status/1268631069051453448
-  * [[https://www.aliexpress.com/item/4000958505670.html|@aliexpress]] +  * The grabber at [[https://www.aliexpress.com/item/4000958505670.html|aliexpress]] 
-  * Should provide audio grabbing, also works nicely on Linux+  * Provides also audio grabbing, works nicely on Linux. 720p at 60Hz, 1080p at lower frequency 
 +  * potentially interesting 4k grabber: [[https://de.aliexpress.com/item/4000315619046.html|link1]] [[https://de.aliexpress.com/item/4000235646966.html|link2]] 
 +  * stereo sound needs a patch as of now: [[https://twitter.com/marcan42/status/1270350309039067140|link]] [[https://www.spinics.net/lists/alsa-devel/msg110886.html|alsa-devel]] 
 +  * USB3/4k/1080p@60Hz [[https://twitter.com/marcan42/status/1281266315831808001|alternative]] 
 +===== Configuration ===== 
 +<code> 
 +dnf -y install v4l-utils ffmpeg 
 + 
 +# video0 is my thinkpad internal camera 
 +[chris@電脳 ~]$ v4l2-ctl --list-devices 
 +UVC Camera (534d:2109): USB Vid (usb-0000:00:14.0-2): 
 +        /dev/video2 
 +        /dev/video3 
 + 
 +Integrated Camera: Integrated C (usb-0000:00:14.0-7): 
 +        /dev/video0 
 +        /dev/video1 
 + 
 +# The hdmi grabber provides video2: 
 +[chris@電脳 ~]$ lsusb 
 +[..] 
 +Bus 001 Device 007: ID 534d:2109 MACROSILICON  
 +[..] 
 +[chris@電脳 ~]$ v4l2-ctl --all -d /dev/video2 
 +Driver Info: 
 +        Driver name      : uvcvideo 
 +        Card type        : UVC Camera (534d:2109): USB Vid 
 +        Bus info         : usb-0000:00:14.0-2 
 +        Driver version   : 5.6.19 
 +        Capabilities     : 0x84a00001 
 +                Video Capture 
 +                Metadata Capture 
 +                Streaming 
 +                Extended Pix Format 
 +                Device Capabilities 
 +        Device Caps      : 0x04200001 
 +                Video Capture 
 +                Streaming 
 +                Extended Pix Format 
 +Priority: 2 
 +Video input : 0 (Camera 1: ok) 
 +Format Video Capture: 
 +        Width/Height      : 1280/720 
 +        Pixel Format      : 'YUYV' (YUYV 4:2:2) 
 +        Field             : None 
 +        Bytes per Line    : 2560 
 +        Size Image        : 1843200 
 +        Colorspace        : sRGB 
 +        Transfer Function : Default (maps to sRGB) 
 +        YCbCr/HSV Encoding: Default (maps to ITU-R 601) 
 +        Quantization      : Default (maps to Limited Range) 
 +        Flags             :  
 +[..] 
 + 
 +# Let's list the supported resolutions: 
 +[chris@電脳 ~]$ ffmpeg -f v4l2 -list_formats all -i /dev/video2 
 +ffmpeg version 4.2.3 Copyright (c) 2000-2020 the FFmpeg developers 
 +[..] 
 +[video4linux2,v4l2 @ 0x558496514600] Compressed:       mjpeg :          Motion-JPEG : 1920x1080 1600x1200 1360x768 1280x1024 1280x960 1280x720 1024x768 800x600 720x576 720x480 640x480 
 +[video4linux2,v4l2 @ 0x558496514600] Raw       :     yuyv422 :           YUYV 4:2:2 : 1920x1080 1600x1200 1360x768 1280x1024 1280x960 1280x720 1024x768 800x600 720x576 720x480 640x480 
 +/dev/video2: Immediate exit requested 
 +[chris@電脳 ~]$  
 +</code> 
 + 
 +===== First test ===== 
 +mplayer does not work for me, so lets use ffmpeg streaming 
 +<code> 
 +# Server: the process to be started on the Raspi 
 +#   exchange the ip with your clients IP, who should receive the stream 
 +ffmpeg -f v4l2 -input_format yuyv422 -s 1280x720 -r 60 -i /dev/video2 \ 
 +  -tune zerolatency -vcodec mpeg2video -f mpegts udp://192.168.0.2:4242 
 + 
 +# Note: also do not forget to deal with the firewall on your client, 
 +# it needs to allow incoming packets! 
 + 
 +# Client process, exchange with the IP of the system sending the stream: 
 +ffplay -an -sn -i -fflags nobuffer udp://192.168.0.3:4242?listen 
 +</code> 
 + 
 +===== Audio test ===== 
 +  * https://wiki.archlinux.org/index.php/Advanced_Linux_Sound_Architecture 
 +  * https://sysplay.in/blog/linux/2019/06/playing-with-alsa-loopback-devices/ 
 +<code> 
 +[chris@電脳 ~]$ arecord -l 
 +**** List of CAPTURE Hardware Devices **** 
 +card 0: PCH [HDA Intel PCH], device 0: ALC257 Analog [ALC257 Analog] 
 +  Subdevices: 0/1 
 +  Subdevice #0: subdevice #0 
 +card 1: U0x534d0x2109 [USB Device 0x534d:0x2109], device 0: USB Audio [USB Audio] 
 +  Subdevices: 0/1 
 +  Subdevice #0: subdevice #0 
 +[chris@電脳 ~]$  
 + 
 +==> The second device is from our HDMI grabber, good! 
 + 
 +[chris@電脳 ~]$ arecord -d 25 -f cd test1.wav 
 + 
 +==> This is recording audio from HDMI! 
 + 
 +### Setup pure audio streaming for a start, no video. 
 +# server: 
 +[chris@電脳 ~]$ ffmpeg -f alsa -ac 1 -ar 44100 \ 
 +  -i hw:1 -acodec mp3 -f mpegts udp://192.168.0.2:4242 
 +# client: 
 +[chris@電脳 ~]$ ffplay -acodec mp3 -i -fflags nobuffer udp://192.168.0.2:4242?listen 
 + 
 +==> This grabs audio from the HDMI stream, makes it audible. 
 +    I had to add my user chris to group "audio", to access 
 +    /dev/snd/* devices 
 +</code> 
 + 
 +===== Best low latency local streaming, including sound ===== 
 +This streams not over the network, just displays on the system where the usb adapter is plugged in. 
 +<code> 
 +### Where is our HDMI grabber soundcard?  Here card 1, device 0: 
 +$ arecord -l 
 +**** List of CAPTURE Hardware Devices **** 
 +card 0: PCH [HDA Intel PCH], device 0: ALC257 Analog [ALC257 Analog] 
 +  Subdevices: 0/1 
 +  Subdevice #0: subdevice #0 
 +card 1: U0x534d0x2109 [USB Device 0x534d:0x2109], device 0: USB Audio [USB Audio] 
 +  Subdevices: 0/1 
 +  Subdevice #0: subdevice #0 
 + 
 +# Quickest local streaming is for me with mplayer: 
 +# - here reading from /dev/video0, 720p 
 +# - alsa audio cart 1, device 0 
 +  
 +$ mplayer tv:// -vo gl_nosw -tv \ 
 +  driver=v4l2:device=/dev/video0:width=1280:height=720:fps=60:outfmt=mjpeg:
 +  alsa:adevice=hw.1,0:forceaudio:immediatemode=0 
 +</code> 
 + 
 +===== Latency testing, local ===== 
 +<code> 
 +### following gives me 220ms-350ms 
 +### Sends data uncompressed, 15MB/sec on usb 
 +$ ffplay /dev/video2 
 + 
 +### following gives me 100-120ms 
 +### Sends data mjpeg compressed, 4MB/sec go over usb 
 +$ xawtv /dev/video2 
 + 
 +### following gives me 124ms latency in average (95ms-164ms) 
 +### Sends data mjpeg compressed, 4MB/sec go over usb 
 +$ mplayer tv:// -vo gl_nosw -tv \ 
 +  driver=v4l2:device=/dev/video2:width=1280:height=720:fps=60:outfmt=mjpeg 
 +   
 +### following gives me 400ms latency in average (321ms-474ms) 
 +### Sends data uncompressed, 18.4MB/sec on usb -> maxed out 
 +$ mplayer tv:// -vo gl_nosw -tv \ 
 +  driver=v4l2:device=/dev/video2:width=1280:height=720:fps=60:outfmt=yuy2 
 +</code> 
 + 
 +===== Latency testing, network ===== 
 +Latency testing with this setup: USB-HDMI-grabber -> Raspi4 -> Gbit -> Fedora desktop. For testing, I send my desktop via HDMI, capture this with the USB-HDMI-grabber, display it eventually on the desktop and measure latency. 
 + 
 +^ latency [1]  ^ command on server                  ^ command on client ^ HDMI latency [2] ^ 
 +| 0.1ms (GBit) | ssh -X / mplayer / xforwarding [3] | -                   ~400ms         | 
 +| 0.1ms (GBit) | ffmpeg [4]                         | ffplay [5]        |   ~700ms         | 
 +| 0.1ms (GBit) | VLC: cvlc [6]                      | ffplay [7]        |   ~330ms         | 
 +| 0.1ms (GBit) | ustreamer [8]                      | firefox [9]         ~120ms         | 
 + 
 +<code> 
 +[1] round trip time over network, between server with usb-grabber connected, 
 +    and the client 
 +[2] Timespan between drawing something in HDMI input, vs. it shown on client 
 +[3] mplayer tv:// -tv driver=v4l2:device=/dev/video1:width=1280:height=720:fps=30:outfmt=yuy2 
 +[4] ffmpeg -f v4l2 -input_format yuyv422 -s 1280x720 -r 60 -i /dev/video2 \ 
 +      -tune zerolatency -vcodec mpeg2video -f mpegts udp://192.168.0.2:4242 
 +[5] ffplay -an -sn -i -fflags nobuffer udp://192.168.0.3:4242?listen 
 +[6] cvlc -vvv v4l2:// :v4l2-dev=/dev/video1 :v4l2-width=1280 
 +      :v4l2-height=720 :v4l2-fps 25 --sout '#standard{access=http,mux=raw,dst=:8081}' 
 +[7] ffplay -f rawvideo -pixel_format yuyv422 -video_size 1280x720 \ 
 +      -framerate 25 http://192.168.0.3:8081 
 +[8] ustreamer --device=/dev/video2 -r 1280x720 --format=jpeg \ 
 +      --workers=4 --persistent --drop-same-frames=30 --host=0.0.0.0 --port=8080 
 +[9] firefox http://<ip-of-raspi>:8080/stream 
 +</code> 
 + 
 +===== Hints ===== 
 +== Building ustreamer == 
 +<code> 
 +# on Fedora/x86_64 
 +sudo dnf -y install libevent-devel libbsd-devel 
 +git clone --depth=1 https://github.com/pikvm/ustreamer 
 +cd ustreamer/ 
 +make 
 +</code>
hardwarerelated/camlink_hdmi_converter.1592177598.txt · Last modified: 2020/06/15 01:33 by chris