User Tools

Site Tools


Sidebar

hardwarerelated:camlink_hdmi_converter

What's here?

My attempts to grab HDMI with an addon board for the Raspi worked, but did not give me audio. For this, I ordered this device here, usb id 534d:2109. Works nicely, low latency and also audio is grabbed. My preferred software: mplayer. Details below.

Configuration

dnf -y install v4l-utils ffmpeg

# video0 is my thinkpad internal camera
[chris@電脳 ~]$ v4l2-ctl --list-devices
UVC Camera (534d:2109): USB Vid (usb-0000:00:14.0-2):
        /dev/video2
        /dev/video3

Integrated Camera: Integrated C (usb-0000:00:14.0-7):
        /dev/video0
        /dev/video1

# The hdmi grabber provides video2:
[chris@電脳 ~]$ lsusb
[..]
Bus 001 Device 007: ID 534d:2109 MACROSILICON 
[..]
[chris@電脳 ~]$ v4l2-ctl --all -d /dev/video2
Driver Info:
        Driver name      : uvcvideo
        Card type        : UVC Camera (534d:2109): USB Vid
        Bus info         : usb-0000:00:14.0-2
        Driver version   : 5.6.19
        Capabilities     : 0x84a00001
                Video Capture
                Metadata Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
Priority: 2
Video input : 0 (Camera 1: ok)
Format Video Capture:
        Width/Height      : 1280/720
        Pixel Format      : 'YUYV' (YUYV 4:2:2)
        Field             : None
        Bytes per Line    : 2560
        Size Image        : 1843200
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Limited Range)
        Flags             : 
[..]

# Let's list the supported resolutions:
[chris@電脳 ~]$ ffmpeg -f v4l2 -list_formats all -i /dev/video2
ffmpeg version 4.2.3 Copyright (c) 2000-2020 the FFmpeg developers
[..]
[video4linux2,v4l2 @ 0x558496514600] Compressed:       mjpeg :          Motion-JPEG : 1920x1080 1600x1200 1360x768 1280x1024 1280x960 1280x720 1024x768 800x600 720x576 720x480 640x480
[video4linux2,v4l2 @ 0x558496514600] Raw       :     yuyv422 :           YUYV 4:2:2 : 1920x1080 1600x1200 1360x768 1280x1024 1280x960 1280x720 1024x768 800x600 720x576 720x480 640x480
/dev/video2: Immediate exit requested
[chris@電脳 ~]$ 

First test

mplayer does not work for me, so lets use ffmpeg streaming

# Server: the process to be started on the Raspi
#   exchange the ip with your clients IP, who should receive the stream
ffmpeg -f v4l2 -input_format yuyv422 -s 1280x720 -r 60 -i /dev/video2 \
  -tune zerolatency -vcodec mpeg2video -f mpegts udp://192.168.0.2:4242

# Note: also do not forget to deal with the firewall on your client,
# it needs to allow incoming packets!

# Client process, exchange with the IP of the system sending the stream:
ffplay -an -sn -i -fflags nobuffer udp://192.168.0.3:4242?listen

Audio test

[chris@電脳 ~]$ arecord -l
**** List of CAPTURE Hardware Devices ****
card 0: PCH [HDA Intel PCH], device 0: ALC257 Analog [ALC257 Analog]
  Subdevices: 0/1
  Subdevice #0: subdevice #0
card 1: U0x534d0x2109 [USB Device 0x534d:0x2109], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0
[chris@電脳 ~]$ 

==> The second device is from our HDMI grabber, good!

[chris@電脳 ~]$ arecord -d 25 -f cd test1.wav

==> This is recording audio from HDMI!

### Setup pure audio streaming for a start, no video.
# server:
[chris@電脳 ~]$ ffmpeg -f alsa -ac 1 -ar 44100 \
  -i hw:1 -acodec mp3 -f mpegts udp://192.168.0.2:4242
# client:
[chris@電脳 ~]$ ffplay -acodec mp3 -i -fflags nobuffer udp://192.168.0.2:4242?listen

==> This grabs audio from the HDMI stream, makes it audible.
    I had to add my user chris to group "audio", to access
    /dev/snd/* devices

Best low latency local streaming, including sound

This streams not over the network, just displays on the system where the usb adapter is plugged in.

### Where is our HDMI grabber soundcard?  Here card 1, device 0:
$ arecord -l
**** List of CAPTURE Hardware Devices ****
card 0: PCH [HDA Intel PCH], device 0: ALC257 Analog [ALC257 Analog]
  Subdevices: 0/1
  Subdevice #0: subdevice #0
card 1: U0x534d0x2109 [USB Device 0x534d:0x2109], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0

# Quickest local streaming is for me with mplayer:
# - here reading from /dev/video0, 720p
# - alsa audio cart 1, device 0
 
$ mplayer tv:// -vo gl_nosw -tv \
  driver=v4l2:device=/dev/video0:width=1280:height=720:fps=60:outfmt=mjpeg:\
  alsa:adevice=hw.1,0:forceaudio:immediatemode=0

Latency testing, local

### following gives me 220ms-350ms
### Sends data uncompressed, 15MB/sec on usb
$ ffplay /dev/video2

### following gives me 100-120ms
### Sends data mjpeg compressed, 4MB/sec go over usb
$ xawtv /dev/video2

### following gives me 124ms latency in average (95ms-164ms)
### Sends data mjpeg compressed, 4MB/sec go over usb
$ mplayer tv:// -vo gl_nosw -tv \
  driver=v4l2:device=/dev/video2:width=1280:height=720:fps=60:outfmt=mjpeg
  
### following gives me 400ms latency in average (321ms-474ms)
### Sends data uncompressed, 18.4MB/sec on usb -> maxed out
$ mplayer tv:// -vo gl_nosw -tv \
  driver=v4l2:device=/dev/video2:width=1280:height=720:fps=60:outfmt=yuy2

Latency testing, network

Latency testing with this setup: USB-HDMI-grabber → Raspi4 → Gbit → Fedora desktop. For testing, I send my desktop via HDMI, capture this with the USB-HDMI-grabber, display it eventually on the desktop and measure latency.

latency [1] command on server command on client HDMI latency [2]
0.1ms (GBit) ssh -X / mplayer / xforwarding [3] - ~400ms
0.1ms (GBit) ffmpeg [4] ffplay [5] ~700ms
0.1ms (GBit) VLC: cvlc [6] ffplay [7] ~330ms
0.1ms (GBit) ustreamer [8] firefox [9] ~120ms
[1] round trip time over network, between server with usb-grabber connected,
    and the client
[2] Timespan between drawing something in HDMI input, vs. it shown on client
[3] mplayer tv:// -tv driver=v4l2:device=/dev/video1:width=1280:height=720:fps=30:outfmt=yuy2
[4] ffmpeg -f v4l2 -input_format yuyv422 -s 1280x720 -r 60 -i /dev/video2 \
      -tune zerolatency -vcodec mpeg2video -f mpegts udp://192.168.0.2:4242
[5] ffplay -an -sn -i -fflags nobuffer udp://192.168.0.3:4242?listen
[6] cvlc -vvv v4l2:// :v4l2-dev=/dev/video1 :v4l2-width=1280  \
      :v4l2-height=720 :v4l2-fps 25 --sout '#standard{access=http,mux=raw,dst=:8081}'
[7] ffplay -f rawvideo -pixel_format yuyv422 -video_size 1280x720 \
      -framerate 25 http://192.168.0.3:8081
[8] ustreamer --device=/dev/video2 -r 1280x720 --format=jpeg \
      --workers=4 --persistent --drop-same-frames=30 --host=0.0.0.0 --port=8080
[9] firefox http://<ip-of-raspi>:8080/stream

Hints

Building ustreamer
# on Fedora/x86_64
sudo dnf -y install libevent-devel libbsd-devel
git clone --depth=1 https://github.com/pikvm/ustreamer
cd ustreamer/
make
hardwarerelated/camlink_hdmi_converter.txt · Last modified: 2020/07/25 01:28 by chris