===== What's here? =====
My setup to measure bandwidth, and make it available as PCP metric. So with this, in regular intervals for example "once per hour", bandwidth to various destinations can be measured. You can for example for your edge-networks see over time how much bandwidth you had available, and visualize with Grafana. The obviously comes at a cost: traffic. 

For bandwidth measurement, I fetch files with wget. I experimented with fetching 500mb files and 70mb files, the difference was small - fetching 70mb files is enough. You have some "rampup time" of tcp sessions, so the files need to have a certain size. Using iperf might also be possible.

Reminder: pmda-netcheck is great to measure latencies to your other IP devices.

===== Setup =====
As normal user:
<code>
# first dir contains my helper scripts,
# second dir the data handed over to openmetrics
mkdir -p /opt/chris/bin/ /opt/store/openmetrics/data/

crontab -e
# ass this line for hourly bandwidth test:
57 * * * * /opt/chris/bin/bandwidth2.sh >/opt/store/openmetrics/data/bandwidth2.txt

cat >/opt/chris/bin/bandwidth2.sh<<EOT
#!/usr/bin/bash

URLADDON1="images/pxeboot/initrd.img"
URLADDON2="images/install.img"

URLBASE="ftp://ftp.fau.de/fedora/linux/releases/33/Server/x86_64/os/"
echo -ne 'ftp.fau.de {filesize="75MB"} '
wget --report-speed=bits -O /dev/null "${URLBASE}${URLADDON1}" 2>&1 | grep saved | \
        sed -e 's,.*(,,' -e 's, .*,,'

URLBASE="http://ftp.riken.jp/Linux/fedora/releases/33/Server/x86_64/os/"
echo -ne 'ftp.riken.jp {filesize="75MB"} '
wget --report-speed=bits -O /dev/null "${URLBASE}${URLADDON1}" 2>&1 | grep saved | \
        sed -e 's,.*(,,' -e 's, .*,,'

URLBASE="http://dfw.mirror.rackspace.com/fedora/releases/33/Server/x86_64/os/"
echo -ne 'rackspace.com {filesize="75MB"} '
wget --report-speed=bits -O /dev/null "${URLBASE}${URLADDON1}" 2>&1 | grep saved | \
        sed -e 's,.*(,,' -e 's, .*,,'

EOT

chmod +x /opt/chris/bin/bandwidth2.sh

# now execute the file once for testing:
/opt/chris/bin/bandwidth2.sh
# should present something like this, depending on your connectivity.
# The last field is the bandwidth:
chris@fluxcoil:~$ /opt/chris/bin/bandwidth2.sh
ftp.fau.de {filesize="75MB"} 912
ftp.riken.jp {filesize="75MB"} 69.9
rackspace.com {filesize="75MB"} 7.90
chris@fluxcoil:~$ 
</code>

The next part is to be done as root:
<code>
# install PCP and pmda-openmetrics if not already done,
# for example on Fedora:
dnf -y install pcp-zeroconf pcp-pmda-openmetrics
cd /var/lib/pcp/pmdas/openmetrics/
./Install

# Let let's create the openmetrics hook to get the data
# from /opt/store/openmetrics/data/bandwidth2.txt available
# as metric
echo 'file:///opt/store/openmetrics/data/bandwidth2.txt' >\
  /var/lib/pcp/pmdas/openmetrics/config.d/bandwidth2.url

# Now verify the metric appeared:
chris@fluxcoil:~$ pminfo openmetrics.bandwidth2
openmetrics.bandwidth2.rackspace.com
openmetrics.bandwidth2.ftp.riken.jp
openmetrics.bandwidth2.ftp.fau.de
chris@fluxcoil:~$
chris@fluxcoil:/opt/web/blog-hugo$ pmrep openmetrics.bandwidth2
  o.b.r.com  o.b.f.r.jp  o.b.f.f.de
  0 filesiz  0 filesize  0 filesize
      9.210      53.400     912.000
      9.210      53.400     912.000
^C

# So now the data is available live as metric.
# Let's ensure it gets archived with pmlogger:
vi /var/lib/pcp/config/pmlogger/config.default
# add this part:
~~~
# DO NOT UPDATE THE FILE ABOVE THIS LINE
# Otherwise any changes may be lost the next time pmlogconf is
# used on this file.
#
# It is safe to make additions from here on ...
#
log advisory on every 1 hour {
        openmetrics.bandwidth2
}
~~~

systemctl restart pmlogger

# now verify the new metric get archived:
less  /var/log/pcp/pmlogger/fluxcoil/pmlogger.log
~~~
Group [18 metrics] {
        openmetrics.bandwidth2.rackspace.com
        openmetrics.bandwidth2.ftp.riken.jp
        openmetrics.bandwidth2.ftp.fau.de
        openmetrics.smtpconnects.incoming
[..]
} logged every 3600 sec: 536 bytes or 0.01 Mbytes/day
~~~
</code>