Streaming with Trunk Recorder and Liquidsoap
From The RadioReference Wiki
Contents
- 1 Introduction
- 2 How Trunk Recorder Works
- 3 How do we stream it?
- 4 Basic LiquidSoap Configuration
- 5 Streamthis Script
- 6 Modifying encode-upload.sh
- 7 Multiple Streams from one System
- 8 Improved Multiple Stream Support
- 9 Code to Drop if Queue is Full
- 10 Code to work with newer Python3 and liquidsoap 2.x
Introduction
Trunk Recorder is a free Linux based application written by Luke Berndt that can decode the following:
- Trunked P25 & SmartNet Systems
- Conventional P25 & analog systems, where each group has a dedicated RF channel
- P25 Phase 1, P25 Phase 2 & Analog voice channels
Using a number of supported SDR hardware dongles Trunk Recorder can take entire P25 systems and simultaneously record every single transmission occurring at any moment in time. This is accomplished by using enough SDR dongles to cover the entire spectrum of voice and control channels in a system.
From a stream provider perspective this gives us the ability to put all transmissions into a stream without losing those that occurred while a physical scanner would be "stuck" on another channel. The cost is a slight delay caused by queuing the transmissions, but the severity of that queue depth is dependent upon how many talk-groups you stream and how chatty they are.
Now that Trunk Recorder supports analog its possible to combine analog and digital transmissions into the stream.
The "streamthis" python script below was written to bridge the gap between the .wav files Trunk Recorder creates, and an idle liquidsoap client already connected to Broadcastify. It made it possible to build the Twinsburg, Hudson and Reminderville Police, Fire / EMS stream and has worked quite well. That particular stream is sourced from two separate computers, one of which is dedicated to conventional analog receive. As those files are generated they are copied to the primary computer via ssh, and the streamthis script is remotely executed to insert the newly delivered analog .mp3 file to the stream queue.
How Trunk Recorder Works
Under the hood Trunk Recorder relies upon GNURadio and OP25. When you install Trunk Recorder you tell it where your system's control channels are, and you define the "center frequency" for your SDR dongles to cover the entire range of a given system. For example, using the Summit County Ohio Regional 800 System, with 20 some voice channels and four possible control channels the configuration looks something like this using four $30 SDR dongles to cover the entire RF range:
Each dongle has a "center" frequency and can listen to approximately 1mhz above and below that center. For brevity, one dongle looks like the section below. This system supports both digital and analog trunked talk-groups, hence the two types of "recorders" which are processes dedicated to listening to a voice channel, decoding the digital data into analog if necessary, and outputting it into a .wav file:
{
"center": 851418750.0,
"rate": 2148000.0,
"squelch": -60,
"error": 0,
"ppm": 0,
"gain": 350,
"antenna": "RX",
"modulation": "fsk4",
"digitalRecorders": 5,
"analogRecorders": 2,
"driver": "osmosdr",
"device": "rtl=1"
},
This block defines the possible control channels, defines it as a SmartNet system, instructs Trunk Recorder that the system is rebanded, and defines a script to run after each transmission is recorded.
"systems": [
{
"control_channels": [858712500, 855262500, 857962500, 858512500],
"type": "smartnet",
"bandplan": "800_reband",
"uploadScript": "encode-upload.sh",
"talkgroupsFile": "ChannelList.csv"
}]
Just like a a typical consumer grade scanner Trunk Recorder listens to the control channel, but since it has a receiver dedicated to that frequency it is always listening and never stops listening. This keeps it aware of every possible transmission on any of the voice channels, where as a normal scanner would leave the control channel to decode a transmission and potentially miss other simultaneous traffic.
Watching it's output we can see it start a recorder when TG 2384 and TG 33712 started talking:
(info) [sys_1] TG: 33712 Freq: 8.51562e+08 Call not found for Update Message, Starting one... (info) [sys_1] TG: 33712 Freq: 8.51562e+08 Starting Recorder on Src: rtl=1 (info) - Starting P25 Recorder Num [0] TG: 33712 Freq: 8.51562e+08 TDMA: false Slot: 0 (info) [sys_1] TG: 2384 Freq: 8.55088e+08 Starting Recorder on Src: rtl=0 (info) - Starting P25 Recorder Num [9] TG: 2384 Freq: 8.55088e+08 TDMA: false Slot: 0 (info) [sys_1] TG: 33712 Freq: 8.51562e+08 Assign Retuning - New Freq: 8.5125e+08 Elapsed: 6s Since update: 2s (info) [sys_1] TG: 2352 Freq: 8.55512e+08 Ending Recorded Call - Last Update: 4s Call Elapsed: 25 (info) - Stopping P25 Recorder Num [10] TG: 2352 Freq: 8.55512e+08 TDMA: false Slot: 0 (info) Running upload script: ./encode-upload.sh /home/scanner/jradio/trunk-recorder/sys_1/2017/7/9/2352-1499630871_8.55512e+08.wav
As you can see from the text above the output is a wav file that may include a single transmission or, should there be very quick back-to-back transmissions that occur on the same voice channel, a string of them in a single .wav file. Each transmission also has an identically named .json file with metadata including the radio IDs and duration of transmission in the file:
{
"freq": 8.55512e+08,
"start_time": 1499630871,
"stop_time": 1499630896,
"emergency": 0,
"talkgroup": 2352,
"srcList": [ {"src": 7609, "time": 1499630871, "pos": 0.000000}, {"src": 24884, "time": 1499630873, "pos": 1.476000},
{"src": 4616, "time": 1499630879, "pos"
: 6.300000}, {"src": 7609, "time": 1499630888, "pos": 13.140000}, {"src": 2360, "time": 1499630889, "pos": 13.140000},
{"src": 7609, "time": 1499630889, "pos
": 13.140000}, {"src": 2360, "time": 1499630890, "pos": 14.400000} ],
"play_length": 15.300000,
"system": 2,
"freqList": [ { "freq": 855512500.000000, "time": 1499630871, "pos": 0.000000, "len": 123664.000000,
"error_count": 677.000000, "spike_count": 0.000000} ]
}
Trunk Recorder is similar to using DSD+ on Windows, but it is superior in some respects because it is a single application. There is no need to run multiple instances of the application per VCO or "recorder". There is no need for virtual audio cables. Everything Trunk Recorder does is self-contained within a single application that produces .wav files and then initiates an exteral script so you can do something automatically with those files.
Installing Trunk Recorder
It should be stated up front that what Trunk Recorder is doing is very CPU intensive. You're not going to have success running 3-4 dongles with 3-5 digital recorders on each dongle trying to decode a number of simultaneous convorsations without a reasonable amount of CPU. You'll know your issue is CPU if after running the recorder process you see a bunch of 0's or O's showing.
As an alternative to installing and building from source which can take quite some time on low powered devices such as respberrypi, you can download and run premade docker images that container trunk recorder and run virtualised under the docker program. Instructions here: https://github.com/robotastic/trunk-recorder/wiki/
Installing Trunk Recorder is relatively simple even for a novice Linux user. The official directions can be found on the project's GitHub page, but here is a the quick approach in Ubuntu:
In Ubuntu you would start by installing some pre-requisites:
sudo apt-get update sudo apt-get upgrade sudo apt-get install gnuradio-dev gr-osmosdr libhackrf-dev libuhd-dev sudo apt-get install git cmake build-essential libboost-all-dev libusb-1.0-0.dev libssl-dev
Then, make a directory to install Trunk Recorder into, download it with git and "compile" the application:
mkdir trunk-build git clone https://github.com/robotastic/trunk-recorder.git cd trunk-build cmake ../trunk-recorder make
If this works you'll see:
[100%] Built target recorder
Configuring Trunk Recorder
Again, the instructions on how to configure Trunk Recorder are found here on the project's GitHub page. You should especially read the lower portion of that page which explains all of the fields of the config.json file, the format of the ChannelsList.csv file.
The more challenging part of building your config.json is determining how to layout each of your SDR dongles by picking a center frequency that allows you to use as few dongles as possible.
Clients for Trunk Recorder
There are multiple web front ends designed to take the output of Trunk Player and present it in a visually pleasing interface. These don't require streaming to Broadcastify. You can use any combination of these (along with Broadcastify), but it requires manually tweaking the upload script.
OpenMHz
The author of Trunk Recorder has his own OpenMHZ hosting platform that supports .m4a audio files. More info on that can be found here: https://github.com/robotastic/trunk-recorder/wiki/Uploading-to-OpenMHz
The Trunk Player Web GUI
Another option is Trunk Player, written by Dylan Reinhold, which you can host on your own either entirely internal to your home network, or by utilizing Amazon AWS hosting to push the audio files to the internet (so you can play them from any internet connected device). The front end looks like this:
The great thing about Trunk Player is it's scanlist concept. If you have 2-3 systems built either on your own or by working with friends, all pushing audio files from different digital systems, you can group talk groups into logical scanlists instead of system based scanlists.
For example, in the Northeast Ohio's Cuyahoga County there are two digital systems: The State of Ohio MARCS P25 and Cleveland, Ohio's regional "county wide" P25 system it lets suburbs use. The numerous communities that make up Cuyahoga County have either gone with MARCS or Cleveland's system. Everything in Summit County (to the south) is slowly moving from a SmartNet system to a regional P25 of it's own. If you just like to hear about any fire scene regardless of what system it came from then in Trunk Player you just define a "Scan List" with all of the Fire/EMS talk-groups you care about. Then you hear every single transmission (even if some of them occurred at the same time) regardless of which digital system they were derived from. You might hear something 30 or so seconds after it occurred, but you hear the entire transmission as opposed to missing things.
Note: Trunk Player is not necessarily the easiest thing to install, and it will require you to understand how to manipulate the encode_upload.sh scripts that come with Trunk Player to move your .wav files somewhere that they can be served up. Either by utilizing the Amazon AWS S3 concept, or by building your own in-house web server and making those .wav (or converted to .mp3) files available. The front end of Trunk Player simply presents a link to the audio file and the device visiting that webpage retrieves it and plays it.
RDIO Scanner
RDIO Scanner is an option that tries to emulate a traditional scanner experience. The audio files are converted to m4a and stored in SQLite. The app is very easy to use, and lets you both listen to live streams and to archives.
How do we stream it?
Trunk Recorder will basically generate a folder full of .wav files. We need an application that will connect to Broadcastify and keep the stream online even when nothing is occuring. liquidsoap is the client used to create a stream connection to the broadcastify server's mount point. When there is nothing in it's queue liquidsoap will send silence towards the stream based on the script below.
Every time Trunk Recorder writes a .wav file it will run a script called encode-upload.sh. (The script is for pushing the files to an external location for the web gui above, but you can add commands to it)
For the purposes of streaming we add a call to the encode-upload.sh script to run a separate script called "streamthis" which is provided below.
Streamthis is a python script that compares the talk-group of a wav file (by looking at the prefix of the file name) to a list of defined interesting talk-groups that make up your Broadcastify stream content. If the file just created by Trunk Recorder is one we want to inject into our stream the python script will connect to an already running instance of liquidsoap and add the new .wav file to the queue. If something is in the queue everything will be played in a FIFO fashion.
If the stream is currently idle, the .wav file is immediately played into the stream. However, since a transmission may be as long as a few seconds to upwards of 90 seconds before it is even written into a .wav file and made available to us a slight delay is introduced. It is imperative that the quantity of talk-groups added to a particular stream not be excessive. Due to the way this solution will queue up transmissions and play every single one of them into the stream a very busy talk group may need to be the only one streamed as the potential to end up 5-10 minutes behind could happen. This is a design obligation on the stream owner to avoid over building.
When the queue is empty, the stream sends silence towards Broadcastify.
Basic LiquidSoap Configuration
This liquidsoap script presumes you've already installed liquidsoap on the system that you're going to use to run Trunk Recorder. On Ubuntu you can install liquidsoap easily:
apt-get install liquidsoap
Once installed, this script can be made executable and run on Linux to initiate your stream connection to Broadcastify. NOTE: This stream will connect and appear "Online" indefinately with zero audio until you build the logic to queue .wav files. The configuration below tells liquidsoap to listen on a socket for connections from the streamthis script:
#!/usr/bin/liquidsoap
# ^-- use -v for verbose otherwise tail the log
set("tag.encodings",["UTF-8","ISO-8859-1"])
# Log dir
set("log.file.path","./basic-radio.log")
# create a socket to send commands to this instance of liquidsoap
set("server.socket",true)
set("server.socket.path","./socket")
# This creates a 1 second silence period generated programmatically (no disk reads)
silence = blank(duration=1.)
# This pulls the alpha tag out of the wav file
def append_title(m) =
# Grab the current title
title = m["title"]
if title != "" then
[("title",title)]
else
[("title","..> Scanning <..")]
end
end
silence = map_metadata(append_title, silence)
#
myqueue = request.queue()
myqueue = server.insert_metadata(id="S4",myqueue)
# If there is anything in the queue, play it. If not, play the silence defined above repeatedly:
stream = fallback(track_sensitive=false, [myqueue, silence])
stream = map_metadata(append_title, stream)
output.icecast(%mp3(stereo=false, bitrate=16, samplerate=22050), host="audio1.broadcastify.com",
port = 80, password = "YOURPASSWORD", genre="Scanner", description="Scanner audio", mount="/YOURMOUNTPOINT",
name="YOUR STREAM NAME HERE", stream)
Streamthis Script
This is the streamthis python script that decides if a given .wav file just generated by Trunk Recorder should be injected into the liquidsoap queue. Think of it as the glue between Trunk Recorder's output and liquidsoap's queue mechanism.
The "touch" commands in this script are used by a watchdog process to see if the stream has died and are optional. These were built because my analog system would occasionally die, and sometimes liquidsoap just goes to lunch. The watchdog script pays attention to how often "things" are happening, and takes automatic action to restart processes if it thinks the stream has gone to lunch. Once I clean that logic up I'll upload the watchdog script as well.
#!/usr/bin/python
import socket
import sys
import os
filepath = sys.argv[1]
filename = os.path.basename(filepath)
a = filename.split('-')
tg,junk = a[0],a[1]
def touch(fname, times=None):
with open(fname, 'a'):
os.utime(fname, times)
# for debug purposes
#print "tg is %s" % tg
# This array defines all of the decimal trunk-groups you want to inject into the stream queue:
streamthese = ["33328", "41168", "41200", "42576","42608","33936","41264","33904","33872","6","7","8"]
if tg in streamthese:
print ">>>>> STREAMTHIS: tg is in list"
match = 1
touch('/tmp/streamthis-lastrun')
else:
print ">>>>> STREAMTHIS: tg is not in the list"
match = 0
# silently exit
exit()
# Create a UDS socket
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
# Connect the socket to the port where the server is listening
server_address = '/PATH/TO/WHERE/LIQUIDSOAP/socket'
#print >>sys.stderr, 'connecting to %s' % server_address
try:
sock.connect(server_address)
except socket.error, msg:
print >>sys.stderr, msg
sys.exit(1)
try:
# Send data
message = 'queue.push {0}\n\r'.format(filepath)
print "message: {0}".format(message)
#print >>sys.stderr, 'sending "%s' % message
sock.sendall(message)
amount_received = 0
amount_expected = len(message)
data = sock.recv(16)
amount_received += len(data)
#print >>sys.stderr, 'received "%s' % data
finally:
#print >>sys.stderr, 'closing socket'
match = 0
sock.close()
Modifying encode-upload.sh
Note: Liquid Audio does not necessarily need to be provided mp3 files. It will happily stream wav files. (user: Hhca)
As I wrote this wiki page I realized the current version of encode_upload.sh is dramatically different than what I've been using. Therefor, I'll show you the modifications I make that you can add to whatever the current version of encode_upload.sh happens to become:
The encode_upload.sh script is passed a path to the .wav file by Trunk Recorder. That value is stored in the variable $1 and used by the shell script. We need to tear that path down and define some variables, so at the top of your encode_upload.sh script add the following if parts of this do not exist already.
CSV_FILE="/home/scanner/radio/dev/trunk-recorder/ChannelList.csv" # Path to YOUR Trunk Recorder CSV file
filename="$1" # This should already exist
basename="${filename%.*}" # This should already exist
filename_only=$(basename $basename)
dir="$(dirname $filename)"
mp3=${dir}/${filename_only}.mp3
I've started converting my .wav file to mp3 prior to sending it to liquidsoap. I believe this takes the burden of transcoding the audio away from liquidsoap, while also giving me an opportunity to inject the Alpha Tag into the .mp3 file. The Alpha tag is derived from the CSV file defined above, which is required to start Trunk Recorder in the first place. Unfortunately its not currently available in the .json.
I place this into the encode_upload.sh file somewhere near the bottom (after it has done any file transfer/uploads):
# this is for streamthis - lets convert to mp3 here, then delete the .wav on exit
# Start by finding the alpha tag for the talk-group:
tg="$( cut -d '-' -f 1 <<< "$filename_only" )"
ALPHA=`grep "^${tg}," ${CSV_FILE} | cut -d , -f 4`
if [ -z "$ALPHA" ]
then
ALPHA="TG: ${tg}"
fi
The following command uses lame to both add the alpha tag and run a highpass filter. This removes any noise such as PL hum if you're injecting any analog audio:
# Insert the alpha tag into a mp3 with highpass:
lame --preset voice --highpass 150 --tt "${ALPHA}" $filename $mp3
Finally, we tell the streamthis script the fully qualified path to the MP3 file we just added an alpha tag to:
/home/scanner/liquid/streamthis $mp3
...if the talk-group in that MP3 matches those you defined above in the "streamthese" array, then the streamthis script will inject the path to the MP3 into the queue and the transmission will get played into your stream
From the logfile of liquidsoap we can see a new transmission arriving in the queue and the transition from silence, to queue, and back to silence. The liquidsoap script provided above will extract the Alpha Tag of a given transmissions out of the .mp3 file as it is added to the queue. When the queue is not being played the while() loop changes the ALpha Tag to ">>> Scanning <<<" or whatever text you choose to use. (Alpha tag insertion takes place back in encode_upload.sh)
2017/07/10 01:24:27 [map_metadata_4977:3] Inserting missing metadata. 2017/07/10 01:24:28 [map_metadata_4977:3] Inserting missing metadata. 2017/07/10 01:24:28 [server:3] New client unix socket "\232\177\143\127". 2017/07/10 01:24:28 [decoder:3] Method "MAD" accepted "/PATH/TO/sys_1/2017/7/10/42576-1499664243_8.51562e+08.mp3". 2017/07/10 01:24:28 [server:3] Client unix socket "\232\177\143\127" disconnected without saying goodbye..! 2017/07/10 01:24:28 [queue:3] Prepared "/PATH/TO/sys_1/2017/7/10/42576-1499664243_8.51562e+08.mp3" (RID 5). 2017/07/10 01:24:28 [fallback_4983:3] Switch to S4 with transition. 2017/07/10 01:24:38 [queue:3] Finished with "/PATH/TO/sys_1/2017/7/10/42576-1499664243_8.51562e+08.mp3". 2017/07/10 01:24:38 [fallback_4983:3] Switch to map_metadata_4977 with forgetful transition. 2017/07/10 01:24:38 [map_metadata_4977:3] Inserting missing metadata. 2017/07/10 01:24:39 [map_metadata_4977:3] Inserting missing metadata.
Note: If anyone familiar with liquidsoap can find a way to only insert the the "Scanning" metadata once instead of every single second while still being able to immediately interrupt with a queue MP3 I am open to suggestions. This constant inserting seems inefficient and unnecessary.
Multiple Streams from one System
Using the original contents of this article, I have modified it a little to fit my needs. I broadcast about 8 streams from the Starcom system in St. Clair County, IL.
Notes:
I placed the recordings in /tmp/capture because the files will be cleaned up daily by systemd. This prevents them from running the disk space out.
The system whee this is running is an Intel(R) Pentium(R) CPU G3258 @ 3.20GHz, with 8GB of ram, and a 60GB disk.
With this configuration, it can do about 8 streams and about 8 digital recorders which is enough for the system in this county.
top - 16:22:17 up 3 days, 1:37, 1 user, load average: 3.27, 3.13, 3.15 Tasks: 154 total, 1 running, 153 sleeping, 0 stopped, 0 zombie %Cpu(s): 63.5 us, 6.8 sy, 0.0 ni, 29.4 id, 0.0 wa, 0.0 hi, 0.3 si, 0.0 st KiB Mem : 8067164 total, 125436 free, 915916 used, 7025812 buff/cache KiB Swap: 2097148 total, 2097148 free, 0 used. 6716772 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1166 medrc 20 0 2643756 150776 60644 S 137.3 1.9 5354:27 recorder 1127 liquids+ 20 0 624700 73120 12056 S 1.0 0.9 38:44.14 liquidsoap 1038 liquids+ 20 0 631464 22072 7372 S 0.7 0.3 23:06.58 liquidsoap 1095 liquids+ 20 0 624700 52236 12044 S 0.7 0.6 23:35.93 liquidsoap 1108 liquids+ 20 0 624700 72972 12024 S 0.7 0.9 18:06.16 liquidsoap 1117 liquids+ 20 0 624700 62960 12196 S 0.7 0.8 27:22.28 liquidsoap 251 root 20 0 441356 258004 257172 S 0.3 3.2 2:28.86 systemd-journal 799 systemd+ 20 0 66288 6728 5384 S 0.3 0.1 10:54.13 systemd-resolve 1026 liquids+ 20 0 631464 22244 7228 S 0.3 0.3 22:52.92 liquidsoap 1078 liquids+ 20 0 624700 61992 12188 S 0.3 0.8 24:02.62 liquidsoap 1138 liquids+ 20 0 624700 60952 12296 S 0.3 0.8 23:42.38 liquidsoap 1149 liquids+ 20 0 624700 57928 11896 S 0.3 0.7 21:48.18 liquidsoap 1159 liquids+ 20 0 631464 22244 7592 S 0.3 0.3 22:36.31 liquidsoap
To make it all start on reboot:
sudo systemctl daemon-reload sudo systemctl enable recorder sudo systemctl enable liquidsoap
/opt/recorder/config.json
{
"sources": [{
"center": 854406250.0,
"rate": 10000000,
"error": 0,
"squelch": -90,
"gain": 54.0,
"antenna": "RX2",
"digitalLevels": 3,
"digitalRecorders": 8,
"analogRecorders": 2,
"driver": "osmosdr",
"device": "airspy",
"modulation": "QPSK"
}],
"systems": [{
"control_channels": [851225000,851562500,852162500],
"type": "p25",
"recordUnknown": true,
"shortName": "stclairco",
"apiKey": "",
"uploadServer": "https://api.openmhz.com",
"talkgroupsFile": "/opt/recorder/ChannelList.csv",
"uploadScript": "encode_upload.sh"
}],
"defaultMode": "analog",
"callTimeout": 5,
"logFile": false,
"uploadServer": "https://api.openmhz.com",
"captureDir": "/tmp/capture"
}
/etc/systemd/system/recorder.service
[Unit] Description = Starting Trunk Recorder Service After = network.target liquidsoap.service [Service] Type=simple User=medrc StandardOutput=syslog StandardError=syslog SyslogIdentifier=recorder WorkingDirectory=/opt/recorder ExecStart=/opt/recorder/recorder --config=/opt/recorder/config.json Restart=on-abort [Install] WantedBy = multi-user.target
/opt/recorder/stream.liq
This is the base stream file used by each stream specific file.
set("tag.encodings",["UTF-8","ISO-8859-1"])
# Configure Logging
set("log.file",false)
set("log.level",1)
set("log.stdout",false)
set("log.syslog",true)
set("log.syslog.facility","DAEMON")
set("log.syslog.program","liquidsoap-#{STREAMID}")
# create a socket to send commands to this instance of liquidsoap
set("server.socket",true)
set("server.socket.path","<sysrundir>/#{STREAMID}.sock")
set("server.socket.permissions",511)
# This creates a 1 second silence period generated programmatically (no disk reads)
silence = blank(duration=1.)
# This pulls the alpha tag out of the wav file
def append_title(m) =
[("title",">> Scanning <<")]
end
silence = map_metadata(append_title, silence)
recorder_queue = request.queue()
recorder_queue = server.insert_metadata(id="S4",recorder_queue)
# If there is anything in the queue, play it. If not, play the silence defined above repeatedly:
stream = fallback(track_sensitive=false, [recorder_queue, silence])
title = '$(if $(title),"$(title)","...Scanning...")'
stream = rewrite_metadata([("title", title)], stream)
output.icecast( %mp3(stereo=false, bitrate=16, samplerate=22050),
host=HOST, port=80, password=PASSWORD, genre="Scanner",
description="Scanner audio", mount=MOUNT, name=NAME, user="source", stream)
/opt/recorder/liquidsoap/<streamid>.liq
Create a symlink to this file at /etc/liquidsoap/<streamid>.liq
sudo ln -s /opt/recorder/liquidsoap/<streamid>.liq /etc/liquidsoap/<streamid>.liq
This will cause liquidsoap to start each of these streams on startup.
#!/usr/bin/liquidsoap HOST="audioX.broadcastify.com" MOUNT="/mount" PASSWORD="password" NAME="name of the stream" STREAMID="<streamid>" %include "/opt/recorder/stream.liq"
/opt/recorder/encode_upload.sh
#! /bin/bash
CSV_FILE="/opt/recorder/ChannelList.csv" # Path to YOUR Trunk Recorder CSV file
#echo "Encoding: $1"
filename="$1"
basename="${filename%.*}"
filename_only=$(basename $basename)
dir="$(dirname $filename)"
mp3=${dir}/${filename_only}.mp3
#mp3encoded="$basename.mp3"
#mp4encoded="$basename.m4a"
json="$basename.json"
# this is for streamthis - lets convert to mp3 here, then delete the .wav on exit
# Start by finding the alpha tag for the talk-group:
tg="$( cut -d '-' -f 1 <<< "$filename_only" )"
ALPHA=`grep "^${tg}," ${CSV_FILE} | cut -d , -f 4`
GROUP=`grep "^${tg}," ${CSV_FILE} | cut -d , -f 5`
if [ -z "$ALPHA" ]
then
ALPHA="TG: ${tg}"
fi
# Insert the alpha tag into a mp3 with highpass:
#lame --quiet --preset voice --highpass 150 --tt "${ALPHA}" $filename $mp3
lame --quiet --preset voice --tt "${ALPHA}" $filename $mp3
#echo "Resulting Alpha is ${ALPHA}"
#echo "Resulting Group is ${GROUP}"
echo "Submitting [${tg}] ${ALPHA}" | logger -p info -t encode_upload
/opt/recorder/streamthis $mp3
/opt/recorder/streamthis
#!/usr/bin/python
import socket
import sys
import os
import logging
import logging.handlers
logger = logging.getLogger('streamthis')
logger.setLevel(logging.INFO)
handler = logging.handlers.SysLogHandler(address = '/dev/log')
formatter = logging.Formatter('streamthis: %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)
filepath = sys.argv[1]
filename = os.path.basename(filepath)
a = filename.split('-')
tg,junk = a[0],a[1]
def touch(fname, times=None):
with open(fname, 'a'):
os.utime(fname, times)
# This array defines all of the decimal trunk-groups you want to inject into the stream queue:
ofallon = ["7199","7200","7201","7202","7203","7205","7206","7207","7208","7209","7210","7211","7212","7214"]
fairview = ["7219","7220","7221","7222","7223","7224","7225","7226","7227","7228","7229","7230","7231"]
#ofallon = ["7199","7200","7201","7202","7203","7205","7206","7207","7208","7209","7210","7211","7212","7214", "7215", "7216", "7204", "7213", "7217", "7218", "7219","7220","7221","7222","7223","7224","7225","7226","7227","7228","7229","7230","7231"]
ares = ["1", "2"]
# idot
#ares = ["7776", "7777", "7778", "7781", "7782", "7788", "7789", "8612", "8613", "8614", "8615", "8616", "8617", "8618", "8619", "8620", "8621", "8622"]
#isp district 11 small
#ares = ["17000", "17001", "17003", "17017", "17018", "17031", "17037", "17043", "17049", "17055", "17078"]
#isp district 11 large
isp = ["17000", "17001", "17003", "17017", "17018", "17031", "17037", "17043", "17049", "17055", "17078", "17065", "17066", "17067", "17068", "17069", "17070", "17071", "17075", "17084", "17085", "17086", "17096", "17097", "17098", "17102", "17103"]
troy = ["7961", "8201"]
fddisp = ["7019", "7023", "7028", "7189", "7079", "7271", "7290", "7221", "7201", "7097"]
belleville = ["7236", "7237", "7252"]
swansea = ["7245", "7243", "7252"]
estl = ["7162", "7188", "7187", "7190", "7252"]
#ofallon = ["7199", "7200", "7201", "7202", "7209", "7211"]
#fairview = ["7219", "7229", "7220", "7221"]
scclaw = ["7017", "7056", "7021", "7018", "7027", "7022", "7188", "7250", "7024", "7236", "7220", "7200", "7211"]
match = 0
servers = []
if tg in ofallon:
logger.info(">>>>> OFALLON STREAM: tg is in list")
match = 1
touch('/tmp/streamthis-ofallon-lastrun')
# Connect the socket to the port where the server is listening
server_address = '/var/run/liquidsoap/ofallon.sock'
servers.append(server_address)
if tg in fairview:
logger.info(">>>>> FAIRVIEW STREAM: tg is in list")
match = 1
touch('/tmp/streamthis-fairview-lastrun')
# Connect the socket to the port where the server is listening
server_address = '/var/run/liquidsoap/fairview.sock'
servers.append(server_address)
if tg in isp:
logger.info(">>>>> ISP STREAM: tg is in list")
match = 1
touch('/tmp/streamthis-isp-lastrun')
# Connect the socket to the port where the server is listening
server_address = '/var/run/liquidsoap/isp.sock'
servers.append(server_address)
if tg in troy:
logger.info(">>>>> TROY STREAM: tg is in list")
match = 1
touch('/tmp/streamthis-troy-lastrun')
# Connect the socket to the port where the server is listening
server_address = '/var/run/liquidsoap/troy.sock'
servers.append(server_address)
if tg in fddisp:
logger.info(">>>>> FIRE STREAM: tg is in list")
match = 1
touch('/tmp/streamthis-fddisp-lastrun')
# Connect the socket to the port where the server is listening
server_address = '/var/run/liquidsoap/fddisp.sock'
servers.append(server_address)
if tg in belleville:
logger.info(">>>>> BELLEVILLE STREAM: tg is in list")
match = 1
touch('/tmp/streamthis-belleville-lastrun')
# Connect the socket to the port where the server is listening
server_address = '/var/run/liquidsoap/bellevile.sock'
servers.append(server_address)
if tg in swansea:
logger.info(">>>>> SWANSEA STREAM: tg is in list")
match = 1
touch('/tmp/streamthis-swansea-lastrun')
# Connect the socket to the port where the server is listening
server_address = '/var/run/liquidsoap/swansea.sock'
servers.append(server_address)
if tg in estl:
logger.info(">>>>> ESTL STREAM: tg is in list")
match = 1
touch('/tmp/streamthis-estl-lastrun')
# Connect the socket to the port where the server is listening
server_address = '/var/run/liquidsoap/estl.sock'
servers.append(server_address)
if tg in scclaw:
logger.info(">>>>> SCCLAW STREAM: tg is in list")
match = 1
touch('/tmp/streamthis-scclaw-lastrun')
# Connect the socket to the port where the server is listening
server_address = '/var/run/liquidsoap/scclaw.sock'
servers.append(server_address)
if match == 0:
logger.debug(">>>>> STREAMTHIS: tg is not in the list")
exit()
for server_address in servers:
logger.info('%s %s', "sending to: ", server_address)
# Create a UDS socket
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
try:
sock.connect(server_address)
except socket.error, msg:
print >>sys.stderr, msg
sys.exit(1)
try:
# Send data
message = 'queue.push {0}\n\r'.format(filepath)
sock.sendall(message)
amount_received = 0
amount_expected = len(message)
data = sock.recv(16)
amount_received += len(data)
#print >>sys.stderr, 'received "%s' % data
finally:
#print >>sys.stderr, 'closing socket'
match = 0
sock.close()
Example Log Output
As you can see, some talkgroups are in more than one stream. An example would be East St. Louis Police. You can see below they are in the "East St. Louis Fire, Police and EMS" stream as well as the "St. Clair County Law Enforcement" channel.
Mar 31 15:54:28 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:28.007373] (info) Running upload script: ./encode_upload.sh /tmp/capture/stclairco/2018/3/31/8619-1522529624_8.53138e+08.wav & Mar 31 15:54:28 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:28.044887] (info) Currently Active Calls: 3 Mar 31 15:54:28 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:28.045001] (info) #011[ 2 ] State: 3 Mar 31 15:54:28 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:28.045039] (info) #011[ 3 ] State: 3 Mar 31 15:54:28 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:28.045073] (info) #011[ 4 ] State: 3 Mar 31 15:54:28 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:28.045086] (info) Recorders: Mar 31 15:54:28 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:28.045096] (info) [ airspy ] Mar 31 15:54:28 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:28.045107] (info) [ 0 ] State: 2 Mar 31 15:54:28 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:28.045120] (info) [ 1 ] State: 2 Mar 31 15:54:28 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:28.045133] (info) [ 2 ] State: 3 Mar 31 15:54:28 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:28.045145] (info) [ 3 ] State: 3 Mar 31 15:54:28 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:28.045156] (info) [ 4 ] State: 3 Mar 31 15:54:28 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:28.045167] (info) [ 5 ] State: 2 Mar 31 15:54:28 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:28.045179] (info) [ 6 ] State: 2 Mar 31 15:54:28 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:28.045191] (info) [ 7 ] State: 2 Mar 31 15:54:28 medrc-recorder-1 encode_upload: Submitting [8619] T8 Bowman Mar 31 15:54:34 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:34.011654] (info) Running upload script: ./encode_upload.sh /tmp/capture/stclairco/2018/3/31/7951-1522529659_8.51175e+08.wav & Mar 31 15:54:34 medrc-recorder-1 encode_upload: Submitting [7951] SIUE Police Mar 31 15:54:39 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:39.012561] (info) Running upload script: ./encode_upload.sh /tmp/capture/stclairco/2018/3/31/7993-1522529663_8.5375e+08.wav & Mar 31 15:54:39 medrc-recorder-1 encode_upload: Submitting [7993] Monr SheriffLaw Mar 31 15:54:48 medrc-recorder-1 recorder[1166]: [2018-03-31 15:54:48.009675] (info) Running upload script: ./encode_upload.sh /tmp/capture/stclairco/2018/3/31/7027-1522529628_8.5385e+08.wav & Mar 31 15:54:48 medrc-recorder-1 encode_upload: Submitting [7027] SCC Law Disp 4 Mar 31 15:54:48 medrc-recorder-1 streamthis: >>>>> SCCLAW STREAM: tg is in list Mar 31 15:54:48 medrc-recorder-1 streamthis: sending to: /var/run/liquidsoap/scclaw.sock Mar 31 15:55:01 medrc-recorder-1 CRON[10784]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) Mar 31 15:55:11 medrc-recorder-1 recorder[1166]: [2018-03-31 15:55:11.012547] (info) Running upload script: ./encode_upload.sh /tmp/capture/stclairco/2018/3/31/7027-1522529690_8.527e+08.wav & Mar 31 15:55:11 medrc-recorder-1 encode_upload: Submitting [7027] SCC Law Disp 4 Mar 31 15:55:11 medrc-recorder-1 streamthis: >>>>> SCCLAW STREAM: tg is in list Mar 31 15:55:11 medrc-recorder-1 streamthis: sending to: /var/run/liquidsoap/scclaw.sock Mar 31 15:55:14 medrc-recorder-1 recorder[1166]: [2018-03-31 15:55:14.012353] (info) Running upload script: ./encode_upload.sh /tmp/capture/stclairco/2018/3/31/7315-1522529684_8.53138e+08.wav & Mar 31 15:55:14 medrc-recorder-1 encode_upload: Submitting [7315] MadisonSheriff 1 Mar 31 15:55:40 medrc-recorder-1 recorder[1166]: [2018-03-31 15:55:40.011816] (info) Running upload script: ./encode_upload.sh /tmp/capture/stclairco/2018/3/31/7188-1522529722_8.5385e+08.wav & Mar 31 15:55:40 medrc-recorder-1 encode_upload: Submitting [7188] EStL Police 1 Mar 31 15:55:40 medrc-recorder-1 streamthis: >>>>> ESTL STREAM: tg is in list Mar 31 15:55:40 medrc-recorder-1 streamthis: >>>>> SCCLAW STREAM: tg is in list Mar 31 15:55:40 medrc-recorder-1 streamthis: sending to: /var/run/liquidsoap/estl.sock Mar 31 15:55:40 medrc-recorder-1 recorder[1166]: [Errno 111] Connection refused Mar 31 15:55:44 medrc-recorder-1 recorder[1166]: [2018-03-31 15:55:44.011828] (info) Running upload script: ./encode_upload.sh /tmp/capture/stclairco/2018/3/31/7027-1522529730_8.517e+08.wav & Mar 31 15:55:44 medrc-recorder-1 encode_upload: Submitting [7027] SCC Law Disp 4 Mar 31 15:55:44 medrc-recorder-1 streamthis: >>>>> SCCLAW STREAM: tg is in list Mar 31 15:55:44 medrc-recorder-1 streamthis: sending to: /var/run/liquidsoap/scclaw.sock Mar 31 15:55:50 medrc-recorder-1 recorder[1166]: [2018-03-31 15:55:50.007195] (info) Running upload script: ./encode_upload.sh /tmp/capture/stclairco/2018/3/31/7315-1522529731_8.53138e+08.wav & Mar 31 15:55:50 medrc-recorder-1 encode_upload: Submitting [7315] MadisonSheriff 1 Mar 31 15:55:51 medrc-recorder-1 recorder[1166]: [2018-03-31 15:55:51.003113] (info) Running upload script: ./encode_upload.sh /tmp/capture/stclairco/2018/3/31/7028-1522529744_8.5375e+08.wav & Mar 31 15:55:51 medrc-recorder-1 encode_upload: Submitting [7028] Caseyville Fire Mar 31 15:55:51 medrc-recorder-1 streamthis: >>>>> FIRE STREAM: tg is in list Mar 31 15:55:51 medrc-recorder-1 streamthis: sending to: /var/run/liquidsoap/fddisp.sock Mar 31 15:55:53 medrc-recorder-1 recorder[1166]: [2018-03-31 15:55:53.013964] (info) Running upload script: ./encode_upload.sh /tmp/capture/stclairco/2018/3/31/7096-1522529737_8.5385e+08.wav & Mar 31 15:55:53 medrc-recorder-1 encode_upload: Submitting [7096] SAFB RampNet
Improved Multiple Stream Support
Combined the encode_upload.sh and the streamthis.py scripts into a single script. Added an additional column to the ChannelList.csv file with a pipe delimited list of streams for each talkgroup.
/opt/recorder/ChannelList.csv
Decimal,Hex,Mode,Alpha Tag,Description,Tag,Tag,Priority,Stream List 7199,1C1F,D,O Fallon Emergency Button,O Fallon Emergency Button,Law Dispatch,Law Dispatch,0,ofallon|scclaw 7219,1C33,D,Fairview Heights Emergency Button,Fairview Heights Emergency Button,Emergency Ops,Emergency Ops,15,fairview|scclaw 7235,1C43,De,Belleville Emergency Button,Belleville Emergency Button,Law Tac,Law Tac,30,belleville|scclaw
/opt/recorder/encode_upload.py
#!/usr/bin/env python3
import socket
import sys
import os
from subprocess import call
from subprocess import PIPE, run
import logging
from logging.config import fileConfig
fileConfig('logging_config.ini')
logger = logging.getLogger('streamthis')
CSV_FILE="/opt/recorder/ChannelList.csv"
FILE_TO_ENCODE=sys.argv[1]
logger.debug("file to encode: %s", FILE_TO_ENCODE)
def touch(fname, times=None):
with open(fname, 'a'):
os.utime(fname, times)
WAV_FILE=os.path.basename(FILE_TO_ENCODE)
logger.debug("WAV_FILE: %s", WAV_FILE)
DIRECTORY=os.path.dirname(FILE_TO_ENCODE)
logger.debug("DIRECTORY: %s", DIRECTORY)
a = WAV_FILE.split('.')
FILENAME = a[0]
logger.debug("FILENAME: %s", FILENAME)
MP3_FILE="{0}/{1}.mp3".format(DIRECTORY, FILENAME)
logger.debug("MP3_FILE: %s", MP3_FILE)
a = FILENAME.split('-')
TALKGROUP = a[0]
logger.debug("TALKGROUP: %s", TALKGROUP)
command = ['/bin/grep', TALKGROUP, CSV_FILE]
result = run(command, stdout=PIPE, stderr=PIPE, universal_newlines=True)
#print(result.returncode, result.stdout, result.stderr)
csvline = result.stdout
logger.debug("csvline: %s", csvline)
a = csvline.split(',')
ALPHA, STREAM_LIST = a[3], a[8]
logger.debug("ALPHA: %s", ALPHA)
logger.debug("STREAM_LIST: %s", STREAM_LIST)
if (STREAM_LIST.strip() == ''):
logger.info("No Stream for [%s] %s", TALKGROUP, ALPHA)
quit(0)
logger.debug("Calling: %s %s %s %s %s %s %s", "/usr/bin/lame", "--quiet", "--preset", "voice", "--tt", "\"{0}\"".format(ALPHA), FILE_TO_ENCODE)
call(["/usr/bin/lame", "--quiet", "--preset", "voice", "--tt", "\"{0}\"".format(ALPHA), FILE_TO_ENCODE])
if os.path.exists(FILE_TO_ENCODE):
os.remove(FILE_TO_ENCODE)
streams = STREAM_LIST.split('|')
servers = []
for stream in streams:
logger.debug(">>>>> %s, %s is in the list", stream, TALKGROUP)
touch("/tmp/streamthis-{0}-lastrun".format(stream))
stream_address = "/var/run/liquidsoap/{0}.sock".format(stream)
logger.debug("[%s] matched for %s, sending to: %s", TALKGROUP, stream, stream_address)
servers.append(stream_address)
if len(servers) < 1:
logger.error("Talkgroup [%s] did not have any streams in the list: %s", TALKGROUP, STREAM_LIST)
exit(1)
for server_address in servers:
logger.debug("address: %s", server_address)
# Create a UDS socket
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
try:
sock.connect(server_address)
except socket.error as msg:
sys.exit(1)
try:
# Send data
logger.info('%s %s %s', TALKGROUP, "sending to: ", server_address)
logger.debug('queue.push {0}\n\r'.format(MP3_FILE))
message = 'queue.push {0}\n\r'.format(MP3_FILE)
sock.sendall(message)
amount_received = 0
amount_expected = len(message)
data = sock.recv(16)
amount_received += len(data)
finally:
match = 0
sock.close()
Example Log Output
Dec 31 23:51:48 medrc-recorder-1 recorder[1433]: [2018-12-31 23:51:48.007674] (info) Running upload script: ./encode_upload.sh /tmp/capture/stclairco/2018/12/31/7057-1546321892_8.5375e+08.wav & Dec 31 23:51:48 medrc-recorder-1 encode_upload: lame --quiet --preset voice --tt SCC IREACH Patch /tmp/capture/stclairco/2018/12/31/7057-1546321892_8.5375e+08.wav Dec 31 23:51:48 medrc-recorder-1 encode_upload: Submitting [7057] SCC IREACH Patch to scclaw
Hardware Used
https://www.amazon.com/hz/wishlist/ls/1T3Q6JW59DL1H?ref_=wl_share
Code to Drop if Queue is Full
Big thanks to the above contributors. You examples are great. However my system is a very large metro area and without logic to limit the LiquidSoap queue, entries get stale very quickly. The following is crude code I wrote to enable a queue awareness to prevent adding more entries if queue is above a coded value. Once upon a time I wrote crude Perl scripts to do automation and this annoyance allowed me to learn a bit of python. My apologies to those whom know better.
It's a small thing but maybe it will help someone down the road if faced with a similar issue. The liquidsoap documentation was a struggle.
The logic is simple, but a quick rundown might help. First file is to give us an easy way to see when a file was recorded when it plays. Second file shows modifications to enable the telnet server on Liquidsoap and how I substituted shoutcast for icecast. Third file is where the work is done. Define a new array of tg that will always be added to the queue. Add a telnet job to gather the current secondary queue depth. Compare that value with an arbitrary number and drop if greater than.
========================================
~/trunk-build/encode-local-sys-0.sh
========================================
#(Optional:) Tack a timestamp on to the end of the ALPHA title for troubleshooting queue delay
ALPHA=`grep "^${tg}," ${CSV_FILE} | cut -d , -f 4` #as in original
CTIME=`date +%R`
FULLTAG="$ALPHA , Recorded at:$CTIME"
lame --nohist --gain 8 --preset voice --tt "${FULLTAG}" $filename $mp3encoded
========================================
~/liquid/startliq.sh
========================================
#Add a telnet interface to allow queue querries
set("server.telnet",true)
set("server.telnet.bind_addr","192.168.1.x")
#Modified for Shoutcast
output.shoutcast(%mp3(stereo=false, bitrate=16, samplerate=22050), host="localhost",
port = 8000, password = "YOURPASS", name="STREAMNAME", stream)
========================================
~/liquid/streamthis
========================================
import telnetlib #add telnet module
# This array defines all of the decimal trunk-groups you want to inject into the stream queue:
streamthese = ["9101", "9102", "9103", "9104","9105","9106","9107","9108","9109","9112","9145"] #previously existed
queuepri = ["9112", "9145"] #these are new to override queue dump further on
#below lines exist already, for landmarks
if tg in streamthese:
<clip>
else:
<clip>
#end existing lines
#Next part goes below the above reference if/else, so only runs if get through that.
#New code to telnet to LiquidSoap and see how deep secondary queue is and dump/exit if queue too big, unless given a pass
HOST = "192.168.1.x"
tn = telnetlib.Telnet(HOST, 1234)
tn.write("\nlist\n")
tn.read_until("END")
tn.write("queue.secondary_queue\n")
secq = tn.read_until("END")
## When there is no secondary queue will return 1 for the END. n-1 will be actual queue size
seclist = secq.split()
print ">>>>> STREAMTHIStel: Sec len is ", len(seclist)
if len(seclist) > 4 and tg not in queuepri:
print ">>>>> STREAMTHIStel: Queue bigger than 3 and tg not priority"
exit()
Code to work with newer Python3 and liquidsoap 2.x
I recently updated to Debian 12 and found that the version of liquidsoap was updated to 2.1.3 and python 2.7 was no longer available so I redid the streamthis script to fix both. Have fun Scott KB2EAR
#!/usr/bin/python3
import socket
import sys
import os
filepath = sys.argv[1]
filename = os.path.basename(filepath)
a = filename.split('-')
tg, _ = a[0], a[1]
def touch(fname, times=None):
with open(fname, 'a'):
os.utime(fname, times)
# for debug purposes
print("tg is %s" % tg)
# This array defines all of the decimal trunk-groups you want to inject into the stream queue:
streamthese = ["21101", "21201", "21301"]
if tg in streamthese:
print(">>>>> STREAMTHIS: tg is in list")
match = 1
touch('/tmp/streamthis-lastrun')
else:
print(">>>>> STREAMTHIS: tg is not in the list")
match = 0
# silently exit
sys.exit()
# Create a UDS socket
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
# Connect the socket to the port where the server is listening
server_address = '/tmp/socket'
print('connecting to %s' % server_address, file=sys.stderr)
try:
sock.connect(server_address)
except socket.error as msg:
print(msg, file=sys.stderr)
sys.exit(1)
try:
# Send data
#new format for liquidsoap 2.x
message = 'request.queue_0.push {0}\n'.format(filepath)
#old format for liquidsoap 1.x
#message = 'queue.push {0}\n\r'.format(filepath)
print("message: {0}".format(message))
sock.send(message.encode())
amount_received = 0
amount_expected = len(message)
data = sock.recv(16)
amount_received += len(data)
print('received "%s' % data.decode('utf-8'))
finally:
print('closing socket', file=sys.stderr)
match = 0
sock.close()