Actions

Difference between revisions of "Streaming with Trunk Recorder and Liquidsoap"

From The RadioReference Wiki

Line 198: Line 198:
 
try:
 
try:
 
     # Send data
 
     # Send data
    # annotate:title="StreamTitle content":/path/to/file.wav
 
    #message = 'queue.push annotate:title="{0}":{1}\n\r'.format(tag,filepath)
 
 
     message = 'queue.push {0}\n\r'.format(filepath)
 
     message = 'queue.push {0}\n\r'.format(filepath)
 
     print "message: {0}".format(message)
 
     print "message: {0}".format(message)

Revision as of 16:06, 9 July 2017

Introduction

Trunk Recorder is a free Linux based application written by Luke Berndt that can decode the following:

  • Trunked P25 & SmartNet Systems
  • Conventional P25 & analog systems, where each group has a dedicated RF channel
  • P25 Phase 1, P25 Phase 2 & Analog voice channels

Using a number of supported SDR hardware dongles Trunk Recorder can take entire P25 systems and simultaneously record every single transmission occurring at any moment in time. This is accomplished by using enough SDR dongles to cover the entire spectrum of voice and control channels in a system.

From a stream provider perspective this gives us the ability to put all transmissions into a stream without losing those that occurred awhile a physical scanner would be "stuck" on another channel. The cost is a slight delay caused by queuing the transmissions, but the severity of that queue depth is dependent upon how many talk-groups you stream and how chatty they are.

Now that Trunk Recorder supports analog its possible to combine analog and digital transmissions into the stream.

How It Works

Under the hood Trunk Recorder relies upon GNURadio and OP25. When you install Trunk Recorder you tell it where your system's control channels are, and you define the "center frequency" for your SDR dongles to cover the entire range of a given system. For example, using the Summit County Ohio Regional 800 System, with 20 some voice channels and four possible control channels the configuration looks something like this using four $30 SDR dongles to cover the entire RF range:

Each dongle has a "center" frequency and can listen to approximately 1mhz above and below that center. For brevity, one dongle looks like the section below. This system supports both digital and analog trunked talk-groups, hence the two types of "recorders" which are processes dedicated to listening to a voice channel, decoding the digital data into analog if necessary, and outputting it into a .wav file:

       {
        "center": 851418750.0,
        "rate": 2148000.0,
        "squelch": -60,
        "error": 0,
        "ppm": 0,
        "gain": 350,
        "antenna": "RX",
	"modulation": "fsk4",
        "digitalRecorders": 5,
        "analogRecorders": 2,
        "driver": "osmosdr",
        "device": "rtl=1"
    },

This block defines the possible control channels, defines it as a SmartNet system, instructs Trunk Recorder that the system is rebanded, and defines a script to run after each transmission is recorded.

    "systems": [
	{
        "control_channels": [858712500, 855262500, 857962500, 858512500],
        "type": "smartnet",
	"bandplan": "800_reband",
	"uploadScript": "encode-upload.sh",
        "talkgroupsFile": "ChannelList.csv"
	}]

Just like a a typical consumer grade scanner Trunk Recorder listens to the control channel, but since it has a receiver dedicated to that frequency it is always listening and never stops listening. This keeps it aware of every possible transmission on any of the voice channels, where as a normal scanner would leave the control channel to decode a transmission and potentially miss other simultaneous traffic.

Watching it's output we can see it start a recorder when TG 2384 and TG 33712 started talking:

(info)   [sys_1]   TG: 33712       Freq: 8.51562e+08     Call not found for Update Message, Starting one...
(info)   [sys_1]   TG: 33712       Freq: 8.51562e+08     Starting Recorder on Src: rtl=1
(info)     - Starting P25 Recorder Num [0] TG: 33712     Freq: 8.51562e+08       TDMA: false     Slot: 0
(info)   [sys_1]   TG: 2384        Freq: 8.55088e+08     Starting Recorder on Src: rtl=0
(info)     - Starting P25 Recorder Num [9] TG: 2384      Freq: 8.55088e+08       TDMA: false     Slot: 0
(info)   [sys_1]   TG: 33712       Freq: 8.51562e+08     Assign Retuning - New Freq: 8.5125e+08  Elapsed: 6s     Since update: 2s
(info)   [sys_1]   TG: 2352        Freq: 8.55512e+08     Ending Recorded Call - Last Update: 4s  Call Elapsed: 25
(info)     - Stopping P25 Recorder Num [10]        TG: 2352        Freq: 8.55512e+08       TDMA: false     Slot: 0
(info)   Running upload script: 
./encode-upload.sh /home/scanner/jradio/trunk-recorder/sys_1/2017/7/9/2352-1499630871_8.55512e+08.wav

As you can see from the text above the output is a wav file that may include a single transmission or, should there be very quick back-to-back transmissions that occur on the same voice channel, a string of them in a single .wav file. Each transmission also has an identically named .json file with metadata including the radio IDs and duration of transmission in the file:

{
"freq": 8.55512e+08,
"start_time": 1499630871,
"stop_time": 1499630896,
"emergency": 0,
"talkgroup": 2352,
"srcList": [ {"src": 7609, "time": 1499630871, "pos": 0.000000}, {"src": 24884, "time": 1499630873, "pos": 1.476000}, 
{"src": 4616, "time": 1499630879, "pos"
: 6.300000}, {"src": 7609, "time": 1499630888, "pos": 13.140000}, {"src": 2360, "time": 1499630889, "pos": 13.140000}, 
{"src": 7609, "time": 1499630889, "pos
": 13.140000}, {"src": 2360, "time": 1499630890, "pos": 14.400000} ],
"play_length": 15.300000,
"system": 2,
"freqList": [ { "freq": 855512500.000000, "time": 1499630871, "pos": 0.000000, "len": 123664.000000, 
"error_count": 677.000000, "spike_count": 0.000000} ]
}

How do we stream it?

liquidsoap is the "client" used to create a stream connection to the broadcastify mount point. When there is nothing in it's queue liquidsoap will send silence towards the stream.

Every time Trunk Recorder writes a .wav file it will optionally run a script called encode-upload.sh. See below for what the script normally is meant to do - but for the purposes of streaming we add a call to the encode-upload.sh script to run a separate script called streamthis.

Streamthis is a python script that compares the talk-group of a wav file (by looking at the prefix of the file name) to a list of defined interesting talk-groups. If the file just created is one we want to inject into our stream the python script will connect to an already running instance of liquidsoap and add the new .wav file to the queue.

If the stream is currently idle, the .wav file is immediately played into the stream. When the queue is empty, the stream sends silence towards Broadcastify.

Basic LiquidSoap Configuration

This liquidsoap script presumes you've already installed liquidsoap on the system that you're going to use to run Trunk Recorder. Instructions on installing LiquidSoap can be found here: Liquidsoap

Once installed, this script can be made executable and run on Linux to initiate your stream connection to Broadcastify. NOTE: This stream will connect and appear "Online" indefinately with zero audio until you build the logic to queue .wav files. The configuration below tells liquidsoap to listen on a socket for connections from the streamthis script:

#!/usr/bin/liquidsoap 
#                                  ^-- use -v for verbose otherwise tail the log
set("tag.encodings",["UTF-8","ISO-8859-1"])

# Log dir
set("log.file.path","./basic-radio.log")

# create a socket to send commands to this instance of liquidsoap
set("server.socket",true)
set("server.socket.path","./socket")

# This creates a 1 second silence period generated programmatically (no disk reads) 
silence = blank(duration=1.)

# This pulls the alpha tag out of the wav file 
def append_title(m) =
  # Grab the current title
	#title = '$(if $(title),"$(title)","..> Scanning <..")'
  #     # Return a new title metadata
	[("title",">> Scanning <<")]
end

silence = map_metadata(append_title, silence)

#
myqueue = request.queue()
myqueue = server.insert_metadata(id="S4",myqueue)

# If there is anything in the queue, play it.  If not, play the silence defined above repeatedly:
stream = fallback(track_sensitive=false, [myqueue, silence])

title = '$(if $(title),"$(title)","...Scanning...")'
stream = rewrite_metadata([("title", title)], stream)

output.icecast(%mp3(stereo=false, bitrate=16, samplerate=22050), host="audio1.broadcastify.com", 
 port = 80, password =  "YOURPASSWORD", genre="Scanner", description="Scanner audio", mount="/YOURMOUNTPOINT", 
 name="YOUR STREAM NAME HERE", stream) 

Streamthis Script

This is they streamthis python script that decides if a given .wav file just generated by Trunk Recorder should be injected into the liquidsoap queue.

The "touch" commands in this script are used by a watchdog process to see if the stream has died and are optional.

#!/usr/bin/python
import socket
import sys
import os

filepath = sys.argv[1]
filename = os.path.basename(filepath)
a = filename.split('-')
tg,junk = a[0],a[1]

def touch(fname, times=None):
    with open(fname, 'a'):
        os.utime(fname, times)

# for debug purposes
#print "tg is %s" % tg

# This array defines all of the decimal trunk-groups you want to inject into the stream queue:
streamthese = ["33328", "41168", "41200", "42576","42608","33936","41264","33904","33872","6","7","8"]

if tg in streamthese:
    print ">>>>> STREAMTHIS:   tg is in list"
    match = 1
    touch('/tmp/streamthis-lastrun')
else:
    print ">>>>> STREAMTHIS:   tg is not in the list"
    match = 0
    # silently exit
    exit()

tag=tg

# Create a UDS socket
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)

# Connect the socket to the port where the server is listening
server_address = '/PATH/TO/WHERE/LIQUIDSOAP/socket'
#print >>sys.stderr, 'connecting to %s' % server_address
try:
        sock.connect(server_address)
except socket.error, msg:
        print >>sys.stderr, msg
        sys.exit(1)

try:
    # Send data
    message = 'queue.push {0}\n\r'.format(filepath)
    print "message: {0}".format(message)
    #print >>sys.stderr, 'sending "%s' % message
    sock.sendall(message)

    amount_received = 0
    amount_expected = len(message)
    
    data = sock.recv(16)
    amount_received += len(data)
    #print >>sys.stderr, 'received "%s' % data

finally:
    #print >>sys.stderr, 'closing socket'
    match = 0
    sock.close()