Parsing 3CX Call Records in Graylog

Introduction

The following is a transcript with configurations attached:

Hello, Jeff Darrington here, senior technical market manager at Graylog. In this video, I’m going to go over logging telecom call detail records into Graylog.

Setting Up the Dashboard

Let me bring up my Graylog, and what you’ll see is that I have created a dashboard for all of the call records coming out of a PBX. In this case, I am using 3CX call data records from their PBX. What I’m doing is logging:

– The number coming in
– The number landing on which extension
– Time answered
– Call duration

I’ve also created some widgets here to count the number of calls each extension has made or received, calls per month, calls per day, and even a small widget to track some of the longest calls. This helps in identifying unusually long calls that may need further investigation.

Configuring 3CX for CDR Logging

To do this in the 3CX call server, you configure a 3CX CDR service as a client, as an active socket, to an IP address on a specific port. What happens inside there is that the logs will be shipped in a comma-delimited format, with the fields you see in their field list.

 

The field list contains a lot of records. You can choose to eliminate or add the ones you want, but make sure you keep the order the same, because when you start parsing the data, the order is crucial.

Field Definitions

If you go to the 3CX website under the CDR records section, you’ll find the definition of all the different types of fields, which will help you understand what the data contains.

Creating a 3CX CDR Input in Graylog

In Graylog, I created a 3CX CDR input, which is simply a plain text TCP connection to port 3000.

Grok Pattern for Parsing

I then created a grok pattern called: 3CX_CDR. This pattern follows the order of the fields that appear inside the PBX system.

%{NUMBER:history_id},(?<call_id>[^,]*),%{TIME:duration},%{TIMESTAMP_ISO8601:time_start},%{TIMESTAMP_ISO8601:time_answered},%{TIMESTAMP_ISO8601:time_end},%{WORD:reason_terminated},(?<from_no>[^,]*),(?<to_no>[^,]*),(?<from_dn>[^,]*),(?<to_dn>[^,]*),(?<dial_no>[^,]*),(?<reason_changed>[^,]*),(?<final_number>[^,]*),(?<final_dn>[^,]*),(?<bill_code>[^,]*),(?<bill_rate>[^,]*),(?<bill_cost>[^,]*),(?<bill_name>[^,]*),(?<chain>[^,]*),(?<from_type>[^,]*),(?<to_type>[^,]*),(?<final_type>[^,]*),(?<from_dispname>[^,]*),(?<to_dispname>[^,]*),(?<final_dispname>[^,]*),(?<missed_queue_calls>[^,]*)

Fields available in order within the PBX System based on this grok pattern:

Then I created the Parsing Rule:

rule "Parse 3CX CDR GROK"
When
   true
       //Route 3CX CDR to Stream old:
then
    let grokp = grok(
        pattern:"%{3CX_CDR}",
        value:to_string($message.message),
        only_named_captures: true
        );
        
    set_fields(grokp);
    set_field("grok_parse",true);
end

It’s important that you don’t reorder these fields unless you also go into Graylog and reorder your grok pattern accordingly. Inside the rule, I’ve referenced the pattern so that when the data comes in, it automatically parses out the records.

 

Also, I added some parsing of the timestamp.

rule "Parse - 3cx - End Call TimeStamp Breakout"
When
    $message.grok_parse == true
then
    let grokp = grok(
        pattern:"%{TIMESTAMP_ISO8601}",
        value:to_string($message.time_end),
        only_named_captures: false
        );
        
    set_fields(fields:grokp,prefix:"TimeEnd_");
    set_field("grok_parse_timeend_timestamp",true);
    remove_field("TimeEnd_TIMESTAMP_ISO8601");
    remove_field("TimeEnd_MINUTE");
    remove_field("TimeEnd_SECOND");
end

 

Conclusion

Hopefully, that gives you a good idea of how to get call records into Graylog. There are many different PBX systems out there, and you can log them into Graylog using similar methods. Thanks for joining us, and happy logging with Graylog!