Exporting data

Influx data

You can use lti-get-influx-data.sh on your analyzer host to export data in CSV format.

Getting the script

Open a terminal session on your analyzer host.

Download the script:

wget https://api.latence.ca/software/lti-get-influx-data.sh

Make it executable:

chmod +x lti-get-influx-data.sh


Call ./lti-get-influx-data.sh with the following parameters:

  1. Agent ID

  2. Protocol name -- one of twamp, icmp, tcp, http, https

  3. Seconds of data to extract -- up to 3 months

  4. You can add --drops at the end of the line to dump drops (if there are) instead of latency values

For instance, running ./lti-get-influx-data.sh 100 tcp 10 will extract the last 10 seconds of tcp latency data for agent id 100. With a sampling rate set to 2 seconds, the result will be something similar to the block below:


You have access to 4 fields :
_time : The exact time the measurement has been done
_value : The latency measured (in ms) or the number of paquet drops
_measurement : Indicate the type of protocol and what kind of value is displayed in the value field : latency (protocolname_result) or drops (protocolname_drop)
agentID : The agent ID used to do the measurement

You may redirect the data into a file using the > redirection.

For example, the command ./lti-get-influx-data.sh 100 tcp 10 > tcp-data.csv will create a text file named tcp-data.csv, containing the dumped tcp data.

Kafka data

You can use kafka-reader program to export Kafka data in CSV format.

Getting the program

Open a terminal session.

Download the files:

wget https://api.latence.ca/software/kafka-reader-files.zip

Extract it:

unzip kafka-reader-files.zip


  1. Go into the kafka-reader directory

    cd kafka-reader-files

  2. Configure the environment variables. Change the kafka.env file and modify the last three variables according to your need. The available topics are twamp_influx, icmp_influx, iperf_influx, traceroute_influx, tcp_influx, udp_influx, http_influx, https_influx.


    Run the following command to load the variables.

    export $(cat kafka.env | xargs)

  3. Install the requirements.dev.txt packages

    pip3 install -r requirements.dev.txt

  4. Change the /etc/hosts to point msgbus.latence.ca to the analyzer host by adding the following line:

    <Replace with analyzer IP> msgbus.latence.ca

  5. Run the program to generate CSV

    python3 run.py

Latency and Geolocation data

You can use the following program to export Latency and Geolocation data in CSV format.

Getting the program

Open a terminal session.

Download the files:

wget https://api.latence.ca/software/latency-geoloc-csv-generator.zip

Extract it:

unzip latency-geoloc-csv-generator.zip


  1. Go into the latency-geoloc-csv-generator directory

    cd latency-geoloc-csv-generator

  2. Install the requirements.dev.txt packages

    pip3 install -r requirements.dev.txt

  3. Use the following command to extract the data.

    python3 csv-writing-script.py <Influx IP> <start time> <end time> <AgentID>

    • Influx IP - IP address of InfluxDB
    • start time - Start time in GMT using the format 2023-06-21T22:00:00Z
    • end time - End time in GMT in the format 2023-06-23T22:00:00Z
    • AgentID - ID of the agent


    python3 csv-writing-script.py 2023-06-21T22:00:00Z 2023-06-23T22:00:00Z 1

Troubleshooting data extraction

Influx data fetch script

Permission denied

If you get the following output:

-bash: ./lti-get-influx-data.sh: Permission denied**

Make sure you have run chmod +x ./lti-get-influx-data.sh

No data

  • Make sure you are using the right agent id

  • Make sure you have requested a sufficient number of seconds of data