Skip to content
Home » GPS-Arduino Integration – Serial Buffer Overflow Challenges

GPS-Arduino Integration – Serial Buffer Overflow Challenges

Currently, I’m engaged in a project involving the development of a self-driving tank. To achieve autonomous navigation, the tank relies on a GPS system to determine its precise location on the Earth. Additionally, due to the project’s complexity and the need for multiple simultaneous operations, we’ve opted to use a RTOS. Specifically, we have selected the open-source FreeROTS, to effectively manage numerous concurrent tasks. Given the RTOS nature of the project, the GPS-Arduino Integration has thrown up several challenges, especially with regards to the serial buffer overflowing.

Initially, I thought integrating the TinyGPS++ and FreeRTOS on an Arduino (or in this case, an STM32F401 using the STM32duino framework), would be as simple as refactoring the TinyGPS++ example sketch to include the relevant FreeRTOS delays etc. How wrong I was.

Baseline Implementation

The project currently makes use of the BN880 GPS, which is connected to the Arduino using a Hardware Serial port. The GPS will then provide GPS information (latitude, Longitude, Altitude, Speed…) using NMEA sentences. The Arduino will then decode these sentences, to do this the TinyGPS++ library is employed.

For the initial testing phase, I opted to use the example provided on the TinyGPS++ GitHub page and customized it to include the necessary FreeRTOS code. The GPS is set to transmit updates to the Arduino at a frequency of 10Hz. Based on this, I determined that a loop delay of 100ms (1000ms/10Hz) would suffice.

Incorporating the required FreeRTOS code and the necessary loop timer into the provided example, I created the following code:

void position_task(void* args)
{
    TinyGPSPlus gps;
    HardwareSerial gpsSerial(UART2_RX, UART2_TX);
    gpsSerial.begin(57600);
    for(;;){
        while(gpsSerial.available() > 0){
            if(gps.encode(gpsSerial.read())){
                if (gps.location.isValid())
                {
                    Serial.print(gps.location.lat(), 6);
                    Serial.print(F(","));
                    Serial.print(gps.location.lng(), 6);
                    Serial.print(F(","));
                    Serial.print(gps.location.age());
                    Serial.print(F(","));
                    Serial.print(gps.altitude.meters());
                    Serial.print(F(","));
                    Serial.println(gps.failedChecksum());
                }
            }
        }
        vTaskDelay(100 / portTICK_PERIOD_MS);
    }
}

From this code, the following results were ascertained (Precise Latitude and Longitude Hidden for privacy):

Latitude	Longitude	Age (ms)	No. Checksum Failed
50.97****,	-2.77****,	0,	        322
50.97****,	-2.77****,	0,	        322
50.97****,	-2.77****,	101,	    322
50.97****,	-2.77****,	0,	        322
50.97****,	-2.77****,	0,	        322
50.97****,	-2.77****,	101,	        322
50.97****,	-2.77****,	0,	        322
50.97****,	-2.77****,	0,	        322
50.97****,	-2.77****,	501,	        323
50.97****,	-2.77****,	2001,	        328
50.97****,	-2.77****,	2601,	        330
50.97****,	-2.77****,	5601,	        340
50.97****,	-2.77****,	8101,	        346
50.97****,	-2.77****,	8601,	        346
50.97****,	-2.77****,	9601,	        349
50.97****,	-2.77****,	10101,	        350

Initially, the outcomes seemed promising as it successfully determined a valid position – a great start! However, concerns arose upon examining the number of messages that failed the checksum. Additionally, worries escalated as the age of the GPS position fix quickly rose from approximately 0ms to 101ms and then to over 10 seconds. Typically, a GPS position is deemed unreliable if the fix is older than 1.5 seconds, making a position age exceeding 10 seconds worrisome, to say the least!

The reasons why this occurs and a workable solution is presented in the following paragraphs.

Background

UART Based GPS units and NMEA Sentences

Most common GPS units (especially those intended for hobbyists) provide GPS data through a UART connection. The GPS data is encoded according to the NMEA standards.

Decoding the NMEA Sentences is outside the scope of this article, but example NMEA Sentences are shown below (My Longitude and Latitude Removed for privacy):

  • $GNGGA,1727,50**.*****,N,002**.*****,W,2,12,0.87,27.0,M,48.7,M,,00006A
  • $GN,A,3,21,02,08,32,14,27,03,22,17,,,,1.52,0.87,1.241F
  • $GNGSA,A,3,0,86,69,76,71,87,,,,,,1.52,0.87,1.2411
  • $GPGSV,4,1,14,01,,,42,02305,36,03,35,216,41,08,43,157,374C
  • $GPGSV,4,2,14,10,17,050,,14,85,38,17,20,309,25,21,80,067,2973
  • $GPGSV,4,3,14,22,25,309,22,24,002,,27,11,144,34,32,33,069,267B
  • $GPGSV,4,4,14,36,27,148,39,49170,3574
  • $GLGSV,3,1,10,69,14,025,10,70,60,087,18,71,36,165,40,7,256,3967
  • $GLGSV,3,2,10,77,23,302,,78,07,358,,85,50,091,31,86,66,2861
  • $GLGSV,3,3,10,87,16,298,22,,,,1759
  • $GNGLL,5058.69172,N246.21290,W,172706.50,A,D*6B

The NMEA standard states that each sentence is required to be less than 82 bytes in length, so clearly, one fully utilised NMEA sentence will cause the default Arduino Serial buffer to overrun.

Arduino UART (Serial) Implementation

In the background, the Arduino framework handles the reception and temporary storage of all UART data, utilizing a storage location referred to as the UART buffer. As an end user, you can access this buffer using the Serial.read() command. However, a drawback of relying on Arduino for UART reception is the default constraint on the buffer size. limited to a maximum of 32 bytes. If incoming data exceeds this buffer’s capacity before it’s emptied, there’s a risk of overwriting data at the beginning of the UART buffer. This will result in lost data / corrupted NMEA sentences.

To address this limitation, it’s possible to expand the buffer size using a custom build flag. It’s important to note that if your Arduino board supports multiple UART ports, such as the Arduino MEGA, increasing the buffer size will increase RAM usage. On an Arduino Mega, expanding the buffer size will raise RAM usage by a factor of 8 due to its 8 serial ports.

TinyGPS++

The majority of Arduino users rely on the open-source TinyGPS++ library to interpret NMEA sentences. The provided example in the library documentation suggests clearing the Arduino Serial buffer each time the main loop iterates. As you clear the loop the example code tells you to encode the characters using the gps.encode() function. This temporarily moves the content of the serial bus into TinyGPS++ own buffer, once a complete NMEA sentence is received, the library will then process it to ascertain the position etc.

While this method may be adequate for situations where the main loop handles only a few tasks, it can present challenges in scenarios involving extensive main loop activity or when employing a Real-Time Operating System (RTOS) for managing concurrent tasks. In such instances, managing GPS encoding in this manner may introduce significant overhead, potentially impacting the execution of other tasks and increasing the risk of overflow in the Arduino Serial buffer. In the event of an overflow in the Arduino Serial buffer, a large number of invalid sentences will probably be received. Consequently, this may lead to corrupted sentences and ultimately cause the checksum validation to fail, resulting in a significant increase in the position age.

Overflow Solutions

The issues surrounding the Serial Buffer overflowing can be mitigated in one of three ways:

  1. Decreasing the refresh period of the GPS – Instructions for doing so here.
  2. Increase the rate at which the Serial Buffer is emptied. This can be done by reducing the number of blocking tasks within the main loop.
  3. Increase the size of the Serial Buffer – This can be done in one of two ways:
    • If using the Arduino IDE, you can change the value of #define SERIAL_RX_BUFFER_SIZE 64 in HardwareSerial.h. Hint, the file is located here: Program Files (x86)/Arduino/hardware/arduino/avr/cores/arduino/HardwareSerial.h
    • If you’re using platform.io (you should be, it’s amazing!), you can just add the following build flag to the platformio.ini file: SERIAL_RX_BUFFER_SIZE=256

The solution I propose to the issues described above, is, in fact a combination of Mitigation options 2 and 3. How much weight you give to each of the two options will depend on the specific microcontroller you have.

I’ve utilized the STM32F401 microcontroller with the STM32duino framework (note that, for practical purposes, the Arduino and STM32duino frameworks are essentially identical). I achieved good results by increasing the buffer size to 256 bytes and reading from the buffer every 25ms. I selected the buffer size by considering the RAM usage and also the amount of CPU time which would be dedicated to reading the serial buffer. Decreasing the CPU time dedicated to reading from the serial buffer means additional RAM is required to be reserved for the serial buffer. The reverse is also true.

The code I used is shown below:

void position_task(void* args)
{
    TinyGPSPlus gps;
    HardwareSerial gpsSerial(UART2_RX, UART2_TX);
    gpsSerial.begin(57600);
    for(;;){
        while(gpsSerial.available() > 0){
            if(gps.encode(gpsSerial.read())){
                if (gps.location.isValid())
                {
                    Serial.print(gps.location.lat(), 6);
                    Serial.print(F(","));
                    Serial.print(gps.location.lng(), 6);
                    Serial.print(F(","));
                    Serial.print(gps.location.age());
                    Serial.print(F(","));
                    Serial.print(gps.altitude.meters());
                    Serial.print(F(","));
                    Serial.println(gps.failedChecksum());
                }
            }
        }
        vTaskDelay(25 / portTICK_PERIOD_MS);
    }
}

I then set the platformiol.ini configuration as follows:

build_flags =
    -D SERIAL_RX_BUFFER_SIZE=256

Implementing the above code, yeilds the following results:

Latitude	Longitude	Age (ms)	No. Checksum Failed
50.97****,	-2.77****,	0,	        0
50.97****,	-2.77****,	25,	        0
50.97****,	-2.77****,	50,	    0
50.97****,	-2.77****,	75,	        0
50.97****,	-2.77****,	101,	        0
50.97****,	-2.77****,	25,	        0
50.97****,	-2.77****,	50,	        0
50.97****,	-2.77****,	75,	        0
50.97****,	-2.77****,	101,	        0
50.97****,	-2.77****,	25,	        0
50.97****,	-2.77****,	50,	        0
50.97****,	-2.77****,	75,	        0
50.97****,	-2.77****,	100,	        0
50.97****,	-2.77****,	25,	        0
50.97****,	-2.77****,	50,	        0
50.97****,	-2.77****,	101,	        0

On inspection of the new results, it can be seen that no NMEA messages fail the checksum. Furthermore, the age of the GPS position fix doesn’t exceed 101ms.

Conclusion

This article has delved into the potential complications arising from irregular clearing of the serial buffer and the subsequent impact on the accuracy of GPS position calculation. Moreover, it has provided comprehensive mitigation strategies to avert such issues.

Leave a Reply