OpenStreetMap logoOpenStreetMapNederland

Divide and map. Now. -- load testing

Posted by qeef on 6/4/2020

I run load testing with locust.io on the damn server, and I think it’s usable for mapathon with 100 mappers.

The damn server is part of the Divide and map. Now. – the damn project I have published at the beginning of the year. And I am still interested in the performance. I was wondering if I can simulate a mapathon event.

Setup

I run the damn server on $5 per month VPS of do.co. The server runs Debian 10.2 x64, 1 vCPU, and 1GB / 25GB Disk. There is nothing special about the deployement I described in the last diary.

For load testing, I prepared the area. I uploaded a big area (JOSM measurement plugin says it’s 42 852 square km) with the damn manager, that split it to 654 squares.

Then, I simulated a mapathon in the locust file. The “participating” test mapper can choose from:

  1. Get the area and all its commits. (In fact, all commits wouldn’t be used in a real client twice, since since for an area is implemented.)

  2. Request a random square for mapping. If the square is available, work on it for between 1 and 5 seconds and then stop with need more mapping (30 %), split the square (10 %), or ready for review (60 %). (In the brackets, there is a probability of stop reason used.)

  3. Request a random square for review. If the square is available, work on it for between 1 and 5 seconds and then stop with need more mapping (30 %), please, review again (10 %), or done (60 %).

When (1), (2), or (3) finishes, wait for between 1 and 2 seconds and choose again. (2) and (3) are chosen five times more often than (1).

Testing 100 mappers

I tested for ten minutes. There were ten new mappers per second until a maximum of 100 mappers.

  • Median Response Time: 2900 ms
  • Average Response Time: 6052 ms
  • Min Response Time: 177 ms
  • Max Response Time: 70272 ms
  • Requests/s: 12.10

The total request count was 7 310. And there was no error!

Testing 500 mappers

I tested for ten minutes again. There were 50 new mappers per second until a maximum of 500 mappers.

  • Median Response Time: 16 000 ms
  • Average Response Time: 30 431 ms
  • Min Response Time: 175 ms
  • Max Response Time: 375 465 ms
  • Requests/s: 13.99

The total request count was 8 499. There were 2.35 failures per second.

I don’t like failures, so what were these errors? Connection refused, connection closed without response, and not found url. I think that I just overloaded the $5 per month server.

Conclusion

First, I don’t think that the average response time of 6 seconds is fantastic. However, I don’t believe that mappers at mapathon work on a square for 1 to 5 seconds, either.

The damn server currently online is not ready for big loads. It looks like one vCPU can’t handle 500 mappers. However, the damn project aims at local communities. Moreover, I have no ambitions to run a production server on my own.

During load testing, I found some database access bugs and fixed them.

The last thing – I would like to see some similar (comparable) load testing of HOT Tasking Manager. I gave up on trying. I made the Tasking Manager run but had problems with login. And I have no idea how to run queries from 100 test mappers anyway. (No, I don’t think that 1000 times GET / is comparable load testing.)