Green-Light Optimized Speed Advisory for Cycling in Hamburg

Hurrying up right before an intersection just to narrowly miss the green light by a few seconds is one of the annoying things which regularly happen when cycling in the city.

Wouldn’t it be just great to know in advance whether a green phase may be catched, and to avoid useless acceleration and waiting phases?

Over the last years, the city of Hamburg (Germany) has put some significant work into their Urban Data Platform and the geo-portal, improving the accessibility of various data sources.

In particular, the Traffic Lights Data Hamburg beta API provides access to real-time signal data of traffic-light systems for a large number of intersections in the city center.

So let’s see what we can get out of that data, and whether we may ease some pain of the daily ride.

Green-light optimized speed advisory

The core idea of green-light optimized speed advisory (“GLOSA”) is to reduce unnecessary stops and accelerations by calculating reasonable movement speeds based on expected future green-light periods of upcoming traffic-lights.

Recently, a couple of implementations have been developed in different cities.
Some examples:

• Gevas launched their app traffic pilot which is available i.a. in Duesseldorf, Frankfurt, Salzburg and Kassel
• Audi provides a GLOSA traffic assistant for their cars in selected cities (i.a. Ingolstadt, Duesseldorf, and New York (US))
• the PrioBike project in Hamburg includes the PrioBike app as well as the PrioBike-Säule (see photo below; a fixed post comprised of a sensor and display to show speed advice to cyclists passing by).
  The PrioBike app is developed by the TU Dresden and is currently (06/23) in closed beta (unfortunately, my registration mail has not been answered yet, however, an official statement confirms a planned release later in 2023).
  The fixed post has been developed together with the company yunex traffic.

Unfortunately, currently (06/2023) none of the app solutions is publically available in Hamburg.

One “PrioBike-Säule” has been installed close to the Hamburg-Dammtor train station, and judging from limited own experience it seems to work pretty well!
(3 out of 4 times it actually showed a fully correct recommendation to catch the next green traffic-light. Sadly, it’s not part of my regular cycling routes.)

Available traffic-lights data in Hamburg

The geo-portal is an interactive map which can visualize some of the data made available by the city of Hamburg.

The screenshot below shows real-time traffic-light data of the intersection Fruchtallee / Doormannsweg:

Colors indicate the current signal of traffic-lights of different lanes.

Corresponding data is provided for about 800 of 1770 traffic-lights in Hamburg:

The data is made available on https://tld.iot.hamburg.de/v1.1 via the standard OGC SensorThings API which is described in detail e.g. here (usage guide).
This paper gives an excellent overview on the system architecture and underlying infrastructure.

The map below shows a part of the route I used to cycle to work as well as markers for all traffic-lights for which real-time data is currently available:

Overall, 13 out of 21 traffic-lights on the route provide real-time data (some are currently disabled due to ongoing construction work, and some may just be too small to be prioritized when planning corresponding IoT equipment. Nevertheless, having speed recommendations for almost two thirds of the encountered traffic-lights could already be a nice thing.)

To give an impression of provided traffic-light real-time data, the following bokeh-chart shows some historical data for the relevant primary signal (south -> north) of the above-mentioned intersection Fruchtallee / Doormannsweg (the most westward on the route, with ID 813_19, Datastream-ID 50850), on a typical morning (Friday, 19.05.2023, 6:00-8:00 UTC):

(> interactive chart)

The y-axis shows the duration of a phase over time (x-axis). Blue indicates the length of complete cycles (red-to-red), and is calculated from the sum of the durations of the different phases, drawn in their corresponding colors. Dashed lines indicate average values over the time-period (arithmetic mean).

On average, a complete cycle took about 90s and the traffic-light showed red about 63s, green for about 23s, and the transitions (amber / red-amber) only account for 3s and 1s respectively.
That makes sense, given that the analyzed traffic-light is pointing towards the lanes going from south to north, while the majority of (car-)traffic at that time can be expected to go from west to east towards the city center.

The observations show a number of larger deviations from the mean, with green phases being almost twice as long as normal (over 40s, at 06:30 and 07:20), followed by accordingly shorter red phases (only slightly longer than 40s).
These deviations may be explained by a dynamic traffic-lights control system, which may extend certain phase durations e.g. to account for bus signal requests (“bus priorization”), or when detecting extraordinarily high volumes of car traffic going into a specific direction.

According to a small sample of own observations, recent data is online available via the API in about 1-2s after a signal phase change.

In addition to the described real-time traffic-light data for car signals, the API also provides further information e.g. on car detectors, bus signal requests, pedestrian signals, and pedestrian signal requests.

Proof-of-Concept: A simple GLOSA web-app

Given quality and quantity of the available data, it does not seem impossible to imagine using it to forecast relevant future signal-phase changes and to calculate reasonable movement speeds from that information.

Below, a small application is sketched which allows a mobile web-client to request speed recommendations:

The main flow of information looks as follows:

1. a client requests a speed-advice, submitting its current location, movement speed, and trip-id
2. based on the trip-id the server can lookup the predefined route and determine the next upcoming traffic-light
3. the latest signal-phase change times can be queried from the traffic-lights-data API (encapsulated by a locally running proxy-service)
4. based on the received observations, a forecast of future phase changes can be created (also outsourced to a dedicated service)
5. the forecast is used to determine the movement speed necessary to catch the next reachable green phase and that information is returned to the client
6. whenever the client location updates, a new GLOSA-request is sent (see step 1., limited to at most 1 request per second)

For the proof-of-concept, the initial route configuration requires some manual effort for importing a defined route (in form of a .gpx file), and for identifying relevant traffic-lights (IDs) with help of the geo-portal.

The next screenshot shows a minimal web UI which provides information about the current location of the device, the next traffic-light (currently showing red), as well as the speed recomendation (“slow down, its not getting green anytime soon..”):

A couple of seconds later, the signal switched to green, and the current speed is sufficient to catch the phase:

GPS-independent testing

To facilitate manual testing without being reliant on actual GPS signals, the UI allows for the simulation of movements by dragging the user marker on the map:

For manual testing, the current speed and the accuracy are set to 20 km/h and 10m.

Used technologies

The client is built with React Material UI components, react-router, Leaflet for map display and interaction (via react-leaflet), and the browser Geolocation API (consent and secure context required).
Since the Geolocation API does not work in the background a wakelock can be requested.

At the server-side, nginx provides basic authentication, ssl-termination, serves static frontend files, and forwards /api-requests.
A “main backend service” is used for central calculations and the “orchestration” of auxiliary services, built with Spring Boot (which recently released the amazing support for compiling to a native-image, even though some stumbling blocks seem to remain).
Access to the latest traffic-light data is implemented using express running on node.js.
Future phase change times are forecasted based on the naive expectation of average-length phases, implemented in Python with pandas and exposed via flask.

For simple testing during development, the public localtunnel service can be used. This way, a server running at home can be accessed e.g. via mobile network (although they recently had to add some annoying password inquiry on their consent page).

The repository is available on github.

GPS accuracy and “route snapping”

One of the interesting aspects relates to the fact that the position the client detects is always somehow offset from the predefined route, be it due to measurement inaccuracies (commonly up to a couple of meters when using GPS), or inaccurate route definitions.

Consider the following part of a piece-wise linear route R1-R4 and the corresponding measurement L:

L indicates the current location sent by the client device, moving along the predefined route R1-R2-R3-R4. To give a speed recommendation, the next traffic-light (TL1 at R4) needs to be identified, and the correct distance along the given route must be calculated.
To determine the current location on the route, potential “snap locations” (SL1=R1, SL2, SL3) are evaluated by calculating the distance to the closest point on each segment (SL2 in the example above). The performed “route snapping” of the sent location is similar to (although simpler than) what some maps APIs offer, e.g. OSRMs match service.
To speed up the calculation, each segment is assumed to be linear, so that the JTS Topology Suite Java library’s DistanceOp can be used (with converted Mercator-coordinates as a fast simplification).

Performance considerations

As the sketched system relies on frequent client-server-communication (~1 request per client per second), performant request handling is of high importance.

For basic analysis of server-side request processing, the open telemetry project provides standard tooling which makes it easy to gather tracing information. This is especially interesting since a client request causes multiple requests to different backend services (database, traffic-lights data API, forecasting service).

Instrumentation support for automatic trace collection and forwarding to a tracing analysis backend are available for many popular languages / frameworks. For the main Java service, the Java instrumentation agent is used, which injects bytecode so that tracing information is automatically collected for popular frameworks / libraries without any code changes. Similar auto-instrumentation functionality is available for the node.js based traffic-lights data API proxy service, and for the Python-based forecast-service.

The following screenshot shows the processing of a single GLOSA-request:

The complete POST to the main backend (locationtracking-0.0.1-SNAPSHOT) takes 105ms to complete, starting with some database accesses (first ~10ms), followed by a long-running 69ms call to the traffic-lights data API proxy service (tld_api), and a 14ms call to the forecasting service (tld_forecast). As we can see, the main share of the overall request processing time is caused by the call to the external traffic-lights data API (68ms), out of which creating the TCP connection and completing the TLS handshake alone require 39ms.

To improve performance and efficiency of the external data API access, we can use a second feature of the sensorthings API. Apart from the normal HTTP-API to access historical data of traffic-light signal phase changes, a (websocket-based) MQTT-API allows to subscribe to new updates of specific traffic-lights. This way, we can extend the functionality of our API proxy service as follows:

1. an initial request of the last N signal phase change timestamps causes a regular HTTP request to the external API to fetch the required observations (which are then returned to the client)
2. furthermore, the received data is cached, and an MQTT-subscription for future updates is created
3. each received update is used to update the cache (evicting the oldest entry)
4. each following client-request for the same traffic-light data can now immediately be answered with locally cached data, without causing any external API request
5. after a configurable time without any client requests for data of a specific traffic-light (e.g. 60s), its subscription is canceled and its cache is cleared (a following client request is again causing an initial external request (1.))

The following screenshot shows the processing of a second client request for the same traffic-light data:

As we can see, the request to the tld_api is processed within 3ms, without any external call.

Assuming that data is requested over the last 1km when approaching a traffic-light, going with 20km/h (3 min travel time), and a frequency of 1 request per second, this means that 180 client requests only cause 1 initial external request and 179 subsequent requests are answered from the cache which is updated by push notifications received via MQTT. Consequently, the average request processing time for clients can be reduced by more than 50% (68/105ms), and the load for the external API server can be drastically reduced (even more when assuming multiple clients request the same traffic-light data).
As a further optimization, a server-push technology may be used to avoid polling and to allow for immediate client-notifications on the arrival of new data which causes changes to the last pushed recommendation.

As a disadvantage, the API proxy service now occupies a long-running MQTT-subscription / web-socket connection to the external API server (which should not cause much performance problems, as modern web-servers / MQTT brokers can be expected to easily handle large numbers of concurrent of connections, even on commodity hardware). Additionally, a subscription is kept alive some time after the last client request occured (as we cannot know whether there will be more), so that a couple of updates can be expected to be received unnecessarily - however, that disadvantage should be more than counterbalanced by the amount of saved external requests.

Apart from the external request handling, further optimization possibilities include database access and calculations at the stateless main backend service. Instead of loading the predefined route from the database on each client request, corresponding accesses can also be cached, e.g. using Springs built-in caching capabilities. As there is currently no way to modify predefined routes, their immutability avoids the need for cache invalidation. Even though there were no detailed measurements, it seems that a couple of milliseconds of request processing time may be saved this way.

To reduce the calculation effort of the “route snapping”, the index of the last reached route-segment is submitted back to the client as part of the response, so that this information can be included into a following request. This way, the number of checked route-segments can be limited, based on the assumption that a client is not expected to move backwards on a route. Additionally, checking further upcoming route-segments can be aborted once the closest distance to the current location stops to decrease (or does increase over a threshold), so that checking all upcoming route-segments can be avoided (at the expense of possibly missing an even closer segment). Other optimizations may include reducing the number of segments on a route by combining adjacent segments with (almost) identical direction into a single segment.
While the described calculation optimizations may be relevant for longer routes with more segments (i.a. GPS waypoints), there was no measurable difference on the test route when reducing from 522 to 310 waypoints.

Experimental evaluation

To be able to quantify the effects of using the GLOSA app, the geolocation and speed data sent with each request is persisted for analysis.

The following chart shows the speed profile of a bike ride to work without following the GLOSA speed recommendations:

The trip had a duration of 00:32:16, and during this time 1,937 samples were collected (almost exactly one GPS update per second).
The theoretical distance measured on a map is 10.7 km, the total sum of the distances between all sampled locations is 11.1 km (slightly longer, probably due to GPS inaccuracies and seemingly “less straight” movements).
Given the duration and (theoretical) distance of the trip, an average speed of 5.53 m/s (19.90 km/h) can be calculated. This is close to the average speed calculated from the 1,937 speed samples (5.38 m/s), so the measurements seem to be quite accurate.
The maximum measured speed was 9.30 m/s (33.47 km/h), the share of samples with a measured speed below 1 km/h (likely stops) was 10.8%.

The next screenshot shows a trip when trying to follow the GLOSA recommendations:

To my disappointment, the share of speed measurements below 1 km/h is still 9.2% - a difference of 1.6 percentage points which may not be interpreted to be of any significance.
Obviously, any serious evaluation should be based on a larger number of trips. Furthermore, the experiment itself was probably also negatively influenced by existing personal experience on the common phase lengths. (Infact, some traffic lights along the route are already quite well-known, so that no external GLOSA recommendation is needed. Any serious evaluation should of course be performed on routes which are unknown to the testers.)

While I did not analyse any more specific data, it seems that the effect of using the GLOSA app is negatively influenced by:

• bad traffic light data for some traffic lights (the “real-time” data is so old that no meaningful predications can be made)
• badly correlated traffic lights (no matter how slow/quick you try to go, between some traffic lights stops are almost inevitable; the phases seem to be optimized for a limited number of cars driving 50 km/h)
• “Bedarfsampeln” (one traffic light on the route only switches to green upon manual triggering, thus also causing unavoidable stops)
• bad predictions (forecasts purely based on historical data tend to be rather inaccurate, given the dynamic traffic control with adaptive phase lengths)

Especially the last point causes a particularly frustrating experience in case a green phase could actually have easily been catched when not following a recommendation.

Conclusion

While digging into the topic, it was quite surprising how and what data is made publically available by the Urban Data Platform of Hamburg. It’s good to see the municipality investing into the digitalization of the traffic-light infrastructure, and also into an IT infrastructure which may make large-scale applications possible.

Concerning GLOSA applications, the practical benefit really depends on the decisive details - predicting the correct phase change times down to the second is a crucial aspect which seems quite hard to get right in this dynamic environment. Nevertheless, I am looking forward to the PrioBike app and may give it a try after its publically released (hopefully open-source).

Potential extensions and improvements

Based on the sketched proof-of-concept, numerous improvements are imaginable.

Starting with the annoying manual setup step to define routes and to identify relevant traffic-light signal head IDs, the identification step could be automated as shown by the PrioBike-app developers from the TU Dresden in their selector implementation.

A main challenge relates to the prediction of future signal phase changes, which could e.g. be extended by incorporating already available data such as bus signal requests.

Another potential for improvement relates to recommendation algorithms, which could e.g. include information on current/past riding speeds, as well as multiple upcoming traffic-lights (maybe the traffic-light right after the next one will be missed anyway).

Finally - to generally improve the cycling experience and the quality of life in the city as a whole - other measures like optimizing traffic-light controls towards an average speed of 20-25 km/h, as well as simply extending the cycling lanes can obviously be expected to have a much greater impact than any GLOSA application.