Watching 50% of Humanity Go Online Over a Single Day is Pretty Mesmerizing
Here at the Monash University IP Observatory, we handle hundreds of millions of observations of internet connectivity and quality every day. While most of our time is spent providing communities around the world with near real-time monitoring of the internet during major natural disasters or periods of intense concern for political and online freedoms, every so often, we allow ourselves a chance to push the chair back, head over to the big monitor on the wall, and toggle the zoom knobs on the dash to planetary mode.
I guess classical astronomers did the same thing every once in a while. For them, the task was actually pretty trivial, but no less awe inspiring. Just the simple act of lifting their eye from the view-finder, descending the ladder, and stepping out into the cool air of the mountain top was all it took for them to see their beloved stars in universe mode. What a job.
Today, at the IP Observatory, we had one of those days. And we couldn’t help but share.
In this piece, we present visualisations that cycle through a single pseudo-24 hour day created by collapsing our entire set of global observations during March 2019 into average readings for each hour of the day at over 10,000 sub-national regional locations world-wide.
In all, that comes to making sense of over 1.3 billion observations.
To accomplish this, we combined our own powerful sensing and analysis technology with an emerging and totally wonderful geo-spatial data visualisation tool, fittingly named, kepler.gl. Kepler.gl comes from the Uber Visualization team, and reduced our task from days to hours. (Thank you Uber.)
First, let’s take a look at the really big picture.
Our observational methodology uses the most basic internet messaging protocol that is widely used billions of times a day to establish routes for your email, tweet, or share. After developing a carefully selected set of internet addresses (IPs) to measure, we periodically send them one of these tiny messages, essentially asking, ‘are on you online?’.
Individually, these measurements are what we call in the trade ‘low dimensional’ data. We learn sparingly little about each individual internet connection. However, when scaled up over an entire region, country, or globe, clear patterns emerge.
In the visualisation above, colours have been adjusted so that the lowest readings for the number of unique connected addresses (‘connectivity’) at a location over a 24 hour cycle are dark red/purple, whereas the highest readings are bright yellow.
What’s sort of mesmerizing is the way that you can see the diurnal cycle of night and day sweep over the face of the map.
To be clear, this isn’t us overlaying the day/night cycle on the map, the periodic rise and fall of colours is generated directly from the variations of internet connectivity in our measurements.
March is a particularly interesting month for this kind of visualisation as it is one of only two months in the year when the sunrise frontier runs almost exactly North-South, parallel to lines of longitude over the Earth’s surface.
This is why Japan and Melbourne, separated by over 8,000km and lying at almost identical but opposite distances from the Equator, appear in connectivity synchrony, as they sit within a few degrees of longitude of each other.
Our team explored this insight in depth in an earlier work [1], showing that as you’d expect, internet activity observations can be used to make accurate predictions of the amount of sleep people are getting (or not getting, in some places!) in their real, offline lives.
I guess the take home point is that everyone, everywhere, eventually succumbs to their mortality and goes to bed.
Next, let’s zoom in a little and see what this picture looks like at a continental scale.
Having zoomed in, what’s now immediately obvious, is the differences in density of the points on the map. Belgium, for instance, is notably off the charts compared to, say, France.
The reason for this is to do with how we measure the internet in the first place. We typically focus on administrative regions at level 2 (ADM2), which are political boundaries defined by each country. For a country so small, Belgium actually has 43 arrondissements, whereas France, many times as large, only has 100 equivalent departments. Hence, Belgium’s data is cut into many more divisions per square kilometre than France.
For the visualisation, we ensure that the bar height represents only the varying component of the day’s connectivity, and takes into account all connections across the entire ADM2 region of measurement. In other words, comparing heights across bars is an ‘apples-to-apples’ comparison.
At this scale, the diurnal cycle is even more pronounced. In fact, what you can see in Europe is something we see quite often in major economies around the world. Obviously, there is a low period of connectivity overnight, typically coinciding with around 4am. Then, at around 6am, there is a strong build into the middle of the day. These two phenomena account for the dark / light cycle you see first.
However, if stare at the visualisation for a while, you’ll see that there is a second ‘bump’ in intensity, after the first rise. This is the evening peak — it comes from anywhere between 6pm to 10pm, but for most locations, 9pm seems to be common. What we think is happening, based on linking our data to granular public transit data, is that people are returning from work, and firing up their home networks.
When at work, you contribute to connectivity, but usually only marginally so given that many users will be connected to a single workplace internet connection. Whereas at home, far fewer users are connected to each internet connection, generating many more online measurements than during the day. Interestingly, intra-diurnal electricity consumption patterns show the same pattern.
Notably, some cities seem immune to this cycle. Take a closer look towards the South East of the United Kingdom.
There’s one marker which appears online 24/7. That’s London.
If you are like us, the remarkable feature of the East Asian visualisation is the immaculately coordinated and thickly described rise and fall of connectivity in India.
The other striking thing is what you don’t see in this visualisation.
Yes, connectivity darkness extends over much of the Himalayan Plateau, lining up with the very low population density there.
But if you look closely to the West of Japan, and North of South Korea, the lights seem to go out abruptly. That’s North Korea.
Our measurements suggest that North Korea has just three ADM2 regions with more than 100 unique IPs during any given hour of the day. And in total, these amount to just a few hundred active IPs.
Contrast this to South Korea, where we see over 200 highly active ADM2 regions, with, at minimum, almost 5000 times more active internet connections. (This number is no doubt a vast underestimate due to our sampling methodology in densely connected regions.)
It’s worth noting that by population, the South has only twice as many people as the North.
No maths required to absorb the connectivity darkness which prevails for the people of the North.
But then again, creating access to a basic human right like the internet does not seem a high priority for present leadership of North Korea.
As with Europe, the evening bump is unmistakable, arriving like a fiery pulse just as the day’s connectivity seems to have worn itself out.
In the US, ADM2 corresponds to counties, closely mapping the population density of the citizens, which explains the huge number of distinct observations possible in the Eastern half of the country.
What’s interesting is that even the notably tech-obsessed San Francisco Bay area drops offline of an evening and gets some shut-eye.
For all the insights of remote, global internet activity measurement, there is one more insight that arises not from the sparks of light, but from depths of the dark.
Take another look at the global visualisation at the top of the piece.
No prizes for picking the odd continent out.
Yes. Africa.
As we prepared this piece, one of us commented immediately on how similar the global connectivity map looked to a visualization we managed to create a few years ago based on data from 2012. We were looking at Africa.
To be sure, there has been enormous change in Africa, most notably in mobile, and our methodology under-represents this important channel. However, the uptake of fixed broadband technology in Africa, especially in sub-Saharan Africa has been glacially slow. Typical penetration rates sit below 1%, compared with around 80% for advanced economies.
To give an example, when we recently provided impact mapping support after the arrival of Hurricane Idai at the Sofala coastline in Mozambique, for our analysis we had to fall back on just three viable locations near the site of landfall and two of which — Maputo (capital of Mozambique) and Harare (capital of Zimbabwe), were hundreds of kilometres away.
The most heavily inundated area, South of the site of landfall, had effectively no internet infrastructure from which those affected could harness information and updates, or get pictures, video, and text out to alert local and international authorities to their needs.
Of course, many players are now trying to change this situation, from the big tech leaders to international governments looking to flex their economic and political muscle in Africa.
Regardless, it’s our hope that the next time we look outside to gaze at the lights of the internet Africa begins to carve out its own rich constellation.
References
[1] K Ackermann, SD Angus, PA Raschky (2017), ‘The Internet as Quantitative Social Science Platform: Insights from a Trillion Observations’, https://arxiv.org/abs/1701.05632
The mission of the Monash University IP Observatory — ‘internet insights for social good’ — is to monitor the availability and quality of the Internet during critical events such as elections, natural disasters or conflict to provide. The observatory was founded by Klaus Ackermann, lecturer in Econometrics and Business Statistics, and Simon Angus, and Paul Raschky, Associate Professors in Economics. The observatory is a project of SoDa Laboratories at the Monash Business School, and tweets @IP_Observatory.
Watching 50% of Humanity Go Online Over a Single Day is Pretty Mesmerizing
Research & References of Watching 50% of Humanity Go Online Over a Single Day is Pretty Mesmerizing|A&C Accounting And Tax Services
Source
0 Comments