We had a busy 2023, engaging the industry in energy and sustainability engineering discussions. Through our LESS Accord projects, we heard from engineers across the industry about where they thought the 'low hanging fruit' for focus on energy optimisation was.
As you may know, the outcome of that has been our four LESS Accord projects, which our members' working groups use as testing grounds and touch points for engagement with the broader industry. With GoS's systemic engineering view, we are keen to stay connected.
That engagement feeds into our working groups, but we are well aware that the activity in our working groups is private, so it has been hard to talk about the efforts in those groups in detail.
In 2024, we will have our first data to base comments on.The engineering work has been on getting actionable energy data in (near) real-time to provide immediate, measurable feedback to experimental changes to the streaming workflows we subsequently want to explore.
It takes a lot of work. :)
Great efforts by academia and other industry bodies based on lab experiments and mathematical modelling have produced data that regularly makes the headlines. We’re genuinely excited to see how our forthcoming real-world energy data will help the industry qualify the accuracy of existing measurements and improve future energy models.
We have four key areas to measure. Referring to our classic live streaming system overview:
As an industry, we can most readily evaluate the energy demands of the top row of processes - informally, 'application layer' or 'layer 4' processes in some network layer models. These streaming-focussed, high-compute and scaled-up 'applications' include (1) video coding and (2) content origination, (3) 'pure-play' CDN services, Operator CDN services at various depths, and (4) the Consumer Premises Equipment (CPE) decoding and stream access.
Up to late autumn, we had been spending a lot of time making sure we got the industry to think about the problem space, to help us identify what engineering work would make an impact, and lay the groundwork for some real-world measurement.
We have established ways to gather energy data into our testing from data centre environments and have tested these with large sporting event measurements. The technology works, but the data points are still being cross-referenced and, to some extent, 'calibrated'.
However, it became clear that getting data flowing from the consumer electronics end of the workflow would require a different approach.
The story is long and detailed, and we will organise a webinar to discuss those details (please watch our linked-in feed for that!).
A reasonably obvious early discovery was that there was no built-in system to generate energy reports from devices at the end of the network. Even laptop operating systems make it challenging to interrogate the energy of a system. Yes, installed applications are an option to some extent, although even these typically use estimation models (representing percentages rather than Watts, etc.).
So, after clarifying that the device industry could not report energy data automatically, we turned to the metering models used for early academic studies and industry analysis.
Two of our affiliates, The Digital Television Group (DTG) and Fraunhofer FOKUS, helped us understand their measurement models.
They both use these in labs to help the TV industry look at a wide variety of aspects of the TV device. Their bench models vary, but all the measurement systems were either relatively expensive or the software approach was limiting them to lab-based readings.
Since GoS is interested in the systemic energy impact of CPE rather than the performance of particular vendors, we want to scale our testing out to reflect the measurement of all the device types in real-world use. As such, our metering system needed to be small, cheap, and simple to set up and have a backend system that would programmatically allow us to capture energy information with regularity.
We identified a simple, smart plug to set up where the 'opting-in' to our testing is quick yet explicit, addressing potential privacy concerns. The plug provider has helped us interrogate those units in our test at a 10s frequency. (Please contact us to know more about the specific models - we want those who start experimenting with them to be aware of our forthcoming tests and hopefully participate!)
All our data from technical tests is collected in a central aggregation 'hub' overseen by members and technical partners. We are developing an interface for our members and academia to query that data for deeper insights into the relationship between energy and streaming.
To bring the technologies together, we have a series of hackathons underway. Once we secure funding to scale with research projects and grants, these will grow into consumer-scale tests and move from the current scale of dozens of testers to hundreds and then thousands.
The initial two hackathons have already been run. The first was to test the measurement systems and identify some 'signal markers' that would produce some 'energy signatures' to help correlate our testing. The second was to attempt to use those signal markers to define areas of known testing and to get a glimpse into the types of responses we may see to a limited number of early tests.
The initial test group has been only our Working Group 8 members, so we have tested about a dozen sites so far.
The publication released today on our www.greeningofstreaming.org/publications page is the first review of these hackathons and provides the first glimpse of the data we are attempting to gather now. While the report itself is only an initial foray out of our Working Groups, it is a culmination of a lot of discussion and effort by the members over an extended period.
Another exciting aspect for the group is that this model will enable us to add measurement to our WG6 encoder systems and potentially to WG4 caching systems. This discussion is in the early days, but the sense is that we can
build an entire 'virtual lab' in a real-world setting and start to see if the data we are getting from the field correlates well with the historical data that has, until our experiment, been limited to lab tests and extrapolation (with commensurate uncertainties).
The next hackathon will align more detailed tests with more participants and capture energy data relating to the packaging and encoding of the tested streams. Logically, from there, we will bring in a measured caching system. At that point, we will have a relatively simple, scalable measurement system to model some at-scale audiences. Through programmes of test video signals, we will reach our goal of ascertaining what settings and architecture in a streaming environment REALLY affect the global systems' energy demands.
Hackathons take weeks to months to set up, run and report on, so keep checking back on our blog every month or follow our LinkedIn page for notifications when new publications and hackathon updates come out.
The review can be found here : www.greeningofstreaming.org/publications Press Release can be found here :
To learn more about joining, please check https://www.greeningofstreaming.org/membership