Applying data science and user-centric design to raise the IQ of smart thermostat demand response.
“Our grid planners and network operators don’t trust demand response, and they don’t count on it. Because to them it’s not real.”
Coming from a program manager overseeing a smart thermostat portfolio that regularly delivers over 70 MW of load shift, the remark caught me off guard. How could a program with a proven track record, backed by years of independently validated measurement and verification data, be dismissed as a superfluous side project rather than recognized as a core grid resource?
It turns out that this perspective isn’t unique to one utility, but rather reveals a more fundamental truth about thermostat-based demand response (DR) programs: historically their performance has been inherently variable, and that variability undercuts their economic value while inhibiting their operational relevance.
Yet smart thermostat programs are the most common and mature form of utility DR in the country. In an environment where utilities are experiencing sudden load growth and upward pressure on customer rates (and where building electrification and smart thermostat adoption are accelerating) thermostat DR can play a pivotal role as a source of clean, cost-effective capacity.
The key to unlocking this potential is to make thermostat DR programs every bit as predictable and reliable as other resources in the utility supply stack.
Following over two years of research, design, and real-world testing across dozens of events comprising over 95,000 thermostats across three utility partners, we have done just that. Today we’re thrilled to introduce Predictive Capacity Dispatch (PCD) as a core feature in Uplight’s Flex DERMS. PCD is a powerful tool that promises to enhance the value of thermostat DR for utilities, while also improving the customer experience.
The Challenges With Thermostats
Since the inception of thermostat DR, its load shift potential has been influenced by four primary factors. First, outdoor temperature impacts the actual load of underlying HVAC equipment. Second, and relatedly, each thermostat OEM has unique control algorithms that shape HVAC behavior before, during, and after an event. Third, the number, frequency, and duration of events impact the rate at which customers opt out of events by overriding thermostat controls. And finally, thermostat DR programs have conventionally applied a “one size fits all” dispatch strategy, activating all enrolled devices simultaneously for the entire event window.
Combined, these factors typically produce a demand curve that initially spikes just before an event starts as HVAC systems pre-condition, then rapidly falls to a maximum performance level at the event start time as all systems curtail at once. Over the course of an event performance steadily “decays” back towards the baseline as customers either proactively opt out or their thermostats exceed the event-mode setpoint thresholds. Finally, there is a “snap-back” effect” as participating systems quickly return back to default set points once the event concludes.
While the shape of this curve is well understood, accurately predicting the absolute magnitude of each phase is difficult. Heuristics can be used to reasonably estimate the initial load shift of a given event, but modeling how it diminishes over an event is more art than science. Based on analysis of the thirty-plus programs that Uplight manages, we observe that performance can decline anywhere between 20-50% for every successive hour during an event due to natural thermal gradients within homes and opt-outs by customers who experience discomfort and event fatigue. Over the course of a typical four-hour event, the worst (last) hour can yield as little as 10% of the load shift as the best (first) hour.
This variability becomes problematic in the mission-critical setting of the utility control room, where engineers need to be highly confident a given resource will behave exactly as they expect it to when called upon. Such confidence is only built through a combination of robust statistical modeling that precisely predicts (and schedules) the availability of a resource in advance and firm delivery against those commitments. Historically these capabilities have been absent from thermostat DR, creating a fatal flaw for relying on thermostat DR in the context of network operations and planning.
But thanks to innovative thinking and collaboration with several of our utility partners, over the last two years Uplight has developed innovative capabilities that are turning thermostat DR into a firm, predictable, and reliable resource–all while enhancing the customer experience.
Moving Towards Data (Science)-Driven Dispatch
In spring of 2023 one of our utility clients requested a solution to improve how program benefits were being measured and claimed, a major pain point for them. The utility’s trading team aimed to leverage thermostat DR to avoid transmission capacity costs during the summer season, as doing so would yield significant benefits for their overall resource adequacy obligations while improving program cost-effectiveness. However, for their thermostat program to participate in the wholesale market, the traders needed the ability to precisely forecast available DR capacity over a four-hour period on a day-ahead basis, and then the capability to deliver a consistent load curve against the forecast commitment throughout the entire event window. At that time, their only option was to heavily discount the capacity of their portfolio – comprising tens of thousands of thermostats – by as much as 75% to hedge against performance degradation throughout events.
Thus began a multiyear collaboration to radically improve the predictability and reliability of thermostat DR by applying elements from data science and behavioral science into a novel dispatch strategy.
The first step was to model how the interdependencies between weather, customer behavior, and OEM-specific controls determine event performance at an individual and aggregate level. Unsurprisingly, the relationship between weather and thermostat controls is relatively straightforward. As exterior temperature is more extreme (i.e. hotter on summer afternoons or colder on winter mornings), the magnitude of pre-conditioning and snapback effects as well as the magnitude of initial load shift and subsequent decay increases.
Layering in how those patterns interact with customer opt-out rates is significantly more complex. Though smart thermostat DR is typically described as “direct load control”, the reality is that utilities are ultimately sharing control with customers, who always have the option to override the control settings of their thermostat. Customers who choose to opt out of events do so overwhelmingly because of discomfort, so predicting opt-out patterns requires understanding how a typical customer’s HVAC will operate and home will perform under a default event scenario, as well as behavioral patterns (e.g. how opt-out rates change as a function of event length and frequency; how messaging strategies impact opt-outs, etc.).
Ultimately it’s impossible to predict how any particular customer will respond in any given event. But by analyzing historical event data and applying insights from Uplight’s extensive behavioral programs, we were able to create a predictive model representing individual load shift profiles for all enrolled customers, segmented by thermostat brand. This yielded a distribution curve of the expected load shape for each OEM that factored in each event’s timing and expected weather conditions.
The final step was to translate these modeled results into a different dispatch paradigm. During an PCD event we created groups within the program, with each group strategically allocated a specific percentage of each OEM based on their unique attributes, and assigned to a specific portion of the overall event window. Rather than following a conventional paradigm of activating all enrolled devices concurrently at the start of the event, we sequenced the groups in multiple dispatch waves. Each group was activated for only a portion of the overall event, leveraging the anticorrelation between waves (e.g. as performance degraded in the first group, the second group began delivering maximum performance) to deliver an aggregated result that our utility partners had long been seeking: a flat, consistent curve that persisted across the entire event while minimizing pre- and post-event “snapback”. Equally important, we demonstrated the ability to predict this load shift on an hourly basis a full day before the event with an error of 2% between the forecast and the actual measured performance post-hoc.
Predictive Capacity Dispatch: A Powerful Tool In Your Flexibility Toolbox
After experimenting with PCD in a single program in summer 2023, we refined our approach and expanded trials across three utility programs comprising over 95,000 thermostats throughout Summer 2024. This year we were able to exceed all of the internal goals that we set out to achieve, most significantly proving that smart thermostat DR can deliver predictable and trustworthy capacity, enabling deeper participation in operational scheduling & wholesale markets.
By achieving day-ahead forecast accuracy of 98% for each hour in an PCD event, while reducing the hour-to-hour variance in event performance by 176%, we were able to provide utility grid operators, traders, and planners with the confidence that thermostat DR can be just as reliable as other resources in their supply stack. This has major implications for the way that smart thermostat programs are utilized and evaluated, especially for utilities who incur capacity charges in wholesale markets. To take just one example, for utilities located within PJM where capacity prices are skyrocketing (resulting in upward pressure on customer rates) PCD represents the possibility to avoid millions of dollars in capacity costs, dramatically improving the cost-effectiveness of their programs.
A secondary, but equally relevant benefit was the impact that PCD had on customer opt-out metrics. Because each individual customer only participates for a portion of a PCD event (typically 1-2 hours, rather than all four), we were able to minimize thermal changes in their homes. This resulted in less discomfort, and a 6% reduction in opt-outs compared to traditional event strategies. While post-season customer surveys and user research initiatives are ongoing, the early feedback indicates that PCD is also a way to improve overall customer satisfaction and program retention, even while running events more frequently.
While PCD has clear advantages, it is not necessarily the best solution for every utility in every situation. PCD achieves consistent performance through the creative way that customers are grouped and sequenced. This approach intentionally and necessarily reserves some customers from the initial dispatch wave, reducing first-hour performance (and accordingly, reduces the maximum load-shift potential in that hour). PCD compensates for this by improving worst-hour performance by over 46%, but the net result is a reduction of approximately 20% in terms of MWh shifted throughout the course of an event.
In this trial, the conventional dispatch yielded approximately 17.5 MWh, with 75% performance degradation between first and last hour.
The PCD dispatch yielded approximately 15 MWh with les than 5% variance between first and last hour.
For utilities that value the ability to precisely and accurately forecast thermostat DR in advance, and subsequently deliver dependable capacity across all hours of an event, PCD is a game changer that enables them to use their portfolio in new ways, especially in the context of wholesale markets and/or Virtual Power Plants. However for utilities who want to maximize load shift in one specific hour, a traditional approach remains the best bet. With Uplight Flex DERMS, both are now options on the menu.
