Valuation of Wind Forecasts in Energy Marketing
Craig Collier, DNV GL Evan Caron, Trailstone Energy Group Mark Smith, DNV GL
1
DNV GL © 2013
SAFER, SMARTER, GREENER
Agenda
• Traditional evaluation metrics • A “market-value” metric • Brief demonstration
2
DNV GL © 2013
How do we measure perfomance? Mean Absolute Percent Error (MAPE)
where
𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 =
1 𝑁𝑁 ∑ 𝑁𝑁 𝑖𝑖=1
(𝐹𝐹𝑖𝑖 − 𝐴𝐴𝑖𝑖 ) x 100
𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 =
1 𝑁𝑁 ∑𝑖𝑖=1 𝑁𝑁
𝐹𝐹𝑖𝑖 − 𝐴𝐴𝑖𝑖
𝐹𝐹𝑖𝑖 ≝ 𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 𝑎𝑎𝑎𝑎 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑖𝑖
𝐴𝐴𝑖𝑖 ≝ "𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎 𝑎𝑎𝑎𝑎 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑖𝑖
Root Mean Square Error (RMSE)
error for increasing forecast horizon.
Mean Absolute Error
shows shows increasing
x 100
18%
MAPE(%(% of Capacity) of wind farm capacity)
Typically, SoA forecast
20%
2
𝑁𝑁 ≝ 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠
16% 14% 12% 10% 8% 6% 4% 2% 0%
1
3
DNV GL © 2013
6
12
18
24
30
Forecast Horizon (Hours) Forecast Horizon (hours)
36
42
48
Bulk measures are intuitive, and often informative
Actual
𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 = 8% 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 = 10%
Forecast 1 Daily MAPE Daily RMSE 4
DNV GL © 2013
Forecast 1
Forecast 2
𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 = 10% 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 = 13%
Forecast 2 Daily MAPE Daily RMSE
All forecasts are not created equal! Wind forecast errors from 7 providers
Forecast 1 5
DNV GL © 2013
2
3
4
5
6
7
Rank forecasts based on traditional metrics MAPE, RMSE
Error Distribution
12
13
13
14
14
7
14
16
15
16
17
6
15
15
14
15
14
5
15
15
14
14
15
4
14
15
14
14
16
3
16
13
16
14
14
2
12
13
13
12
10
1
a large contribution
•
MAPE, RMSE scores would indicate Solutions 1 & 7 are are highest quality forecasts.
•
Error distributions strongly favor Forecast 1 for least bias and all but 2 for least error diversity.
6
DNV GL © 2013
Wind is valuable if there is demand! Wind Forecasting
Load Forecasting
Actual 7
DNV GL © 2013
Forecast
Daily MAPE
Daily RMSE
What is the value?
Supply= Demand
8
DNV GL © 2013
What is the value?
9
DNV GL © 2013
DA not always a good predictor of RT Marginal Price = sum of marginal fuel cost, variable maintenance cost, transmission constraints, transmission losses Jan
Jan - May
Feb
Distributions are not the same
RT Price 10
DNV GL © 2013
DA Price
•
DA: Too few extremes, low & high
•
DA-RT differences market opportunities
DA-RT excursions closely track wind ramps Jan
Load Forecast Error(%) Wind Generation (Norm.)
11
•
In terms of value (revenue), wind forecast error is relatively more important when |DA-RT| large, and when load forecast error (|FLAL|) is small.
•
Need a metric that weights wind forecast error by a market value factor.
DNV GL © 2013
Applying a market value factor Original measure for MAPE given as
𝑆𝑆𝑖𝑖 = 𝐹𝐹𝑖𝑖 − 𝐴𝐴𝑖𝑖
Introduce a new measure for all times (i =1, ..., N) as follows
Si =
RTi − DAi 1 N
1 N ∑ j =1 RT j − DA j N ⋅ Fi − Ai FLi − ALi
∑
N j =1
FL j − AL j
FL − AL RTi − DAi Si = ⋅ Fi − Ai RT − DA FLi − ALi
S i = β ⋅ Fi − Ai 12
DNV GL © 2013
RT ≝ 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 ($)
DA ≝ 𝑑𝑑𝑑𝑑𝑑𝑑 − 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎 𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 ($)
FL ≝ 𝑑𝑑𝑑𝑑𝑑𝑑 − 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎 𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙 𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 (𝑘𝑘𝑘𝑘𝑘) AL ≝ 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎 𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙(𝑘𝑘𝑘𝑘𝑘)
Seasonal dependence of value? ERCOT
13
DNV GL © 2013
Market value weights coincide with DA-RT excursions Jan
Feb
Load Forecast Error
RT Price
DA Price DA Wind Forecast
14
DNV GL © 2013
Wind (Norm.)
*
Upper 10% MV
Market value weights coincide with DA-RT excursions Jan
Load Forecast Error
RT Price
DA Price DA Wind Forecast
15
DNV GL © 2013
Wind (Norm.)
* Upper 10% MV
Market value weights coincide with DA-RT excursions
Load Forecast Error
RT Price
DA Price DA Wind Forecast
16
DNV GL © 2013
Wind (Norm.)
* Upper 10% MV
Market value weights coincide with DA-RT excursions
Load Forecast Error
RT Price
DA Price DA Wind Forecast
17
DNV GL © 2013
Wind (Norm.)
* Upper 10% MV
The tale of two metrics
18
DNV GL © 2013
The tale of two metrics
19
DNV GL © 2013
Forecast Solution
Frequency of Minimum nMAPE
Frequency of Minimum nβ-MAPE
1
9
4
2
1
1
3
0
3
4
1
0
5
0
0
6
0
0
7
4
7
(C)Limatology
0
0
Simulation and Application Which forecast creates most value? Scenario: Buy / sell 100 MWh of energy in DA & RT markets for 15 months •
Sell energy in the DA market (go short) collect revenue at DA price
•
Buy energy in DA market (go long) sell in RT market
•
If forecast under-predicts sell at RT market for less (opportunity cost)
•
If forecast over-predicts buy difference at RT market price
Assumptions: •
Static bidding strategy
•
All market data available publicly from ERCOT: http://ercot.com/mktinfo/
•
Seven unique wind power forecasts from DNV GL
•
Unexpected and planned outages excluded
•
Prices derived from the West Hub node of ERCOT
20
DNV GL © 2013
Relative Forecast Value
• Application of Forecast 7 yielded 5% greater earnings ($135,000) for high DA-RT days, relative to “next best” earner (Forecast 2). • Forecast 7 yielded 15% greater earnings than the average earnings. • Earnings from a climatology forecast 35% lower than average earnings with a SOA forecast. 21
DNV GL © 2013
Key Takeaways
•
Bulk forecast quality metrics not always best for any individual application.
•
Maintain close dialogue with forecast users - optimize a solution around individual needs
•
including performance maintenance.
Energy market provides excellent example for innovating approaches to ensure a forecast is right when it matters.
22
DNV GL © 2013
Thank You
Craig Collier
[email protected] (858) 836-3370 x 118
www.dnvgl.com
SAFER, SMARTER, GREENER
23
DNV GL © 2013
