在分析能源需求和消耗数据时,我遇到了重新采样和插值时间序列趋势数据的问题。
数据集示例:
timestamp value kWh
------------------ ---------
12/19/2011 5:43:21 PM 79178
12/19/2011 5:58:21 PM 79179.88
12/19/2011 6:13:21 PM 79182.13
12/19/2011 6:28:21 PM 79183.88
12/19/2011 6:43:21 PM 79185.63基于这些观察,我希望一些聚合基于一段时间来累加值,并将该频率设置为时间单位。
如在中所示,每小时的时间间隔可以填补任何缺失数据的空白
timestamp value (approx)
------------------ ---------
12/19/2011 5:00:00 PM 79173
12/19/2011 6:00:00 PM 79179
12/19/2011 7:00:00 PM 79186对于线性算法,似乎我会取时间上的差异,并将该值与该因子相乘。
TimeSpan ts = current - previous;
Double factor = ts.TotalMinutes / period;值和时间戳可以基于该因子来计算。
有了这么多可用的信息,我不确定为什么很难找到最优雅的方法来解决这个问题。
也许首先,有没有可以推荐的开源分析库?
对程序化方法有什么建议吗?理想情况下使用C#,或者可能使用SQL?
或者,有没有类似的问题(有答案)可以指向我?
发布于 2011-12-30 05:15:32
通过使用内部用于表示DateTimes的时间标记,您可以获得尽可能最准确的值。由于这些时间刻度不会在午夜零点重新开始,因此在日边界上不会出现问题。
// Sample times and full hour
DateTime lastSampleTimeBeforeFullHour = new DateTime(2011, 12, 19, 17, 58, 21);
DateTime firstSampleTimeAfterFullHour = new DateTime(2011, 12, 19, 18, 13, 21);
DateTime fullHour = new DateTime(2011, 12, 19, 18, 00, 00);
// Times as ticks (most accurate time unit)
long t0 = lastSampleTimeBeforeFullHour.Ticks;
long t1 = firstSampleTimeAfterFullHour.Ticks;
long tf = fullHour.Ticks;
// Energy samples
double e0 = 79179.88; // kWh before full hour
double e1 = 79182.13; // kWh after full hour
double ef; // interpolated energy at full hour
ef = e0 + (tf - t0) * (e1 - e0) / (t1 - t0); // ==> 79180.1275 kWh公式的解释
在几何学中,相似三角形是形状相同但大小不同的三角形。上面的公式是基于这样一个事实,即对于类似三角形的相应边,一个三角形中任意两条边的比率是相同的。
如果你有一个三角形a,b,c和一个类似的三角形a,b,c,那么A : B = a : b。两个比率的相等称为比例。
我们可以将这个相称性规则应用于我们的问题:
(e1 – e0) / (t1 – t0) = (ef – e0) / (tf – t0)
--- large triangle -- --- small triangle --

发布于 2014-05-05 05:36:41
我已经编写了一个LINQ函数来对时间序列数据进行插值和归一化,以便可以对其进行聚合/合并。
重采样函数如下所示。我在代码项目中写了一篇关于这项技术的short article。
// The function is an extension method, so it must be defined in a static class.
public static class ResampleExt
{
// Resample an input time series and create a new time series between two
// particular dates sampled at a specified time interval.
public static IEnumerable<OutputDataT> Resample<InputValueT, OutputDataT>(
// Input time series to be resampled.
this IEnumerable<InputValueT> source,
// Start date of the new time series.
DateTime startDate,
// Date at which the new time series will have ended.
DateTime endDate,
// The time interval between samples.
TimeSpan resampleInterval,
// Function that selects a date/time value from an input data point.
Func<InputValueT, DateTime> dateSelector,
// Interpolation function that produces a new interpolated data point
// at a particular time between two input data points.
Func<DateTime, InputValueT, InputValueT, double, OutputDataT> interpolator
)
{
// ... argument checking omitted ...
//
// Manually enumerate the input time series...
// This is manual because the first data point must be treated specially.
//
var e = source.GetEnumerator();
if (e.MoveNext())
{
// Initialize working date to the start date, this variable will be used to
// walk forward in time towards the end date.
var workingDate = startDate;
// Extract the first data point from the input time series.
var firstDataPoint = e.Current;
// Extract the first data point's date using the date selector.
var firstDate = dateSelector(firstDataPoint);
// Loop forward in time until we reach either the date of the first
// data point or the end date, which ever comes first.
while (workingDate < endDate && workingDate <= firstDate)
{
// Until we reach the date of the first data point,
// use the interpolation function to generate an output
// data point from the first data point.
yield return interpolator(workingDate, firstDataPoint, firstDataPoint, 0);
// Walk forward in time by the specified time period.
workingDate += resampleInterval;
}
//
// Setup current data point... we will now loop over input data points and
// interpolate between the current and next data points.
//
var curDataPoint = firstDataPoint;
var curDate = firstDate;
//
// After we have reached the first data point, loop over remaining input data points until
// either the input data points have been exhausted or we have reached the end date.
//
while (workingDate < endDate && e.MoveNext())
{
// Extract the next data point from the input time series.
var nextDataPoint = e.Current;
// Extract the next data point's date using the data selector.
var nextDate = dateSelector(nextDataPoint);
// Calculate the time span between the dates of the current and next data points.
var timeSpan = nextDate - firstDate;
// Loop forward in time until wwe have moved beyond the date of the next data point.
while (workingDate <= endDate && workingDate < nextDate)
{
// The time span from the current date to the working date.
var curTimeSpan = workingDate - curDate;
// The time between the dates as a percentage (a 0-1 value).
var timePct = curTimeSpan.TotalSeconds / timeSpan.TotalSeconds;
// Interpolate an output data point at the particular time between
// the current and next data points.
yield return interpolator(workingDate, curDataPoint, nextDataPoint, timePct);
// Walk forward in time by the specified time period.
workingDate += resampleInterval;
}
// Swap the next data point into the current data point so we can move on and continue
// the interpolation with each subsqeuent data point assuming the role of
// 'next data point' in the next iteration of this loop.
curDataPoint = nextDataPoint;
curDate = nextDate;
}
// Finally loop forward in time until we reach the end date.
while (workingDate < endDate)
{
// Interpolate an output data point generated from the last data point.
yield return interpolator(workingDate, curDataPoint, curDataPoint, 1);
// Walk forward in time by the specified time period.
workingDate += resampleInterval;
}
}
}
}发布于 2011-12-30 04:33:02
Maby类似于:
SELECT DATE_FORMAT('%Y-%m-%d %H', timestamp) as day_hour, AVG(value) as aprox FROM table GROUP BY day_hour您使用的是什么数据库引擎?
https://stackoverflow.com/questions/8672998
复制相似问题