time-series

What are some compact algorithms for generating interesting time series data?

The question sort of says it all. Whether it's for code testing purposes, or you're modeling a real-world process, or you're trying to impress a loved one, what are some algorithms that folks use to generate interesting time series data? Are there any good resources out there with a consolidated list? No constraints on values (except pl...

What's a good way to store raster data?

I have a variety of time-series data stored on a more-or-less georeferenced grid, e.g. one value per 0.2 degrees of latitude and longitude. Currently the data are stored in text files, so at day-of-year 251 you might see: 251 12.76 12.55 12.55 12.34 [etc., 200 more values...] 13.02 12.95 12.70 12.40 [etc., 200 more values...] [etc., ...

"Join" of time series

I am designing a simple internal framework for handling time series data. Given that LINQ is my current toy hammer, I want to hit everything with it. I want to implement methods in class TimeSeries (Select(), Where() and so on) so that I can use LINQ syntax to handle time series data Some things are straight forward, e.g. (from x in A ...

Linear regression for time series with Gnuplot

I am a big fan of Gnuplot and I used it all along my studies for various projects. Lately I wanted to use Gnuplot to chart some time series like weight loss, exercising results, gas consumptions etc. Therefore I scale the x-axis like set xdata time set timefmt "%d.%m %Y" set format x "%d.%m" Now I want to use the fit-function to gi...

rrd tool alternative for high volume

I am interested in knowing if there is any alternative to rrdtool for logging time series data. I am looking at something that can scale for a large number of devices to monitor. From what I read on this subject, rrdtool is I/O bound when you hit it with large amounts of data. Since I envision this to scale to a very large number of dev...

Improving the accuracy of Microsoft Time Series Algorithm

Situation is that we have branches in every city, selling food. I feed the time series algorithm with the actual date, as the key time, and total sales of that day, as the input and predict. Predictions are not bad. But, I would like to know if i can improve the predictions by for example feeding with the number of branches(a new bran...

Java library to do time series analysis

I need to do some analysis of an arbitrary amount of time series in Java. Among others i need to be able to use Linear regression, various smoothing techniques, filtering, etc. I'm not very keen of writing all this from scratch so, do you know of any good Java libraries for these kind of analysis? Edit: R- Seems like an good choice....

Time-series and correlation strategies

I have various time-series I'd like to correlate and present as either a csv-file or in-memory datatable (.NET). These time-series are arrays of time-value-pairs (actually these are objects containing more than just time and value). The time-series may span across different overlapping periods, and some might even have holes (missing val...

Probability time series, observed data probabilities (deja vu)

okay folks...thanks for looking at this question. I remember doing the following below in college however I forgotten the exact solution. Any takers to steer in the right direction. I have a time series of data (we'll use three) of N. The data series is sequential in order of time (e.g. obsOne[1] occurred along with obsTwo[1] and obs...

Reconstructing state from time series data events

For a particular project, we acquire data for a number of events and collect variables about those events at the same time. After the data has been collected, we perform a user-customizable analysis on said data to determine whatever it is that the user is interested in. The data is collected in a form similar to this: Timestamp E...

JFreechart Realtime Combined Plot - Rendering the previous value for a step subchart if no data point has been received

I have a combined plot in JFreechart consisting of 4 timeseries charts with a common time domain axis. The data for the subplots arrives at different rates. For example I may receive a price change event every couple of miliseconds and a position change every minute. I'm currently rendering these series with the XYStepRenderer to produc...

JFreechart Rendering a timeseries on a tick scale

I have a timeseries of data which i'd like to plot on a tick scale rather than a time scale. e.g. If the series contains points received at times: 10, 15, 30, 100. Instead of plotting these against a regular time series axis where the distance between the 30 and 100 point would be 70, i'd like the distance between each of these points ...

Cumulative sums, moving averages, and SQL "group by" equivalents in R

What's the most efficient way to create a moving average or rolling sum in R? How do you do the rolling function along with a "group by"? ...

Reshaping time series data from wide to tall format (for plotting)

I have a data frame containing multiple time series of returns, stored in columns. The first column contains dates, and subsequent columns are independent time series each with a name. The column headers are the variable names. ## I have a data frame like this t <- seq(as.Date('2009-01-01'),by='days',length=10) X <- rnorm(10,0,1) Y <- ...

dimensional and unit analysis in SQL database

Problem: A relational database (Postgres) storing timeseries data of various measurement values. Each measurement value can have a specific "measurement type" (e.g. temperature, dissolved oxygen, etc) and can have specific "measurement units" (e.g. Fahrenheit/Celsius/Kelvin, percent/milligrams per liter, etc). Question: Has anyone bui...

What is the best open source solution for storing time series data?

I am interested in monitoring some objects. I expect to get about 10000 data points every 15 minutes. (Maybe not at first, but this is the 'general ballpark'). I would also like to be able to get daily, weekly, monthly and yearly statistics. It is not critical to keep the data in the highest resolution (15 minutes) for more than two mont...

Changing annotations in time series plots in R

While I can change annotations with the generic plot command turning off axes and annotations and specifying them again using the axis command e.g. cars <- c(1, 3, 6, 4, 9) plot(cars, type="o", col="blue", ylim=range(0, cars), axes=FALSE, ann=FALSE) axis(1, at=1:5, lab=c("Mon","Tue","Wed","Thu","Fri")) I cant do it with time series o...

On the issue of automatic time series fitting using R

we have to fit about 2000 or odd time series every month, they have very idiosyncratic behavior in particular, some are arma/arima, some are ewma, some are arch/garch with or without seasonality and/or trend (only thing in common is the time series aspect). one can in theory build ensemble model with aic or bic criterion to choose the b...

which is the most efficient XML Parser for C++ ?

I need to write an application that fetches element name value (time-series data) pair from any xml source, be it file, web server, any other server. the application would consume the XML and take out values of interest, it has to be very very fast (lets say 50000 events/seconds or more) also the XML document size would be huge and frequ...

lag in apply statement doesn't work in R

I'm trying to "apply" a function that does "lag"s on zoo objects in R. The function works correctly if I pass a single zoo vector - it applys the lag and everything works. However, if I "apply( data, 1, function )" then the lag doesn't work. There is no error, just the equivalent of a zero lag. This is also the case with a simple "ap...