Plotting many lines as a heatmap

Question

I have a large number (~1000) of files from a data logger that I am trying to process.

If I wanted to plot the trend from a single one of these log files I could do it using

plot(timevalues,datavalues)

I would like to be able to view all of these lines at same time in a similar way to how an oscilloscope has a "persistant" mode.

I can probably cobble together something that uses histograms but am hoping there is pre-existing or more elegant solution to this problem.

Hypercatalectic

you basically want to have 1000 lines in one plot? – Impressment

I want to use the data from 1000 lines to produce one plot, like in the oscilloscope plot, it shows many updates at the same time, by the use of the colours it shows where more of these wave-forms are "stacked" (i.e. the red parts) and where there is the occasional difference or glitch in waveforms (i.e. the blue parts) – Hypercatalectic

will this be helpful? mathworks.com/help/comm/ref/commscope.eyediagram.html – Embryogeny

Answer 1

You can do exactly what you are suggesting yourself, i.e. plotting the heatmap of the signals.

Consider the following: I'll build a test signals (out of sine waves of different amplitude), then I'll plot the heatmap via hist3 and imagesc.

The idea is to build an auxiliary signal which is just the juxtaposition of all your time histories (both in x and y), then extract basic bivariate statistics out of that.

 % # Test signals
 xx = 0 : .01 : 2* pi;
 center = 1;
 eps_ = .2;
 amps = linspace(center - eps_ , center + eps_ , 100 );

 % # the auxiliary signal will be stored in the following variables
 yy = [];
 xx_f = [];

 for A = amps
   xx_f = [xx_f,xx];
   yy = [yy A*sin(xx)];
 end 

 % # final heat map
 colormap(hot)
 [N,C] = hist3([xx_f' yy'],[100 100]);
 imagesc(C{1},C{2},N')

You can use also jet colormap instead of hot colormap for readability. In the following the amplitude is gaussian instead of homogeneus.

Disregardful

Thanks this was just what I was looking for, I had looked at hist3 briefly but wasnt sure how to put the pieces together. Before I accept the answer, is there any way besides interpolation that can be used to add more data to the heat map when there is a rapid change in the signal, resulting in the line dissapearing. – Hypercatalectic

@Hugoagogo, I see your problem. I guess interpolation is a neat way to deal with the problem. Nonetheless, you must increase time resolution by interpolation at every time and not just where you have the problem, otherwise the bivariate stat will get ruined. – Disregardful

I found this code example mathworks.com.au/matlabcentral/answers/… that seems to work well enough for my purposes. Marking this answer as accepted – Hypercatalectic

Another important note is appending all the points in each file as in the example lead to memory issues as well as large slowdowns when the array was reallocated. I was able to get a large speed by precalculating bin centers, calculating the histogram for each file with those centers and adding the results together, this prevented ever increasing memory consumption. – Hypercatalectic

Answer 2

here's a "primitive" solution that is just using hist:

%# generate some fake data

x=-8:0.01:8;
y=10*sinc(x);
yy=bsxfun(@plus,y,0.1*randn(numel(x),1000)' );
yy(randi(1000,1,200),:)= 5-randi(10)+ circshift(yy(randi(1000,1,200),:),[1 randi(numel(x),1,200)]); 

%# get plot limit parameters

plot(x,yy)
yl=get(gca,'Ylim');
xl=get(gca,'Xlim');
close all;


%# set 2-d histogram ranges

ybins=100;
xbins=numel(x);
yrange=linspace(yl(1),yl(2),ybins);
xrange=linspace(xl(1),xl(2),xbins);

%# prealocate

m=zeros(numel(yrange),numel(xrange));

% build 2d hist
for n=1:numel(x)
    ind=hist(yy(:,n),yrange);
    m(:,n)=m(:,n)+ind(:);
end

imagesc(xrange,yrange,m)
set(gca,'Ydir','normal')

Embryogeny

This is the approach I was considering and works great, the solution @Disregardful provides is I think a nice more elegant solution so I have accepted that answer. Your sample data is better though. – Hypercatalectic

Answer 3

Why don't you normalize the data and then add all the lines together? You could then plot the heatmap from the single datafile.

Smothers