How to make a 4d plot with matplotlib using arbitrary data

Question

This question is related to this one.

What I would like to know is how to apply the suggested solution to a bunch of data (4 columns), e.g.:

0.1 0 0.1 2.0
0.1 0 1.1 -0.498121712998
0.1 0 2.1 -0.49973005075
0.1 0 3.1 -0.499916082038
0.1 0 4.1 -0.499963726586
0.1 1 0.1 -0.0181405895692
0.1 1 1.1 -0.490774988618
0.1 1 2.1 -0.498653742846
0.1 1 3.1 -0.499580747953
0.1 1 4.1 -0.499818696063
0.1 2 0.1 -0.0107079119572
0.1 2 1.1 -0.483641823093
0.1 2 2.1 -0.497582061233
0.1 2 3.1 -0.499245863438
0.1 2 4.1 -0.499673749657
0.1 3 0.1 -0.0075248589089
0.1 3 1.1 -0.476713038166
0.1 3 2.1 -0.49651497615
0.1 3 3.1 -0.498911427589
0.1 3 4.1 -0.499528887295
0.1 4 0.1 -0.00579180003048
0.1 4 1.1 -0.469979974092
0.1 4 2.1 -0.495452458086
0.1 4 3.1 -0.498577439505
0.1 4 4.1 -0.499384108904
1.1 0 0.1 302.0
1.1 0 1.1 -0.272727272727
1.1 0 2.1 -0.467336140806
1.1 0 3.1 -0.489845926622
1.1 0 4.1 -0.495610916847
1.1 1 0.1 -0.000154915998165
1.1 1 1.1 -0.148803329865
1.1 1 2.1 -0.375881358454
1.1 1 3.1 -0.453749548548
1.1 1 4.1 -0.478942841849
1.1 2 0.1 -9.03765566114e-05
1.1 2 1.1 -0.0972702806613
1.1 2 2.1 -0.314291859842
1.1 2 3.1 -0.422606253083
1.1 2 4.1 -0.463359353084
1.1 3 0.1 -6.31234088628e-05
1.1 3 1.1 -0.0720095219203
1.1 3 2.1 -0.270015786897
1.1 3 3.1 -0.395462300716
1.1 3 4.1 -0.44875793248
1.1 4 0.1 -4.84199181874e-05
1.1 4 1.1 -0.0571187054704
1.1 4 2.1 -0.236660992042
1.1 4 3.1 -0.371593983211
1.1 4 4.1 -0.4350485869
2.1 0 0.1 1102.0
2.1 0 1.1 0.328324567994
2.1 0 2.1 -0.380952380952
2.1 0 3.1 -0.462992178846
2.1 0 4.1 -0.48400342421
2.1 1 0.1 -4.25137933034e-05
2.1 1 1.1 -0.0513190921508
2.1 1 2.1 -0.224866151101
2.1 1 3.1 -0.363752470126
2.1 1 4.1 -0.430700436658
2.1 2 0.1 -2.48003822279e-05
2.1 2 1.1 -0.0310025255124
2.1 2 2.1 -0.158022037087
2.1 2 3.1 -0.29944612818
2.1 2 4.1 -0.387965424205
2.1 3 0.1 -1.73211484062e-05
2.1 3 1.1 -0.0220466245862
2.1 3 2.1 -0.12162780064
2.1 3 3.1 -0.254424041889
2.1 3 4.1 -0.35294082311
2.1 4 0.1 -1.32862131387e-05
2.1 4 1.1 -0.0170828002197
2.1 4 2.1 -0.0988138417802
2.1 4 3.1 -0.221154587294
2.1 4 4.1 -0.323713596671
3.1 0 0.1 2402.0
3.1 0 1.1 1.30503380917
3.1 0 2.1 -0.240578771191
3.1 0 3.1 -0.41935483871
3.1 0 4.1 -0.465141248676
3.1 1 0.1 -1.95102493785e-05
3.1 1 1.1 -0.0248114638773
3.1 1 2.1 -0.135153019304
3.1 1 3.1 -0.274125336409
3.1 1 4.1 -0.36965644171
3.1 2 0.1 -1.13811197906e-05
3.1 2 1.1 -0.0147116366819
3.1 2 2.1 -0.0872950700627
3.1 2 3.1 -0.202935925412
3.1 2 4.1 -0.306612285308
3.1 3 0.1 -7.94877050259e-06
3.1 3 1.1 -0.0103624783432
3.1 3 2.1 -0.0642253568271
3.1 3 3.1 -0.160970897235
3.1 3 4.1 -0.261906474418
3.1 4 0.1 -6.09709039262e-06
3.1 4 1.1 -0.00798626913355
3.1 4 2.1 -0.0507564081263
3.1 4 3.1 -0.133349565782
3.1 4 4.1 -0.228563754423
4.1 0 0.1 4202.0
4.1 0 1.1 2.65740045079
4.1 0 2.1 -0.0462153115214
4.1 0 3.1 -0.358933906213
4.1 0 4.1 -0.439024390244
4.1 1 0.1 -1.11538537794e-05
4.1 1 1.1 -0.0144619860317
4.1 1 2.1 -0.0868190343718
4.1 1 3.1 -0.203767982755
4.1 1 4.1 -0.308519215265
4.1 2 0.1 -6.50646078271e-06
4.1 2 1.1 -0.0085156584289
4.1 2 2.1 -0.0538784714494
4.1 2 3.1 -0.140215240068
4.1 2 4.1 -0.23746380125
4.1 3 0.1 -4.54421180079e-06
4.1 3 1.1 -0.00597669061814
4.1 3 2.1 -0.038839789599
4.1 3 3.1 -0.106675396816
4.1 3 4.1 -0.192922262523
4.1 4 0.1 -3.48562423225e-06
4.1 4 1.1 -0.00459693165308
4.1 4 2.1 -0.0303305231375
4.1 4 3.1 -0.0860368842133
4.1 4 4.1 -0.162420599686

The solution to the initial problem is:

# Python-matplotlib Commands
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
import matplotlib.pyplot as plt
import numpy as np

fig = plt.figure()
ax = fig.gca(projection='3d')
X = np.arange(-5, 5, .25)
Y = np.arange(-5, 5, .25)
X, Y = np.meshgrid(X, Y)
R = np.sqrt(X**2 + Y**2)
Z = np.sin(R)
Gx, Gy = np.gradient(Z) # gradients with respect to x and y
G = (Gx**2+Gy**2)**.5  # gradient magnitude
N = G/G.max()  # normalize 0..1
surf = ax.plot_surface(
    X, Y, Z, rstride=1, cstride=1,
    facecolors=cm.jet(N),
    linewidth=0, antialiased=False, shade=False)
plt.show()

As far as I can see, and this applies to all matplotlib-demos, the variables X, Y and Z are nicely prepared. In practical cases this is not always the case.

Ideas how to reuse the given solution with arbitrary data?

Atwood

is it reading X and Y from the columns that you're having trouble with? – Geraldine

No, but the fact that (X;Y;Z) are not on a grid like the example. – Atwood

Answer 1

Great question Tengis, all the math folks love to show off the flashy surface plots with functions given, while leaving out dealing with real world data. The sample code you provided uses gradients since the relationships of a variables are modeled using functions. For this example I will generate random data using a standard normal distribution.

Anyways here is how you can quickly plot 4D random (arbitrary) data with first three variables are on the axis and the fourth being color:

from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
import numpy as np

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

x = np.random.standard_normal(100)
y = np.random.standard_normal(100)
z = np.random.standard_normal(100)
c = np.random.standard_normal(100)

img = ax.scatter(x, y, z, c=c, cmap=plt.hot())
fig.colorbar(img)
plt.show()

Note: A heatmap with the hot color scheme (yellow to red) was used for the 4th dimension

Result:

]1

Egalitarian

Great answer! How would one attach a colorbar as a scale for the heatmap? I am having difficulties with the code. – Prosthesis

Hi, this is a great solution for the type of graph I am aiming for. One question I have is really if it works with large datasets? Mine is 3mil + rows. – Bierce

@DrPepper I edited the answer with an example. You need to use colorbar to do this. – Egalitarian

@JoCosta Yes no problem so long as it can fit in memory. How large is your dataset and how much ram do you have? Pandas requires up to 10x the size of the dataset in Ram – Egalitarian

Is the from mpl_toolkits.mplot3d import Axes3D needed? It is colored gray when I paste it into PyCharm, but even PyCharm may miss some mysterious back-end importing issue. – Lemire

Answer 2

I know that the question is very old, but I would like to present this alternative where, instead of using the "scatter plot", we have a 3D surface diagram where the colors are based on the 4th dimension. Personally I don't really see the spatial relation in the case of the "scatter plot" and so using 3D surface help me to more easily understand the graphic.

The main idea is the same than the accepted answer, but we have a 3D graph of the surface that allows to visually better see the distance between the points. The following code here is mainly based on the answer given to this question.

import numpy as np
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
import matplotlib.tri as mtri

# The values ​​related to each point. This can be a "Dataframe pandas" 
# for example where each column is linked to a variable <-> 1 dimension. 
# The idea is that each line = 1 pt in 4D.
do_random_pt_example = True;

index_x = 0; index_y = 1; index_z = 2; index_c = 3;
list_name_variables = ['x', 'y', 'z', 'c'];
name_color_map = 'seismic';

if do_random_pt_example:
    number_of_points = 200;
    x = np.random.rand(number_of_points);
    y = np.random.rand(number_of_points);
    z = np.random.rand(number_of_points);
    c = np.random.rand(number_of_points);
else:
    # Example where we have a "Pandas Dataframe" where each line = 1 pt in 4D.
    # We assume here that the "data frame" "df" has already been loaded before.
    x = df[list_name_variables[index_x]]; 
    y = df[list_name_variables[index_y]]; 
    z = df[list_name_variables[index_z]]; 
    c = df[list_name_variables[index_c]];
#end
#-----

# We create triangles that join 3 pt at a time and where their colors will be
# determined by the values ​​of their 4th dimension. Each triangle contains 3
# indexes corresponding to the line number of the points to be grouped. 
# Therefore, different methods can be used to define the value that 
# will represent the 3 grouped points and I put some examples.
triangles = mtri.Triangulation(x, y).triangles;

choice_calcuation_colors = 1;
if choice_calcuation_colors == 1: # Mean of the "c" values of the 3 pt of the triangle
    colors = np.mean( [c[triangles[:,0]], c[triangles[:,1]], c[triangles[:,2]]], axis = 0);
elif choice_calcuation_colors == 2: # Mediane of the "c" values of the 3 pt of the triangle
    colors = np.median( [c[triangles[:,0]], c[triangles[:,1]], c[triangles[:,2]]], axis = 0);
elif choice_calcuation_colors == 3: # Max of the "c" values of the 3 pt of the triangle
    colors = np.max( [c[triangles[:,0]], c[triangles[:,1]], c[triangles[:,2]]], axis = 0);
#end
#----------
# Displays the 4D graphic.
fig = plt.figure();
ax = fig.gca(projection='3d');
triang = mtri.Triangulation(x, y, triangles);
surf = ax.plot_trisurf(triang, z, cmap = name_color_map, shade=False, linewidth=0.2);
surf.set_array(colors); surf.autoscale();

#Add a color bar with a title to explain which variable is represented by the color.
cbar = fig.colorbar(surf, shrink=0.5, aspect=5);
cbar.ax.get_yaxis().labelpad = 15; cbar.ax.set_ylabel(list_name_variables[index_c], rotation = 270);

# Add titles to the axes and a title in the figure.
ax.set_xlabel(list_name_variables[index_x]); ax.set_ylabel(list_name_variables[index_y]);
ax.set_zlabel(list_name_variables[index_z]);
plt.title('%s in function of %s, %s and %s' % (list_name_variables[index_c], list_name_variables[index_x], list_name_variables[index_y], list_name_variables[index_z]) );

plt.show();

Another solution for the case where we absolutely want to have the original values ​​of the 4th dimension for each point is simply to use the "scatter plot" combined with a 3D surface diagram that will simply link them to help you see the distances between them.

name_color_map_surface = 'Greens';  # Colormap for the 3D surface only.

fig = plt.figure(); 
ax = fig.add_subplot(111, projection='3d');
ax.set_xlabel(list_name_variables[index_x]); ax.set_ylabel(list_name_variables[index_y]);
ax.set_zlabel(list_name_variables[index_z]);
plt.title('%s in fcn of %s, %s and %s' % (list_name_variables[index_c], list_name_variables[index_x], list_name_variables[index_y], list_name_variables[index_z]) );

# In this case, we will have 2 color bars: one for the surface and another for 
# the "scatter plot".
# For example, we can place the second color bar under or to the left of the figure.
choice_pos_colorbar = 2;

#The scatter plot.
img = ax.scatter(x, y, z, c = c, cmap = name_color_map);
cbar = fig.colorbar(img, shrink=0.5, aspect=5); # Default location is at the 'right' of the figure.
cbar.ax.get_yaxis().labelpad = 15; cbar.ax.set_ylabel(list_name_variables[index_c], rotation = 270);

# The 3D surface that serves only to connect the points to help visualize 
# the distances that separates them.
# The "alpha" is used to have some transparency in the surface.
surf = ax.plot_trisurf(x, y, z, cmap = name_color_map_surface, linewidth = 0.2, alpha = 0.25);

# The second color bar will be placed at the left of the figure.
if choice_pos_colorbar == 1: 
    #I am trying here to have the two color bars with the same size even if it 
    #is currently set manually.
    cbaxes = fig.add_axes([1-0.78375-0.1, 0.3025, 0.0393823, 0.385]);  # Case without tigh layout.
    #cbaxes = fig.add_axes([1-0.844805-0.1, 0.25942, 0.0492187, 0.481161]); # Case with tigh layout.

    cbar = plt.colorbar(surf, cax = cbaxes, shrink=0.5, aspect=5);
    cbar.ax.get_yaxis().labelpad = 15; cbar.ax.set_ylabel(list_name_variables[index_z], rotation = 90);

# The second color bar will be placed under the figure.
elif choice_pos_colorbar == 2: 
    cbar = fig.colorbar(surf, shrink=0.75, aspect=20,pad = 0.05, orientation = 'horizontal');
    cbar.ax.get_yaxis().labelpad = 15; cbar.ax.set_xlabel(list_name_variables[index_z], rotation = 0);
#end
plt.show();

Finally, it is also possible to use "plot_surface" where we define the color that will be used for each face. In a case like this where we have 1 vector of values ​​per dimension, the problem is that we have to interpolate the values ​​to get 2D grids. In the case of interpolation of the 4th dimension, it will be defined only according to X-Y and Z will not be taken into account. As a result, the colors represent C (x, y) instead of C (x, y, z). The following code is mainly based on the following responses: plot_surface with a 1D vector for each dimension; plot_surface with a selected color for each surface. Note that the calculation is quite heavy compared to previous solutions and the display may take a little time.

import matplotlib
from scipy.interpolate import griddata

# X-Y are transformed into 2D grids. It's like a form of interpolation
x1 = np.linspace(x.min(), x.max(), len(np.unique(x))); 
y1 = np.linspace(y.min(), y.max(), len(np.unique(y)));
x2, y2 = np.meshgrid(x1, y1);

# Interpolation of Z: old X-Y to the new X-Y grid.
# Note: Sometimes values ​​can be < z.min and so it may be better to set 
# the values too low to the true minimum value.
z2 = griddata( (x, y), z, (x2, y2), method='cubic', fill_value = 0);
z2[z2 < z.min()] = z.min();

# Interpolation of C: old X-Y on the new X-Y grid (as we did for Z)
# The only problem is the fact that the interpolation of C does not take
# into account Z and that, consequently, the representation is less 
# valid compared to the previous solutions.
c2 = griddata( (x, y), c, (x2, y2), method='cubic', fill_value = 0);
c2[c2 < c.min()] = c.min(); 

#--------
color_dimension = c2; # It must be in 2D - as for "X, Y, Z".
minn, maxx = color_dimension.min(), color_dimension.max();
norm = matplotlib.colors.Normalize(minn, maxx);
m = plt.cm.ScalarMappable(norm=norm, cmap = name_color_map);
m.set_array([]);
fcolors = m.to_rgba(color_dimension);

# At this time, X-Y-Z-C are all 2D and we can use "plot_surface".
fig = plt.figure(); ax = fig.gca(projection='3d');
surf = ax.plot_surface(x2, y2, z2, facecolors = fcolors, linewidth=0, rstride=1, cstride=1,
                       antialiased=False);
cbar = fig.colorbar(m, shrink=0.5, aspect=5);
cbar.ax.get_yaxis().labelpad = 15; cbar.ax.set_ylabel(list_name_variables[index_c], rotation = 270);
ax.set_xlabel(list_name_variables[index_x]); ax.set_ylabel(list_name_variables[index_y]);
ax.set_zlabel(list_name_variables[index_z]);
plt.title('%s in fcn of %s, %s and %s' % (list_name_variables[index_c], list_name_variables[index_x], list_name_variables[index_y], list_name_variables[index_z]) );
plt.show();

Chromophore

Hi Vincent, could you please take a look at this question and see if you could kindly answer it too? Many thanks! – Bruni

great work..thanks – Jock

@Vincent Could you please review this [question ](#74369516) – Ear

Answer 3

I would like to add my two cents. Given a three-dimensional matrix where every entry represents a certain quantity, we can create a pseudo four-dimensional plot using Numpy's unravel_index() function in combination with Matplotlib's scatter() method.

import numpy as np
import matplotlib.pyplot as plt


def plot4d(data):
    fig = plt.figure(figsize=(5, 5))
    ax = fig.add_subplot(projection="3d")
    ax.xaxis.pane.fill = False
    ax.yaxis.pane.fill = False
    ax.zaxis.pane.fill = False
    mask = data > 0.01
    idx = np.arange(int(np.prod(data.shape)))
    x, y, z = np.unravel_index(idx, data.shape)
    ax.scatter(x, y, z, c=data.flatten(), s=10.0 * mask, edgecolor="face", alpha=0.2, marker="o", cmap="magma", linewidth=0)
    plt.tight_layout()
    plt.savefig("test_scatter_4d.png", dpi=250)
    plt.close(fig)


if __name__ == "__main__":
    X = np.arange(-10, 10, 0.5)
    Y = np.arange(-10, 10, 0.5)
    Z = np.arange(-10, 10, 0.5)
    X, Y, Z = np.meshgrid(X, Y, Z, indexing="ij")
    density_matrix = np.sin(np.sqrt(X**2 + Y**2 + Z**2))
    plot4d(density_matrix)

Robinia

This is an amazing solution. I wish this was part of matplotlib. – Noble

Gilfoyle, could you please look into this question – Ear

Answer 4

I also wanted to add my twocents!

I created a program for this a while back for a multivariable calculus course I was teaching and figured I would include it here.

My approach is similar to others that have posted (using a scatterplot), however, I made some changes to really make the graphics pop.

Specifically, I included a function to remove a portion of the Alpha channel range to make portions of the range transparent. This is controlled by the function f_AlphaControl in the code below.

The function I used in the demo is the function f(x,y,z)=xyz*exp(-x^2-y^2-z^2). It has 4 local max and 4 local min, all of which are visualized in the plots below. I think the results speak for themselves so please take a look at them and let me know what you think 😃.

2700 points: epsilon=2 , epsilon=1 , epsilon=.5 , epsilon=.1 , epsilon=5Andx0=-1

1000000 points: epsilon=5 , epsilon=1

This code allows for creation of isovolume plots that rival Mayavi-VTK and/or OpenGL but without all of the effort. I have similar routines coded up in Mayavi, however, since OP just asked about matplotlib, I wanted to show how powerful it can be.

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm

from matplotlib.colors import ListedColormap

## The following code creates the figure plotting function
def MakePlot(xx,yy,zz,ww,cmapO=cm.jet): 
    ##Create Custom Colormap with Alpha varying depending on the functions behavior. 
    ## This produces very nice isovolume plots similar to Mayavi and OpenCV
    
    ##Preallocate new colormap
    my_cmapN=cmapO(np.arange(cmapO.N))
    
    #set Alpha of new colormap to be small in the middle of the colormap range using a bump function
    # this can be changed to emphasize different areas of the range that are of interest.   
    nA=cmapO.N    
    xA=np.linspace(-1,1,nA)
    epsilon=1.5 #Width of range to exclude from alpha channel
    x_0=0#Center of range to exclude from alpha channel
    
    def f_AlphaControl(x):
        u=(x-x_0)/epsilon
        return 1-np.exp(-u**2/(1-u**2))*(np.abs(u)<1.)

    yA=f_AlphaControl(xA) 
    plt.plot(xA,f_AlphaControl(xA))
    plt.xlim([-1,1])
    plt.ylim([0,1])
   
    my_cmapN[:,-1]=yA
    
    fig = plt.figure(dpi=200) 
    
    # Create new colormap
    my_cmap = ListedColormap(my_cmapN) 
    
    
    plt.style.use('dark_background')
    fig = plt.figure(dpi=200) 
    ax = fig.add_subplot(projection='3d')
    points=ax.scatter(xx,yy,zz,c=ww,cmap=my_cmap)   
    cbar=fig.colorbar(points)
    # cbar.solids.set_rasterized(True)
    
    cbar.set_alpha(1)
    cbar.draw_all()
    
    ## Make Title for plot
    ax.set_title(r'Plot of $f:\mathbb{R}^3\rightarrow \mathbb{R}$'+'\n'+r'$w=f(x,y,z)$')
      
    ##Plot x, y, and z axis useful for visual referencing when viewing the plot
    eps=.3
    tt=np.linspace(-(1+eps)*L,(1+eps)*L,2)
    ax.plot(tt,0*tt,0*tt,c='magenta',linewidth=2);
    ax.plot(0*tt,tt,0*tt,c='magenta',linewidth=2);
    ax.plot(0*tt,0*tt,tt,c='magenta',linewidth=2)
      
    ## Set viewing angle
    xang=-76;pang=12;
    ax.view_init(pang, xang)
    
    # Set axis limits
    ax.set_xlim([-(1+eps)*L,(1+eps)*L]);ax.set_ylim([-(1+eps)*L,(1+eps)*L]);ax.set_zlim([-(1+eps)*L,(1+eps)*L]);
       
    #Set axis labels
    ax.set_xlabel('$x$');ax.set_ylabel('$y$');ax.set_zlabel('$z$')
    plt.savefig("ScatterPlotVaryingAlpha.png",dpi=200)

if __name__ == "__main__":

    #Set Plot Grid 
    
    L=1.5
    x_C=0.0;y_C=0.0;z_C=0.0;
    # Set XYZ Plotting Grid
    a1=x_C-L;b1=x_C+L;
    a2=y_C-L;b2=y_C+L;
    a3=z_C-L;b3=z_C+L;
    
    n=30;
    NT=n**3
    
    
    
    ##The following if statement determines whether you want to use a random grid R=1 
    ## or a uniform grid R=0
    
    grid_flag=1
    
    if grid_flag==1:
       ## Random Grid
       xx=np.random.uniform(a1,b1,NT);
       yy=np.random.uniform(a2,b2,NT);
       zz=np.random.uniform(a3,b3,NT)
    else:
        ## Even Grid
        x = np.linspace(a1,b1,n);
        y = np.linspace(a2,b2,n);
        z = np.linspace(a3,b3,n);
    
        X, Y, Z = np.meshgrid(x, y, z, indexing='ij', sparse=False)
        
        xx=X.reshape(X.size);yy=Y.reshape(Y.size);zz=Z.reshape(Z.size);
    
    
    
    ## The following code defines the function of 3 variables that we wish to visualize
    ## This can be replaced with the flattened data array that you wish to plot
    
    def f(x,y,z):
        return x*y*z*np.exp(-(x**2+y**2+z**2))
    
    ww=f(xx,yy,zz)
    ww[np.isinf(ww)]=np.nan
    
  
    MakePlot(xx,yy,zz,ww)
    plt.show()

Proa

Answer 5

One possibility would be use a color space, for example RGBA or HSVA, they are 4 dimensional, but displaying the alpha (transparency) well may be a problem.

Other possibility would be a dynamic plot with a slider. One of the dimensions would be represented by the slider.

I am not sure if that is what you are asking, though.

Odessa