tl;dr You're conflating instant/range vectors with instant/range queries.
These are 3 time series for the same metric (container_memory_rss
):
time (min) 0 1 2 3 4 5 now
------------------------------------------------------------------------------------
container_memory_rss{container="fluentbit"} 34 55 32 34 45 34
container_memory_rss{container="grafana"} 467 647 654 564 563 455
container_memory_rss{container="mysql"} 9744 9394 6345 8895 8867 9645
Instant vectors and range vectors are both PromQL expression types (the others being strings, scalars, and native histograms (experimental). Whenever you evaluate PromQL expressions like container_memory_rss
or 5-2
, those expression evaluates to one of these types (instant vector and scalar, respectively).
When the PromQL expression is a time series selector (e.g. container_memory_rss
), Prometheus will take a single sample from each time series that is closest to the evaluation time, giving you an instant vector. So an instant vector may look like this:
container_memory_rss{container="fluentbit"} 34
container_memory_rss{container="grafana"} 455
container_memory_rss{container="mysql"} 9645
When the PromQL expression is a time series selector with a time duration (e.g. container_memory_rss[3m]
), Prometheus will take all the samples from each time series within the specified duration, giving you a range vector that may look like this:
container_memory_rss{container="fluentbit"} 34 45 34
container_memory_rss{container="grafana"} 564 563 455
container_memory_rss{container="mysql"} 8895 8867 9645
Now, let's talk about instant queries and range queries.
Instant queries show the result of a PromQL expression evaluated at a single point in time (called the 'evaluation timestamp'), whereas range queries show the result of a PromQL expression evaluated at multiple, discrete steps (called a 'resolution step') between a start and end time.
A instant query of a PromQL expression that evaluates to an instant vector may look like this:
container_memory_rss{container="fluentbit"} 34
container_memory_rss{container="grafana"} 455
container_memory_rss{container="mysql"} 9645
This is typically shown as a table (below is a screenshot from Grafana):
A range query of a PromQL expression that evaluates to an instant vector may look like this:
t0:
container_memory_rss{container="fluentbit"} 34
container_memory_rss{container="grafana"} 467
container_memory_rss{container="mysql"} 9744
t2:
container_memory_rss{container="fluentbit"} 32
container_memory_rss{container="grafana"} 654
container_memory_rss{container="mysql"} 6345
t4:
container_memory_rss{container="fluentbit"} 45
container_memory_rss{container="grafana"} 563
container_memory_rss{container="mysql"} 8867
This is typically visualized as a line graph, but to demonstrate the point more clearly, here it is shown as a points:
Each vertical slice on the graph is a single evaluation of the PromQL expression at that point in time. In effect, a range query is many instant queries stitched together.
A instant query of a PromQL expression that evaluates to an range vector may look like this:
container_memory_rss{container="fluentbit"} 34 45 34
container_memory_rss{container="grafana"} 564 563 455
container_memory_rss{container="mysql"} 8895 8867 9645
It can be plotted on Grafana like this:
But note that each point in this plot is not multiple evaluations of a PromQL expression that each produced an instant query, but a single evaluation of a PromQL expression that produces a range query containing multiple samples. Prometheus just looks back for 5 minutes (in this example) at the instant the PromQL expression is evaluated.
A range query of a PromQL expression that evaluates to an range vector may look like this:
t3:
container_memory_rss{container="fluentbit"} 55 32 34
container_memory_rss{container="grafana"} 647 654 564
container_memory_rss{container="mysql"} 9394 6345 8895
t4:
container_memory_rss{container="fluentbit"} 32 34 45
container_memory_rss{container="grafana"} 654 564 563
container_memory_rss{container="mysql"} 6345 8895 8867
t5:
container_memory_rss{container="fluentbit"} 34 45 34
container_memory_rss{container="grafana"} 564 563 455
container_memory_rss{container="mysql"} 8895 8867 9645
Trying to plot this on Grafana will give the error:
bad_data: invalid parameter "query": invalid expression type "range vector" for range query, must be Scalar or instant Vector
This may be confusing because, to us, a range vector contains a number of samples and we can logically plot those samples on a graph. However, Prometheus treats the range vector as a single value, just like a single scalar (232
). Prometheus doesn't care that a range vector contains samples; that is opaque to Prometheus.
For range queries, for each resolution step, each time series must provide only a single data point.
Therefore, another way to think of it is: given that each time series within a range vector contains multiple samples, if Prometheus were to try to plot it, how do you plot multiple sample values as a single point? Or - how can it plot container_memory_rss{container="fluentbit"}55 32 34
as a single point? It can't.
To visualize a range vector as a range query, you must do some processing on the range vector to turn it into an instant vector. For example, you can use the max_over_time
range function that returns the maximum value of all samples in the range vector, and use that as the value for the time series at each resolution step.
start
,end
, andstep
implicitly if you open the debug window observing the request. so you inputInstant vector
and the server returnsRange vector
. if you inputRange vector
, it would be range of range and illegal. – Plater