R: how to check how many cores/CPU usage available

Question

R is single-threaded.

1. Using R, how to check how many cores/threads are running R in Windows and Linux? (Or how many Rs are running)
2. Using R, how to check the usage of each core that is running R in Windows and Linux? (Or the percentage of CPU each R is using)

For example, if I have two R opened running projects. I would expect that there are 2 threads running R with some % of CPU for each thread. Then I open another R. How to use the third R to check the number of threads (2 in this case) and percentage of CPU being used by R?

Care

Don't think you'll find much (though I'm frequently surprised). Your best bet is to use task manager (windows) or something like top or htop (linux) to see what each core is doing, remembering that R is single-threaded by default. There are math libraries that R might use that take advantage of multiple threads, but R doesn't report on that, only the OS does. – Debauch

If the system can do it, then I guess I can use system() in R to do so as well? – Care

Sure, depending on your OS and tools available. – Debauch

Answer 1

On Linux you can send ps command to the system: it gives you the average cpu usage and the memory usage of the program called rsession:

splitted <- strsplit(system("ps -C rsession -o %cpu,%mem,pid,cmd", intern = TRUE), " ")
df <- do.call(rbind, lapply(splitted[-1], 
                            function(x) data.frame(
                                cpu = as.numeric(x[2]),
                                mem = as.numeric(x[4]),
                                pid = as.numeric(x[5]),
                                cmd = paste(x[-c(1:5)], collapse = " "))))
df
#  cpu mem   pid   cmd
#1 0.8 0.7 11001  /usr/lib/rstudio/bin/rsession 
#2 0.0 0.2 12397  /usr/lib/rstudio/bin/rsession
#3 0.1 0.7 14960  /usr/lib/rstudio/bin/rsession
#4 0.4 0.2 26122  /usr/lib/rstudio-server/bin/rsession 
#5 0.3 8.3 35782  /usr/lib/rstudio/bin/rsession

You can probably improve it to get the parent id and the instantaneous CPU usage with other options passed to ps or top and deduce the number of cores used by each session.

On Windows you can try this:

a <- system("wmic path Win32_PerfFormattedData_PerfProc_Process get Name,PercentProcessorTime", intern = TRUE)
df <- do.call(rbind, lapply(strsplit(a, " "), function(x) {x <- x[x != ""];data.frame(process = x[1], cpu = x[2])}))
df[grepl("Rgui|rstudio", df$process),]
#     process cpu
# 105    Rgui   0
# 108 rstudio   0

Dobbs

Thanks. How would the commands look like for Mac OS? – Katey

Answer 2

If you open multiple R windows, each window will be running on a different core up to the maximum number of cores that you have. This is automatically implemented on windows and mac computers. If you want to know how many cores you have, you can run:

library(parallel)
detectCores()

Ruhl

Thanks but this is not what I am asking. I’d like to know how to use R to check how many Rs or CPU usage of each R I am running. – Care

Answer 3

Using R, how to check how many cores/threads are running R in Windows and Linux? (Or how many Rs are running)

One valid answer that I haven't read yet here is simply using the ps R package with the function ps() you can then subset the table returned by processes with the name "rsession":

ps::ps()[ps::ps()$name == "rsession",]

Number of rows will give you the number of sessions existing on the computer/server:

nrow(ps::ps()[ps::ps()$name == "rsession",])

I am not entirely sure about what the function ps_num_threads() does but it might also be interesting to check if the result make sense:

ps::ps_num_threads(ps::ps_handle())

I unfortunately did not found anything about %CPU usage in the ps R package but you can give a try to the function I quote in my other answer, it should work under Linux.

Blessington

Answer 4

For those who would like to know the number of cores/cpus and/or number of workers/compute nodes that 1) are available in the machine or 2) are allocated by HPC clusters on which current R programm is running, try this (using functions from parallel and future packages):

library(parallel) # for using parallel::mclapply() and checking #totalCores on compute nodes / workstation: detectCores()
library(future) # for checking #availble cores / workers on compute nodes / workstation: availableWorkers() / availableCores() 

workers <- availableWorkers()
cat(sprintf("#workders/#availableCores/#totalCores: %d/%d/%d, workers:\n", length(workers), availableCores(), detectCores()))
print( workers )

Doublebank

Answer 5

There is a simpler method, using benchmarkme package.

library(benchmarkme)
get_cpu()$no_of_cores

Deuteragonist

Sorry, this only will show you the number of cores that the PC has, not the ones available – Deuteragonist

Answer 6

For those interested, a friend and I have created a Github Repository with functions to probe ongoing processes on a computer/server from a R console.
Here is the link: https://github.com/mathosi/cluster_check

To answer the question you can use the function I made ps.to.df():

devtools::source_url("https://github.com/mathosi/cluster_check/blob/master/ps_to_df.R?raw=TRUE")
ps.to.df() #List All processes sorted by %CPU usage
ps.to.df(bylist.selection = "C rsession") #List All processes named 'rsession' sorted by %CPU usage

The output is a data.frame so you can then sort it subset it the way you want in R to look for whatever you want to see!
I haven't try yet all the possible query ps.to.df() supports, but I guess it should support others.
There is room to improve flexibility and readability of the outputs, maybe to create additionnal functions also. Anyone interested can join and contribute.

Blessington

Answer 7

Maybe I found a way to fetch core workload in percent while using powershell on windows systems:

library(tidyverse)

command = 'Get-WmiObject -Query \'select Name, PercentProcessorTime from Win32_PerfFormattedData_PerfOS_Processor\' | foreach-object { write-host "$($_.Name): $($_.PercentProcessorTime)" }; '
output = system2('powershell', c('-Command', command), stdout = T)

output_lst = strsplit(output, split = " : ") 
cores = data.frame(t(sapply(output_lst,c))) %>% 
  rename( "Core" = X1 , `CPU_%` = X2)

Gives you a list of cores with their workload:

> cores
    Core CPU_%
1       0    62
2       1    19
3      10     0
4      11    12
5      12     6
6      13     0
7      14    12
8      15     0
9      16    56
10     17    31
11     18    12
12     19    12
13      2    50
14     20     0
15     21     6
16     22    12
17     23     6
18     24    12
19     25    19
20     26    19
21     27    43
22     28    31
23     29    12
24      3    12
25     30     6
26     31    25
27      4    43
28      5    37
29      6     0
30      7     6
31      8     0
32      9     0
33 _Total    18

Filtering the unloaded cores:

free_cores = cores %>% filter(`CPU_%` == 0) %>% nrow()
> free_cores
[1] 7

Tendance