How to assign a "memory priority" to a linux process?

Question

I am running tor on a small OpenWRT router, where swapping can't be avoided due to the limited amount of ram available (32mb).

Most of the time the router doesn't do anything else, however from time to time a postgresql database also running on the router is accessed. Due to tor constantly running, postgresql is completly swapped out and the first few accesses have very high latency, which is bad because it is a interactively used system.

I've already assigned a nice value of -15 to postgres and +15 to tor, but it doesn't seem to affect memory management very much. Setting swappiness=1 globally doesn't change things too, because swapping can not be avoided, and because postgresql isn't running most of the time, it is swapped out anyway.

Is there any way to a something like a memory priority to a Linux process? I had a look at cgroup specific swappiness, however the only description I found was that it affects the decision page-cache vs swap.

What I am looking for is a parameter to tell the linux kernel to not swap out postgresql as aggresivly as the other processes (but I don't want to mlock the whole process). Or would assigning swappiness=80 system-wide and swapiness=1 for postgresql keep postgresql in memory while swaping out everything else when required?

Canonize

Answer 1

Strictly speaking, in Linux you can't prevent a process from swapping out. You can avoid swapping totally using swapoff -a (and adding some RAM) but this can lead the system to instability.

But in this case Linux is doing a good job: a process used "from time to time" must be swapped out, no matter how many free RAM you have. Maybe you are using a wrong configuration. Can you put postgres on another host, maybe with a faster hard disk?

BTW: if you want to prevent the swapping out of postgres process in you current configuration, I think that you can try to use something like a keep alive: let a deamon (or a simple bash script) to periodically send some query to let the system see that the process active.

I.E. you can do something like:

#!/bin/bash
DBHOST=localhost
DBPORT=5432
DBNAME=theDBname
DBUSER=theUserName
THEQUERY="SELECT 1"

psql -h $DBHOST -p $DBPORT -d $DBNAME -U $DBUSER  -c "$THEQUERY"

And let the cron call it each minute or so.

If you wants something more sophisticated, you can create a daemon to send some "real" query and cache the results, so postgres can swap out while you already have cached results.

Clouet

> Strictly speaking, in Linux you can't prevent a process from swapping out......You can, simply call mlock with the right parameters and the process will be pinned in memory. Thanks for the daemon idea - this is what I do currently, causing postgres to be periodically swapped in/out. What I would like to achieve is that postgresql is swapped out with lower priority than other prozesses. – Canonize

Yes you can use mlock but in this case you must modify postgres source code, and it's a not so easy task. – Clouet

Answer 2

I think what you should be doing is configuring PostgreSQL to use less memory. It uses a large amount of memory in an attempt to improve performance, but in your case its not working. The default configuration for your OpenWRT PostgreSQL installation presumably already reduces its memory footprint from its massive normal defaults, but it looks like you need to go further.

The PostgreSQL documentation lists a number of options that affect its resource usage.

Failing that the other suggestions by StefanoF and Basile Starynkevitch to either move PostgreSQL server to another machine or use more efficient database like sqlite are better solutions than trying to improve your swapping performance from absolutely terrible to merely awful.

Blocker

Answer 3

You might use mlock(2) and more probably madvise(2) inside your program (probably Postgresql).

However, I believe that Postgresql (or Mysql) is too big for a 32 Mbytes RAM system. Did you consider sqlite ? It is a library (not a server) that you could link into your application, and with it you can do SQL "requests" on a local sqlite file (acting as a database). Of couse, you might need to make your application able to interact thru the web (e.g. with FASTCGI or with some HTTP server library like libonion).

Perhaps ionice(1) might be relevant (but I guess not).

Mudguard

what do you mean by sqlite? as far as i know sqlite does not run a daemon nor serve tcp/ip. – Alit