If I want to let a python process to use maximum the 90% of the total memory available, I'm using this command:
ulimit -Sv $((`python3 -c'import psutil;print(psutil.phymem_use().total)'`*9/10/1024)); python3 mystuff.py
With ulimit you can set the limit parameter for the current session.
To dump the corrent config, simple run the following command:
>ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 63785
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 63785
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
You can change the values with the -S and use the param argument from the brackets.
In this case we set the virtual memory parameter with -v
To get the total available memory, you have multiple options:
- Option
cat /proc/meminfo | awk '/MemTotal/ {print $2}' - Option
free | awk '/^Mem:/ {print $2}' - Option
python3 -c 'import psutil; print(psutil.phymem_use().total)
Please keep in mind, you need to change this value to KB,
Examples in a 16 GB RAM machine:
>cat /proc/meminfo | awk '/MemTotal/ {print $2}'
16351304
>free | awk '/^Mem:/ {print $2}'
16351304
>python3 -c 'import psutil; print(psutil.phymem_usage().total)'
16743735296
As you can see, in the last option, it is in bytes, so need to divide with 1024.
If your application reaches the limit, it will get a signal 6 - Abort signal from the Linux.
Notes:
- You can use bc to calculate the value to set, but in this example it was not available, so simple bash calculation was used
- You can calculate the value with python, but this example keeps the chance to replace that part.
