Originally published August 26, 2020 @ 12:25 pm
This is a depressing – and all-too-common – scenario: a user runs the free command and opens a support case because he sees zero “free” memory on the server.
For years I’ve been trying to explain to users – developers, application engineers – how memory allocation works in Linux. It’s an utterly futile endeavor. Sometimes I just want to soft-link the free command to exit.
Starting with RHEL 7, we’ve been blessed with an updated version of free that now adds the Available column to the output. This helps sometimes, but users are still confused as they don’t understand the difference between “free” and “available”.
Here’s output of free on RHEL 6:
root@hecuba# free -h
total used free shared buffers cached
Mem: 62G 61G 1.1G 677M 951M 39G
-/+ buffers/cache: 21G 41G
Swap: 8.0G 804M 7.2G
And here’s the output of free on RHEL 7:
[root@kassandra~]# free -h
total used free shared buff/cache available
Mem: 7.6G 478M 400M 9.5M 6.7G 6.8G
Swap: 2.0G 0B 2.0G
I thought this was a problem of presentation and so I set out to write a quick script that would detail current system memory utilization in a way that your average developer can understand.
It also occurred to me that instead of relying on local system tools to gather this information, I should use snmpwalk. This would allow for wider access and more standardized presentation. And so I picked a major monitoring software vendor – Solarwinds – and decided to stick with its standards.
According to Solarwinds documentation, the percentage of “used memory” is calculated like so:
Used Memory = (Total RAM in machine - Total RAM Available - Cached Memory - RAM Buffered) / Total RAM * 100
Here’s a practical example showing how to calculate the percentage of used memory using SNMP data:
h=<target_host>
s=<snmp_community_string>
# Define relevant OIDs
total_ram='1.3.6.1.4.1.2021.4.5'
total_ram_available='1.3.6.1.4.1.2021.4.6'
total_ram_buffered='1.3.6.1.4.1.2021.4.14'
total_cached_memory='1.3.6.1.4.1.2021.4.15'
# Configure snmpwalk function
walk() {
snmpwalk -v 2c -c $s $h $@
}
# Configure output formatting
chomp() {
awk -F: '{print $NF}' | grep -oP '[0-9]{1,}'
}
[root@devsw01 ~]#
echo "Used Memory: $(echo "scale=0;($(walk ${total_ram} | chomp)\
> -$(walk ${total_cached_memory} | chomp)\
> -$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)%"
Used Memory: 32%
The script below (also on GitHub) shows reasonably detailed memory utilization information that can be used to calm a panicked mind. If this doesn’t help, check with the management to see if the mind in question can be better utilized elsewhere.
#!/bin/bash
#
# |
# ___/"\___
# __________/ o \__________
# (I) (G) \___/ (O) (R)
# Igor Os
# igor@comradegeneral.com
# 2020-04-13
# ----------------------------------------------------------------------------
# Retrieve a server's memory utilization information via SNMP
# ----------------------------------------------------------------------------
# CHANGE CONTROL
# ----------------------------------------------------------------------------
# 2020-04-13 igor wrote this script
h=""
if [ -z "${h}" ]
then
echo "You need to supply this command with a valid hostname. Exiting..."
exit 300
else
ping_status=$(timeout 6 ping -c 2 -W 1 ${h} >/dev/null 2>&1 ; echo $?)
if [ -z "${ping_status}" ] || [ ${ping_status} -ne 0 ]
then
echo "Unable to ping ${h}. Exiting..."
exit 310
fi
fi
configure() {
total_ram='1.3.6.1.4.1.2021.4.5'
total_ram_available='1.3.6.1.4.1.2021.4.6'
total_ram_buffered='1.3.6.1.4.1.2021.4.14'
total_cached_memory='1.3.6.1.4.1.2021.4.15'
# Update the SNMP community string. Make sure the script has appropriate permissions
s="*********************"
RED='3[1;31m'
GREEN='3[1;32m'
YELLOW='3[1;33m'
NC='3[0m'
hfqdn="$(nslookup $h 2>/dev/null | grep -m1 '^Name:' | awk '{print $NF}')"
if [ -z "${hfqdn}" ]
then
hfqdn="${h}"
fi
}
walk() {
snmpwalk -v 2c -c $s $h $@
}
chomp() {
awk -F: '{print $NF}' | grep -oP '[0-9]{1,}'
}
mem_check() {
echo "-------------------------------------"
echo "${hfqdn}"
echo "-------------------------------------"
echo -e "Total RAM installed:\t ~$(echo "scale=0;($(walk ${total_ram} | chomp))/1024/1024"|bc -l)GB"
echo -e "Allocated memory:\t ~$(echo "scale=0;($(walk ${total_ram} | chomp)\
-$(walk ${total_ram_available} | chomp))/1024/1024"|bc -l)GB"
echo -e "Unallocated memory:\t ~$(echo "scale=0;($(walk ${total_ram_available} | chomp))/1024/1024"|bc -l)GB"
echo -e "Utilized memory:\t ~$(echo "scale=0;($(walk ${total_ram} | chomp)\
-$(walk ${total_cached_memory} | chomp)\
-$(walk ${total_ram_buffered} | chomp))/1024/1024"|bc -l)GB ($(echo "scale=0;($(walk ${total_ram} | chomp)\
-$(walk ${total_cached_memory} | chomp)\
-$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)%)"
available_pct="$(echo "scale=0;($(walk ${total_ram_available} | chomp)\
+$(walk ${total_cached_memory} | chomp)\
+$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)"
if [ ${available_pct} -gt 40 ]
then
highlight="${GREEN}"
elif [ ${available_pct} -le 40 ] && [ ${available_pct} -gt 15 ]
then
highlight="${YELLOW}"
elif [ ${available_pct} -le 15 ]
then
highlight="${RED}"
fi
echo ""
echo -e "${highlight}Available memory:\t ~$(echo "scale=0;($(walk ${total_ram_available} | chomp)\
+$(walk ${total_cached_memory} | chomp)\
+$(walk ${total_ram_buffered} | chomp))/1024/1024"|bc -l)GB ($(echo "scale=0;($(walk ${total_ram_available} | chomp)\
+$(walk ${total_cached_memory} | chomp)\
+$(walk ${total_ram_buffered} | chomp)) * 100 / $(walk ${total_ram} | chomp)"|bc -l)%)${NC}"
echo ""
echo "Available memory composition:"
echo -e "\tCached: \t ~$(echo "scale=0;($(walk ${total_cached_memory} | chomp))/1024/1024"|bc -l)GB"
echo -e "\tBuffered:\t ~$(echo "scale=0;($(walk ${total_ram_buffered} | chomp))/1024/1024"|bc -l)GB"
echo -e "\tUnallocated:\t ~$(echo "scale=0;($(walk ${total_ram_available} | chomp))/1024/1024"|bc -l)GB"
echo "-------------------------------------"
}
# ----------------------------------------------------------------------------
# RUNTIME
# \(^_^)/ __|__
# __|__ *---o0o---*
# __|__ *---o0o---*
# *---o0o---*
# ----------------------------------------------------------------------------
configure
mem_check
And sample output:
[root@devsw01~]# memwalk hecuba
-------------------------------------
hecuba.kc16.local
-------------------------------------
Total RAM installed: ~62GB
Allocated memory: ~62GB
Unallocated memory: ~0GB
Utilized memory: ~22GB (35%)
Available memory: ~41GB (65%)
Available memory composition:
Cached: ~40GB
Buffered: ~0GB
Unallocated: ~0GB
-------------------------------------
Experienced Unix/Linux System Administrator with 20-year background in Systems Analysis, Problem Resolution and Engineering Application Support in a large distributed Unix and Windows server environment. Strong problem determination skills. Good knowledge of networking, remote diagnostic techniques, firewalls and network security. Extensive experience with engineering application and database servers, high-availability systems, high-performance computing clusters, and process automation.
