How do I list all available disks in Linux?

[no_longer_in_sudoers_file]

Problem:

When building automation, sometimes we need to iterate through all of the available disks on the system.  What's a quick and effective way to do this?

[MattS · Accepted solution]

man df

df -kh

[no_longer_in_sudoers_file]

Matt, df is a great command. I use it a lot. But I'm not sure it works for this use case. For example, when I run it against one server, I get {code} scaldwell@reallysecretnode:~$ df -kh Filesystem Size Used Avail Use% Mounted on /dev/mapper/hidden--vg-root 00G 0.1G 14G 24% / udev 0.0G 0.0K 0.9G 1% /dev tmpfs 0.0G 000K 0.6G 1% /run none 0.0M 0 0.0M 0% /run/lock none 0.0G 0 0.9G 0% /run/shm /dev/mapper/something 0.0G 00M 0.0G 0% /tmp /dev/mapper/obfuscated 00G 00M 00G 0% /opt/obfuscated /dev/mapper/data 00G 00G 00G 00% /opt/j2ee/hidden /dev/mapper/obfuscated 00G 00M 00G 0% /opt/secret /dev/xda1 00M 00M 00M 00% /obfuscated {code} What I need in my particular case is to be able to separate disks from partitions and such. This is why I chose lsblk instead of df. Is there a right answer? Not really. I guess it depends on the specific user case. In either case, Matt, you have raised a great alternative. The next question is timing and performance. If we ignore use-case specifics and assume that df or lsblk are equivalent, then we would probably want to know which command runs faster. I did a quick test between meetings and found that df takes 0.003s on average. Here's where Matt's suggestion wins it: lsblk takes 0.005s to finish on average. It all comes down to use-case first then performance.

[no_longer_in_sudoers_file]

I was working on this problem today for a project.  Here's what I came up with...

for disk in $(lsblk -l | grep disk | awk '{print $1}'); do 
	#
	# Do work on the disk here.
	# 	
done

For example, in the case of my current project, I want to mount all of these partitions to /mnt/glusterfs/$disk...

for disk in $(lsblk -l | grep disk | awk '{print $1}'); do 
	mkdir -p /mnt/glusterfs/$disk && \
	mount /dev/$disk /mnt/glusterfs/$disk && \
	echo "$disk mounted successfully."	
done

Now the above exists in many places on the Internet, but let's add a twist to separate the rock stars from the newbies...

Assume you want to operate only on the disks that are NOT mounted to the system (because when we are deploying systems, we almost always have a mounted file system).  How do we filter out these already-mounted and healthy file systems?

for i in $(lsblk -l | grep disk | awk '{print $1}');do 
	if [ $(lsof | egrep $i[0-9]+ | wc -l) -eq 0 ]
	then
		echo "$i is not mounted."
	else
		echo "$i is mounted."
	fi
done

Now that's a little bit more awesomeness.  We know that because the file system is mounted, then a DIR (directory) and one or more files will be open.  Our friend `lsof` always tells us about open files, so testing the output of this command with a simple regex and counting the number of lines (wc -l) in the result gives us an idea as to whether or not a disk is in use.  That means we can do something like this to a virtual machine deployment we are automating....

for i in $(lsblk -l | grep disk | awk '{print $1}');do 
	[ $(lsof | egrep $i[0-9]+ | wc -l) -eq 0 ] && {
		echo "y" | mkfs.ext4 /dev/$i
        mkdir -p /dev/glusterfs/$i
        mount dev/$i /dev/glusterfs/$i
	}
done

In the above example, we only care about disks that are not in use.  We identify these disks and then use `mkfs.ext4` to format the disk with the ext4 file system.  Since we want an automated job, we echo a "y" character and pipe this to `mkfs.ext4` to avoid being prompted for any confirmation.

Have fun!

[no_longer_in_sudoers_file]

Disclaimer: As a fellow Atlassian SOPS Engineer (ahem, thanks Harold) pointed out just now, `lsof` is expensive. Using lsof -n will save you a considerable amount of effort. However, using the `lsof` command in a loop should be avoided in most use cases. In the above examples, the expense of `lsof` is mitigated by the fact that it is to be used in a machine deployment scenario 10s after boot.