Python Running Shell Command and Capturing the Output. In automation, and scripting the ability to interact with the underlying operating system through shell commands and capture the output for further processing is important. Python offers several method to execute these commands and captured the output. In this article, we will discuss the best methods for this task.
1. Quick Examples – Running Shell Command and Capturing the Output
Let’s take a quick look at the below quick examples. These code examples provide a high level overview of how each method captures shell command output in Python.
import subprocess
import sh
# Using subprocess.Popen to Capture Output
process = subprocess.Popen("ls -l", shell=True, stdout=subprocess.PIPE)
output = process.stdout.read()
# Using the sh Module for Capturing Output
output = sh.ls("-l")
# Using check_output for Capturing Output
output = subprocess.check_output("ls -l", shell=True)
2. subprocess.Popen method to Capture Output
The subprocess module is actually used to run shell commands and capture their output. The subprocess.Popen allows you to interact with the shell, execute commands, and retrieve both the standard output and standard error streams from the executed command.
To use subprocess.Popen, you need to create an instance of the class and specify the command you want to execute. See the below example:
import subprocess
# Define the shell command you want to run
cmd = "ls -l"
# Create a subprocess.Popen instance
process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
# Wait for the command to complete and capture the output
stdout, stderr = process.communicate()
# Check the return code to see if the command succeeded
return_code = process.returncode
# Print the captured output and return code
print("Standard Output:")
print(stdout.decode("utf-8"))
print("Standard Error:")
print(stderr.decode("utf-8"))
print(f"Return Code: {return_code}")
Yields the below output:
3. The ‘sh’ Module for Capturing Output
The sh module is another best method just like the popen to run shell commands from Python and capture their output. This method provides a more Pythonic and user-friendly interface for executing shell commands compared to the lower-level subprocess module.
Before using the sh module, you need to install it. You can do this using pip:
pip install sh
See the below example:
import sh
# Define the shell command you want to run
cmd = "ls -l"
# Run the command and capture the output
output = sh.ls("-l")
# Print the captured output
print(output)
To capture and process the output line by line, you can iterate over the lines in the output object. This is useful for commands that produce multiline output:
for line in output:
print(line.strip())
4. Using check_output for Capturing Output
The check_output function in the subprocess module is a simple and effective way to run shell commands from Python and capture their output. It is useful when you need to execute a command and retrieve its output as a single string.
You pass the shell command as a string to the function, and it returns the command’s output as a byte string. Here’s a basic example:
import subprocess
# Define the shell command you want to run
cmd = "ls -l"
# Run the command and capture the output
output = subprocess.check_output(cmd, shell=True)
# Decode the byte string to a regular string
output = output.decode("utf-8")
# Print the captured output
print(output)
# Output:
# -rw-rw-r-- 1 ali ali 288 Oct 13 02:02 app.py
# drwxrwxr-x 5 ali ali 4096 Oct 12 19:39 myvenv
5. Using InteractiveShellEmbed from IPython
IPython is an enhanced interactive Python shell, which provides an environment for interactive computing with Python and offers several features and improvements over the standard Python shell.
It provides a tool called InteractiveShellEmbed that allows you to interact with a Python shell environment from within your script. Make sure you have installed the ipython package.
from IPython.terminal.embed import InteractiveShellEmbed
# Create an instance of InteractiveShellEmbed
shell = InteractiveShellEmbed()
# Run a shell command and capture its output
output = shell.getoutput("ls -l")
# Print the captured output
print(output)
6. Use shlex.split for Safer Command Execution
One way to enhance safety is by using the shlex.split function to parse and split command strings correctly. Executing shell commands with Python often involves passing strings to the subprocess module. If you construct these strings improperly, you might inadvertently expose your code to command injection vulnerabilities. Command injection occurs when an attacker manipulates the input to include malicious commands.
import subprocess
filename = "file.txt"
command = f"cat {filename}"
output = subprocess.check_output(command, shell=True)
If the filename variable contains a malicious value (e.g., file.txt; rm -rf /), the command injection can delete files from your system. To prevent this, you can use shlex.split.
shlex.split takes a command string and splits it into a list of arguments, respecting quotes and escaping. Here’s how to use it:
import subprocess
import shlex
filename = "file.txt"
command = f"cat {filename}"
args = shlex.split(command)
output = subprocess.check_output(args)
7. Summary and Conclusion
In this article, we have discussed different methods for capturing shell command output in Python. These methods provide different approaches for capturing command output in Python, Depending on your specific use case, you can choose the most suitable method. If you have any questions please feel free to ask in the comment section.
Happy Coding!