I am having hard time parsing the arguments to subprocess.Popen. I am trying to execute a script on my Unix server. The script syntax when running on shell prompt is as follows:
/usr/local/bin/script hostname = <hostname> -p LONGLIST. No matter how I try, the script is not running inside subprocess.Popen
The space before and after “=” is mandatory.
import subprocess Out = subprocess.Popen(['/usr/local/bin/script', 'hostname = ', 'actual server name', '-p', 'LONGLIST'],shell=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
The above does not work.
And when I use shell=False, I get
OSError: [Errno 8] Exec format error
OSError: [Errno 8] Exec format error can happen if there is no shebang line at the top of the shell script and you are trying to execute the script directly. Here’s an example that reproduces the issue:
>>> with open('a','w') as f: f.write('exit 0') # create the script ... >>> import os >>> os.chmod('a', 0b111101101) # rwxr-xr-x make it executable >>> os.execl('./a', './a') # execute it Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python2.7/os.py", line 312, in execl execv(file, args) OSError: [Errno 8] Exec format error
To fix it, just add the shebang e.g., if it is a shell script; prepend
#!/bin/sh at the top of your script:
>>> with open('a','w') as f: f.write('#!/bin/shnexit 0') ... >>> os.execl('./a', './a')
exit 0 without any errors.
On POSIX systems, shell parses the command line i.e., your script won’t see spaces around
= e.g., if
#!/usr/bin/env python import sys print(sys.argv)
then running it in the shell:
$ /usr/local/bin/script hostname = '<hostname>' -p LONGLIST
['/usr/local/bin/script', 'hostname', '=', '<hostname>', '-p', 'LONGLIST']
Note: no spaces around
'='. I’ve added quotes around
<hostname> to escape the redirection metacharacters
To emulate the shell command in Python, run:
from subprocess import check_call cmd = ['/usr/local/bin/script', 'hostname', '=', '<hostname>', '-p', 'LONGLIST'] check_call(cmd)
shell=True. And you don’t need to escape
<> because no shell is run.
"Exec format error" might indicate that your
script has invalid format, run:
$ file /usr/local/bin/script
to find out what it is. Compare the architecture with the output of:
$ uname -m
I will hijack this thread to point out that this error may also happen when target of Popen is not executable. Learnt it hard way when by accident I have had override a perfectly executable binary file with zip file.
It wouldn’t be wrong to mention that
Pexpect does throw a similar error
#python -c "import pexpect; p=pexpect.spawn('/usr/local/ssl/bin/openssl_1.1.0f version'); p.interact()" Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib/python2.7/site-packages/pexpect.py", line 430, in __init__ self._spawn (command, args) File "/usr/lib/python2.7/site-packages/pexpect.py", line 560, in _spawn os.execv(self.command, self.args) OSError: [Errno 8] Exec format error
Over here, the
openssl_1.1.0f file at the specified path has
exec command specified in it and is running the actual openssl binary when called.
Usually, I wouldn’t mention this unless I have the root cause, but this problem was not there earlier. Unable to find the similar problem, the closest explanation to make it work is the same as the one provided by @jfs above.
what worked for me is both
/bin/bashat the beginning of the command or file you are
facing the problem with, or
- adding shebang
#!/bin/shas the first line.
#python -c "import pexpect; p=pexpect.spawn('/bin/bash /usr/local/ssl/bin/openssl_1.1.0f version'); p.interact()" OpenSSL 1.1.0f 25 May 2017
If you think the space before and after “=” is mandatory, try it as separate item in the list.
Out = subprocess.Popen(['/usr/local/bin/script', 'hostname', '=', 'actual server name', '-p', 'LONGLIST'],shell=True,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
Have you tried this?
Out = subprocess.Popen('/usr/local/bin/script hostname = actual_server_name -p LONGLIST'.split(), shell=False,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
Edited per the apt comment from @J.F.Sebastian