Self contained reproducible example
I need to strace the output of any command through python. But I only need to translate the following amd linux command to mac m1/arm commands (so python is likely irrelevant):
strace -e trace=execve -v -s 100000000 -xx -ttt -ff -o output.txt sh -c 'echo hi'
How do I do that?
This fails for me:
❯ sudo dtruss -t execve -f sh -c 'echo hi'
dtrace: system integrity protection is on, some features will not be available
dtrace: failed to execute sh: Operation not permitted
Note:
- I have complete control of the input so I can do sudo and related commands in my mac (it's mainly to debug my code, so it works on pycharm)
Strace from brew fails
I can't seem to install strace from brew:
❯ brew install strace
Running `brew update --auto-update`...
strace: Linux is required for this software.
[email protected]: Linux is required for this software.
Error: strace: Unsatisfied requirements failed this build.
Execsnoop also failed
❯ sudo execsnoop sh -c 'echo hi'
dtrace: system integrity protection is on, some features will not be available
dtrace: invalid probe specifier
/*
* Command line arguments
*/
inline int OPT_dump = 0;
inline int OPT_cmd = 0;
inline int OPT_time = 0;
inline int OPT_timestr = 0;
inline int OPT_zone = 0;
inline int OPT_safe = 0;
inline int OPT_proj = 0;
inline int FILTER = 0;
inline string COMMAND = ".";
#pragma D option quiet
#pragma D option switchrate=10hz
/*
* Print header
*/
dtrace:::BEGIN
{
/* print optional headers */
OPT_time ? printf("%-14s ", "TIME") : 1;
OPT_timestr ? printf("%-20s ", "STRTIME") : 1;
OPT_zone ? printf("%-10s ", "ZONE") : 1;
OPT_proj ? printf("%5s ", "PROJ") : 1;
/* print main headers */
/* APPLE: Removed "ZONE" header, it has no meaning in darwin */
OPT_dump ? printf("%s %s %s %s %s %s %s\n",
"TIME", "PROJ", "UID", "PID", "PPID", "COMM", "ARGS") :
printf("%5s %6s %6s %s\n", "UID", "PID", "PPID", "ARGS");
}
/*
* Print exec event
*/
/* SOLARIS: syscall::exec:return, syscall::exece:return */
proc:::exec-success
/(FILTER == 0) || (OPT_cmd == 1 && COMMAND == strstr(COMMAND, execname)) || (OPT_cmd == 1 && execname == strstr(execname, COMMAND))/
{
/* print optional fields */
OPT_time ? printf("%-14d ", timestamp/1000) : 1;
OPT_timestr ? printf("%-20Y ", walltimestamp) : 1;
OPT_zone ? printf("%-10s ", zonename) : 1;
OPT_proj ? printf("%5d ", curpsinfo->pr_projid) : 1;
/* print main data */
/* APPLE: Removed the zonename output, it has no meaning in darwin */
OPT_dump ? printf("%d %d %d %d %d %s ", timestamp/1000,
curpsinfo->pr_projid, uid, pid, ppid, execname) :
printf("%5d %6d %6d ", uid, pid, ppid);
OPT_safe ? printf("%S\n", curpsinfo->pr_psargs) :
printf("%s\n", curpsinfo->pr_psargs);
}
: probe description proc:::exec-success does not match any probes. System Integrity Protection is on
I inherited this code and in it calls strace from within python. In particular it calls:
def strace_build(executable: str,
regex: str,
workdir: Optional[str],
command: List[str],
strace_logdir=None) -> List[str]:
''' trace calls of executable during access to files that match regex
in workdir while executing the command and returns the list of pycoq_context
file names
In the simplest case strace runs the specified command until it
exits. It intercepts and records the system calls which are
called by a process and the signals which are received by a
process. The name of each system call, its arguments and its
return value are printed on standard error or to the file
specified with the -o option.
https://mcmap.net/q/1297671/-how-to-call-an-equivalent-command-to-strace-on-mac-ideally-from-python
'''
print('---- Calling strace_build ----')
def _strace_build(executable, regex, workdir, command, logdir):
logfname = os.path.join(logdir, 'strace.log')
logging.info(f"pycoq: tracing {executable} accesing {regex} while "
f"executing {command} from {workdir} with "
f"curdir {os.getcwd()}")
print(f"pycoq: tracing {executable} accesing {regex} while "
f"executing {command} from {workdir} with "
f"curdir {os.getcwd()}")
with subprocess.Popen(['strace', '-e', 'trace=execve',
'-v', '-ff', '-s', '100000000',
'-xx', '-ttt',
'-o', logfname] + command,
cwd=workdir,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE) as proc:
for line in iter(proc.stdout.readline, ''):
logging.debug(f"strace stdout: {line}")
print(f"strace stdout: {line=}")
logging.info(f"strace stderr: {proc.stderr.read()}"
"waiting strace to finish...")
proc.wait()
logging.info('strace finished')
res: list[str] = parse_strace_logdir(logdir, executable, regex)
print('---- Done with strace_build ----')
return res
if strace_logdir is None:
with tempfile.TemporaryDirectory() as _logdir:
return _strace_build(executable, regex, workdir, command, _logdir)
else:
os.makedirs(strace_logdir, exist_ok=True)
strace_logdir_cur = tempfile.mkdtemp(dir=strace_logdir)
return _strace_build(executable, regex, workdir, command, strace_logdir_cur)
but because it calls strace it only works on linux. I want it to work on my mac -- ideally if possible in the most pythonic way possible. I believe what it does is strace a terminal command that is called from within python.
What would be an equivalent way to call this command on mac using the same flags so that it works (ideally exactly) the same?
Not sure if this matters but I am using an m1 mac.
some output when the above function is used:
--done with make attempt--
---- Calling strace_build ----
pycoq: tracing /home/bot/.opam/ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1/bin/coqc accesing .*\.v$ while executing ['opam', 'reinstall', '--yes', '--switch', 'ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1', '--keep-build-dir', 'debug_proj'] from None with curdir /home/bot
strace stdout: line='\n'
strace stdout: line='<><> Synchronising pinned packages ><><><><><><><><><><><><><><><><><><><><><><>\n'
strace stdout: line='[debug_proj.~dev] no changes from file:///home/bot/iit-term-synthesis/coq_projects/debug_proj\n'
strace stdout: line='\n'
strace stdout: line='debug_proj is not installed. Install it? [Y/n] y\n'
strace stdout: line='Sorry, no solution found: there seems to be a problem with your request.\n'
strace stdout: line='\n'
strace stdout: line='No solution found, exiting\n'
---- Done with strace_build ----
...
---- Calling strace_build ----
pycoq: tracing /home/bot/.opam/ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1/bin/coqc accesing .*\.v$ while executing ['make', '-C', '/home/bot/iit-term-synthesis/coq_projects/debug_proj'] from None with curdir /home/bot
strace stdout: line="make: Entering directory '/home/bot/iit-term-synthesis/coq_projects/debug_proj'\n"
strace stdout: line='coq_makefile -f _CoqProject -o CoqMakefile\n'
strace stdout: line='make --no-print-directory -f CoqMakefile \n'
strace stdout: line='COQDEP VFILES\n'
strace stdout: line='COQC debug_0_plus_n_eq_n.v\n'
strace stdout: line='COQC debug1_n_plus_1_greater_than_n.v\n'
strace stdout: line='COQC debug2_n_plus_0_eq_n.v\n'
strace stdout: line="make: Leaving directory '/home/bot/iit-term-synthesis/coq_projects/debug_proj'\n"
---- Done with strace_build ----
def strace_build_mac_m1(executable: str,
regex: str,
workdir: Optional[str],
command: List[str],
strace_logdir=None) -> List[str]:
''' trace calls of executable during access to files that match regex
in workdir while executing the command and returns the list of pycoq_context
file names
In the simplest case strace runs the specified command until it
exits. It intercepts and records the system calls which are
called by a process and the signals which are received by a
process. The name of each system call, its arguments and its
return value are printed on standard error or to the file
specified with the -o option.
https://mcmap.net/q/1297671/-how-to-call-an-equivalent-command-to-strace-on-mac-ideally-from-python
plan:
- get the command we are running
- pip push my pycoq with no name changes so code doesn't break
- pull the rest of the repos needed, I don't think anything else since lf is here
- harcode test
- actually, look at commands...we need to provide for reproducibility a way to install opam and all this stuff
without docker but in the mac since we are trying to do a mac install. Argh...
COMMANDS:
pycoq: tracing /home/bot/.opam/ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1/bin/coqc accesing .*\.v$ while executing ['opam', 'reinstall', '--yes', '--switch', 'ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1', '--keep-build-dir', 'lf'] from None with curdir /home/bot
executable='/home/bot/.opam/ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1/bin/coqc'
regex='.*\\.v$'
workdir=None
command=['opam', 'reinstall', '--yes', '--switch', 'ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1', '--keep-build-dir', 'lf']
curdir: os.getcwd()='/home/bot'
'''
print('---- Calling strace_build_mac_m1 ----')
def _strace_build(executable, regex, workdir, command, logdir):
logfname = os.path.join(logdir, 'strace.log')
logging.info(f"pycoq: tracing {executable} accesing {regex} while "
f"executing {command} from {workdir} with "
f"curdir {os.getcwd()}")
print(f"pycoq: tracing {executable} accesing {regex} while "
f"executing {command} from {workdir} with "
f"curdir {os.getcwd()}")
print(f'{executable=}')
print(f'{regex=}')
print(f'{workdir=}')
print(f'{command=}')
print(f'curdir: {os.getcwd()=}')
with subprocess.Popen(['dtruss', '-e', 'trace=execve',
'-v', '-ff', '-s', '100000000',
'-xx', '-ttt',
'-o', logfname] + command,
cwd=workdir,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE) as proc:
for line in iter(proc.stdout.readline, ''):
logging.debug(f"strace stdout: {line}")
print(f"strace stdout: {line=}")
logging.info(f"strace stderr: {proc.stderr.read()}"
"waiting strace to finish...")
proc.wait()
logging.info('strace finished')
res: list[str] = parse_strace_logdir(logdir, executable, regex)
print('---- Done with strace_build_mac_m1 ----')
return res
if strace_logdir is None:
with tempfile.TemporaryDirectory() as _logdir:
return _strace_build(executable, regex, workdir, command, _logdir)
else:
os.makedirs(strace_logdir, exist_ok=True)
strace_logdir_cur = tempfile.mkdtemp(dir=strace_logdir)
return _strace_build(executable, regex, workdir, command, strace_logdir_cur)
# -
def code_for_mac_m1():
coq_package = 'lf'
coq_package_pin = '~/pycoq/pycoq/test/lf'
coq_package_pin = os.path.expanduser(coq_package_pin)
print(f'coq_package: {coq_package=}')
print(f'coq_package_pin: {coq_package_pin=}')
### pycoq: tracing /home/bot/.opam/ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1/bin/coqc accesing .*\.v$ while executing ['opam', 'reinstall', '--yes', '--switch', 'ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1', '--keep-build-dir', 'lf'] from None with curdir /home/bot
# executable='/home/bot/.opam/ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1/bin/coqc'
# regex='.*\\.v$'
# workdir=None
# command=['opam', 'reinstall', '--yes', '--switch', 'ocaml-variants.4.07.1+flambda_coq-serapi.8.11.0+0.11.1', '--keep-build-dir', 'lf']
# curdir: os.getcwd()='/home/bot'
# - get the filename in split
# path2filenames: list[str] = pycoq.opam.opam_strace_build(coq_proj, coq_proj_pin)
path2filenames_raw: list[str] = strace_build_mac_m1()
path2filenames_raw.sort()
print(f'\n====----> Populate coq pkg/proj data with files: {path2filenames_raw=}')
if __name__ == '__main__':
code_for_mac_m1()
print('Done!\n\a')
why doesn't this meet the SO guidelines? I am not asking for a software request clearly. From the discussion it's much harder to run in python + having the mac setup as needed + the tool with the equivalent flags to strace for mac.
google for replacement (analogue) of functionality of strace for Mac OS and modify the parser that parses output of strace for that “mac-strace” The strace functionality is used to inspect the Coq building system to record all options and arguments and environment for coqc in which each individual .v file has be processed by coqc during the build process. The Coq build system is complicated, and i didn’t parse but resolved to just observing by strace of what the actual Coq does and strace simply records all options and arguments (like -R etc) of what the actual Coq does so that pycoq could call coqc with exactly the same options and arguments
Using modified dtruss.sh
Answer by philipe didn't work:
(meta_learning) brandomiranda~ ❯ cat dtruss.sh
#!/bin/bash
file=$(type -P $1); shift
c=/tmp/$(basename "$file")
cp "$file" "$c"
codesign --remove-signature "$c"
sudo dtruss "$c" "$@"
(meta_learning) brandomiranda~ ❯ chmod +x dtruss.sh
(meta_learning) brandomiranda~ ❯ ./dtruss.sh echo hi
dtrace: system integrity protection is on, some features will not be available
dtrace: failed to execute /tmp/echo: Could not create symbolicator for task
To fix sudo to not prompt for password see:
# root and users in group wheel can run anything on any machine as any user
root ALL = (ALL) ALL
%admin ALL = (ALL) NOPASSWD: ALL
#%admin ALL = (ALL) ALL
Related:
- Is it possible to run dtruss without doing the odd things suggested here?
- Cross, unix reddit.
- Cross, apple devs reddit.
dtruss, but it hasn't been well-maintained for a few releases now, and won't trace vendor-provided binaries used without modification unless you weaken your system's security settings. – Threefoldstraceexternally from your favourite package manager, saybrewas in here? You already have root privilege, don't you? This waystracewill be called normally in MacOS with full compatibility. – HortDTracethat is found on macos but not linux or windows – SongsterDTraceis only for D-language, but there are ways to use it with python. IfDTracehappens to answer your QNS, I can form an answer – Songster