Send code from Vim to an external application for execution

Question

Using Stata my text editor is gVim. Using scripts from here and here to send code from Vim to Stata keeps me from switching to Linux. The scripts are in AutoIt I cannot use in Linux. They are independent of text editor; people who wrote them use Notepad++. These scripts with a few lines in my .vimrc file allow me to send selections or the whole file to a Stata window.

I am looking for this in Linux. There are Stata for command line and xstata is the GUI version. I use the GUI version so Screen and Tmux are ruled out. I wasn't able to find a plugin for Vim. Bash I want to look into. Python would be OK.

AutoIt script I need to translate I prefer doesn't overwrite content of clipboard. It checks for an open Stata window, selects or executes one, pastes contents to be executed into temporary file, switches to Stata window, selects command line with Ctrl + 1 (and anything already be written with Ctrl + A) then pastes "tempfile" into command line, which then executes the code. I have a solution in Bash.

; Declare variables
Global $ini, $statapath, $statawin, $statacmd, $dofile, $clippause, $winpause, $keypause

; File locations
; Path to INI file
$ini = @ScriptDir & "\rundo.ini"
;; contents of ini file are the following
    ;[Stata]
    ;; Path to Stata executable
    ;statapath = "C:\Program Files\Stata11\StataSE.exe"
    ;; Title of Stata window
    ;statawin = "Stata/SE 11.2"
    ;; Keyboard shortcut for Stata command window
    ;statacmd = "^1"
    ;[Delays]
    ;; Pause after copying of Stata commands to clipboard, in milliseconds
    ;; Use higher number if script fails (default: 100, recommended range: 0 - 200)
    ;clippause = 100
    ;; Pause between window-related operations, in milliseconds
    ;; Use lower number to speed up script, higher number if script fails (default: 200)
    ;winpause = 200
    ;; Pause between key strokes sent to Stata, in milliseconds
    ;; Use lower number to speed up script, higher number if script fails (default: 1)
    ;keypause = 1


; Path to Stata executable
$statapath = IniRead($ini, "Stata", "statapath", "C:\Program Files\Stata11\StataSE.exe")
; Title of Stata window
$statawin = IniRead($ini, "Stata", "statawin", "Stata/SE 11.2")

; Keyboard shortcut for Stata command window
$statacmd = IniRead($ini, "Stata", "statacmd", "^1")

; Path to do-file that is passed to AutoIt
; Edit line to match editor used, if necessary
$dofile = $CmdLine[1]

; Delays
; Pause after copying of Stata commands to clipboard
$clippause = IniRead($ini, "Delays", "clippause", "100")
; Pause between window-related operations
$winpause = IniRead($ini, "Delays", "winpause", "200")
; Pause between keystrokes sent to Stata
$keypause = IniRead($ini, "Delays", "keypause", "1")

; Set WinWaitDelay and SendKeyDelay to speed up or slow down script
Opt("WinWaitDelay", $winpause)
Opt("SendKeyDelay", $keypause)

; If more than one Stata window is open, the window that was most recently active will be matched
Opt("WinTitleMatchMode", 2)

; Check if Stata is already open, start Stata if not
If WinExists($statawin) Then
  WinActivate($statawin)
  WinWaitActive($statawin)
  ; Activate Stata command window and select text (if any)
  Send($statacmd)
  Send("^a")
  ; Run saved do-file
  ; Double quotes around $dofile needed in case path contains blanks
  ClipPut("do " & '"' & $dofile & '"')
  ; Pause avoids problem with clipboard, may be AutoIt or Windows bug
  Sleep($clippause)
  Send("^v" & "{Enter}")
Else
  Run($statapath)
  WinWaitActive($statawin)
  ; Activate Stata command window
  Send($statacmd)
  ; Run saved do-file
  ; Double quotes around $dofile needed in case path contains blanks
  ClipPut("do " & '"' & $dofile & '"')
  ; Pause avoids problem with clipboard, may be AutoIt or Windows bug
  Sleep($clippause)
  Send("^v" & "{Enter}")
EndIf

Steele

Have you (1) looked at fmwww.bc.edu/repec/bocode/t/texteditors.html#unix? (2) asked at statalist (stata.com/support/faqs/res/statalist.html)? I personally work in text editor and just type do the_whole_thing in Stata. If the_whole_thing takes too long to run, you need to break it down into smaller manageable pieces, which is the standard programming advice, and store the intermediate results, which is available for both data and estimation results in Stata. (See also stata.com/bookstore/wdaus.html) – Dissected

I know nothing about stata, however there are scripts out there that use screen/tmux to do this sort of things. Check out this one for an example: vim.org/scripts/script.php?script_id=2711 – Corncrib

@Randy: I guess the biggest problem is that you need to get inside a running executable file; the OP was not so much interested in the "batch" mode, sending to which does not represent any problem. (I am at the opposite side of the spectrum: I know nothing about vim, but I know quite a bit about Stata.) – Dissected

@Dissected with that script you can send input to an executable running within gnu-screen or tmux as if the user typed it. This wouldn't work if stata doesn't have a command line interface though. – Corncrib

@Randy: it does have a command line interface, but the OP indicated that he looked at tmux, and it did not work out. The executable may not be particularly receptive to stuff coming from outside. It was intended to be used in an interactive data analysis, but you can run scripts in it, as well, either in batch mode, or by requesting it to run a script from that interactive command line mode. This is a commerical software, so asking the developers to make it more receptive is probably out of question. – Dissected

I have done both (1) and (2). As said, I can use the screen.vim plugin in combination with tmux to send commands to the cli version of stata. but the cli version has definite disadvantages over the gui, e.g. not being able to output graphics. – Steele

@ilprincipe: the batch version can output graphics, it's just that you don't see it. You would need to graph save or graph export whatever you have produced, even though the graph window may not be actually created. (I understand that I am proposing kludges; I don't really know if a good solution exists, otherwise I would've offered it right away.) – Dissected

The first Autoit script does the following: (1) get the filename of the current text file (2) switch "windows" to Stata (or open stata if it's not running) and (3) paste "do <filename>" into Stata's command line. The second copies selected text into a temp file, saves the temp file's name in step (1). – Insalivate

P.S. I posted the code at code.google.com/p/notepad-stats-integration If you get a solution, please let me know and I'll post the Linux instructions on my site. Thanks. – Insalivate

It seems that the crux is figuring out how to get xstata to accept input from the outside. Everything else is trivial. Is there any automation available in X for finding windows and controls and filling them with data? Even better would be if xstata could be invoked directly from the command line to drive itself (i.e. to control the open window rather than spawning another). – Melchor

the windows script does it by creating a temp file that is called via do "filepath" in stata. an automation to find the window would be necessary. I guess the solution for windows is not the most elegant, but it works perfectly fine. essentially I only need to translate one autoit script. I have turned this script into an exe file and some lines in my vimrc enable me to run both single lines or the entire file when calling it. I have added the script's contents to my post. – Steele

Answer 1

IronAHK is a Linux/Mono rewrite of the AutoHotKey scripting language, which is similar to AutoIt (a GUI automation / keyboard remapping tool). I haven't used IronAHK, but AutoHotkey can run AutoIt v2 scripts.

You can also look @Project Sikuli: "Sikuli is a visual technology to automate and test graphical user interfaces (GUI) using images (screenshots). Sikuli includes Sikuli Script, a visual scripting API for Jython, and Sikuli IDE, an integrated development environment for writing visual scripts with screenshots easily" (from the sikuli front page)

Another good option is Linux Desktop Testing Project (LDTP), scriptable with Python:

example:

from ldtp import *
from ldtputils import *

try:
    launchapp("gedit")
    if waittillguiexist("*.gedit")==0:
        raise LdtpExecutionError("Gedit window does not exist")

    selectmenuitem("*-gedit", "mnuFile;mnuOpen")
    selectrow("dkgOpenFiles...", "tblFiles", fileName[0])
    ...

Dortheydorthy

I'm looking into it, but development of IronAHK seems to have slowed down somewhat recetnly. Running make already produces an error, and half of the links on their hp dont work. Also, the scripts are written AutoIT 3. I'm guessing a native linux solution might be easier. – Steele

@ilprincipe: Project Sikuli (sikuli.org) may also worth a look: "Sikuli is a visual technology to automate and test graphical user interfaces (GUI) using images (screenshots). Sikuli includes Sikuli Script, a visual scripting API for Jython, and Sikuli IDE, an integrated development environment for writing visual scripts with screenshots easily" – Dortheydorthy

LDTP seems exactly like the solution I was looking for. I'll probably accept this as an answer, however I have in the meantime stumbled across wmctrl and xte and written a somewhat clunky hack in bash. I'll add it to my post later. – Steele

Answer 2

Maybe you could use a mechanism similar to used by this vim plugin, which perform a similar task:

R.vim : Send R code from a VIM buffer to R

This plugin sends R code to a R tool, under unix and windows (R programming language) is widely used for statistical software development and data analysis).

I don't know about Stata or R language, but it seems that you could control Stata using R, as stated in Why use R?:

You can easily use it anywhere. It's platform-independent, so you can use it on any operating system. And it's free, so you can use it at any employer without having to persuade your boss to purchase a license.
:
:
R allows you to integrate with other languages (C/C++, Java, Python) and enables you to interact with many data sources: ODBC-compliant databases (Excel, Access) and other statistical packages (SAS, Stata, SPSS, Minitab).

Some Stata commands translated to R:

Stata or R

If you could perform the desired task through R then probably you could use the Vim plugin above unchanged.

Rivard

I know some R, and would prefer to switch to it completely, but for some projects with my coworkers I have to use stata, and I need the gui version. The plugin uses tmux as well. Sorry, but this will not do. Also, the plugin you linked to is not supported anymore. It is now maintained and developed by a different person. This is the preferred R plugin for vim: http://www.vim.org/scripts/script.php?script_id=2628. – Steele

That script depends on support from R for the communication, i.e. on Windows it uses OLE. The question is still "is there any way to send data to xstata from the outside?" If not, you'll be stuck looking for an automation tool for X. – Melchor

I guess it can be done in the same way the autoit script does it. It checks for an open stata window, selects it (or executes one), pastes the contents that are to be executed into a temporary file, switches to the stata window, selects the command line with ctrl-1 (and anything that might already be written there with ctrl-a) and then pastes do "tempfile" into the commandline, which then executes the code that was sent. At least thats how I understand it. – Steele

so i suspect it would have to be xautomation or something similar? – Steele

Answer 3

I've used the VI map function to define macros to send my file to a C compiler, and retrieve the results. It's not very robust (no if/then programming), but it's pretty simple to implement, and I have a lot of standard mappings I use. For example &T uppercases the line I'm on while &t lowercases it. I use &S to run my spell checker (gspell), etc. You don't need to begin your macros with an ampersand, but this way I know it's an unlikely combination of letters I'd be typing.

Setting up a Map is pretty easy. You use the :map ex command, space, a word used to invoke the map, a space, and then the keystrokes you want to execute. If you need to insert something like a return or escape, prefix it with a Ctrl-V.

You can use map! to map a macro that can be executed while in Insert or Replace mode.

Uprising

I know how to map keys. I am looking for help on how to pipe code from vim or basically any text editor to a running session of another program, in this case xstata. – Steele