[slimpl] some comments about vim internals

[Jim Bailey]

Hello everyone,

I have been having a look at VIM's internals to see if there are any really good ways for handling async communications to swank that havn't been discussed before.

The bad news is that VIM generally relies on keyboard input to trigger anything. GVIM uses the appropriate event loop the toolkit provides and these do tend to come with io watchers, which is how netbeans provides async input on a toolkit by toolkit basis. I don't really like this solution as it is relatively painful to maintain and port and does not work on a terminal.

The Python extension gives access to Python's threads, and thus async operation on VIM's internals. Clearly this could lead to threading issues (VIM's internals are not thread safe, it is single threaded throughout) which is why if_python.c (the Python extension) uses Python_Lock_Vim and Python_Unlock_Vim at lots of appropriate places - unfortunately both those functions are empty, suggesting that this problem is yet to be solved and multi-threaded Python scripts should not update VIM.

Time for some good news: Adding ECL as another VIM extension language is very easy. I can currently evaluate forms from the ex prompt (e.g. :ecl (print "test")<cr>), *standard-output* and *error-output* are redirected via VIM for sane results on the terminal and guis, and ECL can call ex commands (e.g. (vim:command "w") to write the current buffer). I will also add access to the windows list, buffers list, and add getting/setting of a buffer's contents. So, if it helps, we can have a common-lisp client. This might be an easier target for the swank.el port than existing VIM languages, and would also provide a good platform for implementing a good indentation engine (maybe just a port of cl-indent.el).

As for async events, it is possible to use (copy) the netbeans code to add socket based events for those UIs that are supported (gtk/motif/win32 out of gtk/gtk2/gnome/gnome2/motif/athena/neXtaw/beos/photon/carbon/win32), and also extend the terminal key handling to be able to listen to sockets. The terminal already uses select()/poll() to get keystrokes, and already has one extension (SNiFF) hard coded into that to handle async input from a stream. If this was made extensible, with an abstraction layer on the toolkit/terminal API, then it could possibly make its way into VIM proper, as it would tidy up the SNiFF stuff, solve the Python threads issue (using an internal socket-pair), extend the netbeans support to the terminal, etc.

If the async problem was solved in VIM's core, then any embedded language interface that was kind enough to pass on the functionality would make an excellent base to build a SWANK client. The ECL interface would just be the first.

Would it be useful? Would it be worth it? Is requiring a particular extension language reasonable? Does anyone mind compiling their own VIM until the async stuff goes into VIM proper (wich is possible) and the ECL interface goes in too (less likely), just to be able to have something similar to SLIME in VIM?

Jim