How do I serialize a variable in VimScript?

Question

I wish to save a random Vim dictionnary, let's say:

let dico = {'a' : [[1,2], [3]], 'b' : {'in': "str", 'out' : 51}}

to a file. Is there a clever way to do this? Something I could use like:

call SaveVariable(dico, "safe.vimData")
let recover = ReadVariable("safe.vimData")

Or should I build something myself with only textfiles?

Vanir

Answer 1

Thanks to VanLaser (cheers), I've been able to implement these functions using string, writefile and readfile. This is not binary serialization but it works well :)

function! SaveVariable(var, file)
    " turn the var to a string that vimscript understands
    let serialized = string(a:var)
    " dump this string to a file
    call writefile([serialized], a:file)
endfun

function! ReadVariable(file)
    " retrieve string from the file
    let serialized = readfile(a:file)[0]
    " turn it back to a vimscript variable
    execute "let result = " . serialized
    return result
endfun

Use them this way:

call SaveVariable(anyvar, "safe.vimData")
let restore = ReadVariable("safe.vimData")

Enjoy!

Vanir

I would simply write a bunch of string() formatted variables to the same file, then source the entire file (all variables) in one go. Perhaps add an "append" flag? But you know your requirements best - I think you prefer to collect everything in a single, to-be-serialized, dictionary. – Mirnamirror

@Mirnamirror I didn't for a reason: when I source the file from inside a function, the variable scoping is not what I expect (looks like they are declared outside and can't be reached from the function itself), which is annoying.. Do you know a fine way to control scoping when soing? – Vanir

Thanks for answer. Actually saving/loading a prepared dictionary (can contain anything) is pretty neat. – Mirnamirror

@Mirnamirror Is it not? :) – Vanir

Answer 2

You can put to good use the :string() function. Test these:

let g:dico = {'a' : [[1,2], [3]], 'b' : {'in': "str", 'out' : 51}}
let str_dico = 'let g:dico_copy = ' . string(dico)
echo str_dico
execute str_dico
echo g:dico_copy

... so you can save the str_dico string as a line of a vimscript file (e.g. using writefile()), and then source the vim file directly.

Mirnamirror

Fair enough. I'll use this to implement Save and Read myself then. Thank you! I'll be back to post them here :) – Vanir

You're welcome! I think this is the best job split, interest-wise :) – Mirnamirror

Answer 3

Vim has built-in json serialization:

let dico = { "a" : 10, "b" : 20, }
call writefile(json_encode(dico)->split(), 'saved.json')
let recover = json_decode(readfile('saved.json')->join())

Using a data-only format for storing your dictionary lets you avoid code injection.

To support old versions of vim, see this answer.

Soapberry

Answer 4

I used @iago-lito's answer in a script I wrote a few years ago. Yesterday I spent some time improving on it. The vim dictionary is very similar to a JSON object, but:

1. when I open the file and set filetype=json, the linter complains about the single quotes around the strings, and
2. the JSON formatter splits the text into multiple lines, and indents them to make a pretty file. As a result, reading only the 0'th line of text doesn't give a complete dictionary object.

Here are my modifications to fix both issues.

function! SaveVariable(var, file)
    " Change all single quotes to double quotes.
    " {'x':'O''Toole','y':['A',2,'d']} -> {"x":"O""Toole","y":["A",2,"d"]}
    let serialized = substitute(string(a:var),"'",'"','g')
    " Change all escaped double quotes back to apostrophes.
    " {"x":"O""Toole","y":["A",2,"d"]} -> {"x":"O'Toole","y":["A",2,"d"]}
    let serialized = substitute(serialized,'""', "'",'g')
    call writefile([serialized], a:file)
endfunction

function! ReadVariable(file)
    execute 'let result = [' . join(readfile(a:file),'') . ']'
    return result[0]
endfunction

This seems to work well for all kinds of data. I tested it with an object, a list, and number and string scalar values.

STRANGER, DANGER!

Here is a word of warning that goes along with this and any other dynamically-generated code. Both @iago-lito's and my solution are vulnerable to code injection, and if you are reading files that are out of your control, bad things can happen to your machine. For example, if someone sneaks this into the file:

42|call system('rmdir /s /q c:\')|call system('rm -rf /')

calling @iago-lito's ReadVariable() will return 42, but your computer will be toast, whether it's a Windows, Mac, or Linux machine. My version also fails, albeit with a more complex version of the statement:

42]|call system('rmdir /s /q c:\')|call system('rm -rf /')|let x=[

A proper solution would be to parse the text, looking for the end of the actual data, and dumping everything after it. This means you lose the simplicity of this approach. Vim and Neovim have come a long way in recent years. lua and python are, from what I've read, easier than ever to integrate into vimscript. I wouldn't be surprised if either of those languages has a built-in answer to this question.

Fraenum

Even Vimscript now has a built-in answer to this question: json_encode. – Soapberry

Even better. Thanks for pointing that out, @idbrii. Don't know how I missed that one. Do you know when it was added? – Fraenum

Looks like v7.4.1304. Older than I thought! – Soapberry