How to setup YouCompleteMe for kernel and device driver development?

Question

I would like to setup vim for kernel hacking so I have installed YouCompleteMe for auto completion. However, no matter what I do It looks like I can't configure it properly. It does not do semantic completion properly; It only suggests the semantics already used in the current file not even the ones in the headers or other translation units and if that's impossible then ycm is pretty useless. Does anyone know how to do it for the particular purpose? If it's necessary to include my .vimrc or .ycm_extra_conf.py please ask in the comments. Also if I need extra tools please specify them so I can set them up as well.

Hypotrachelium

Note: There is a dedicated site for vi & vim. – Maelstrom

@Maelstrom I'm aware of that dedicated site but this is related to programming not just vim. – Hypotrachelium

I just let you know, as there you would probably get more expert answers. – Maelstrom

Answer 1

Here is how I setup my kernel programming environment with vim.

Tools used:

• OmniCompletion: integrated feature of vim (no extra plugins required) for auto-completion
• ctags: creates code index database, which is needed for OmniCompletion (and some other vim plugins)
• cscope: tool for navigating the code (e.g. you can immediately jump to the function definition under cursor, etc.). Creates it's own code index database (different than ctags).

1. Install tools

First, install vim, ctags and cscope:

$ sudo aptitude install vim vim-gtk cscope exuberant-ctags

2. Create index database

Now create cscope and ctags databases. In your kernel sources root directory run next script (I used this tutorial as reference):

#!/bin/bash

list_sources() {
    echo "---> Listing sources..."

    find .                                   \
        -path "./arch*" -prune -o            \
        -path "./tmp*" -prune -o             \
        -path "./Documentation*" -prune -o   \
        -path "./scripts*" -prune -o         \
        -type f -name "*.[chsS]" -print >cscope.files

    find arch/arm/include/                   \
        arch/arm/kernel/                     \
        arch/arm/common/                     \
        arch/arm/boot/                       \
        arch/arm/lib/                        \
        arch/arm/mm/                         \
        arch/arm/mach-omap2/                 \
        arch/arm/plat-omap/                  \
        -type f -name "*.[chsS]" -print >>cscope.files
}

create_cscope_db() {
    echo "---> Creating cscope DB..."
    cscope -k -b -q
}

create_ctags_db() {
    echo "---> Creating CTags DB..."
    ctags -L cscope.files
}

cleanup() {
    echo "---> Removing garbage..."
    rm -f cscope.files
}

list_sources
create_cscope_db
create_ctags_db
cleanup

This script intended for working with ARM architecture code (with OMAP platform). If you need to work with another architecture (e.g. with x86), you need to change find lines in script appropriately.

As you can see from script above, it does next:

1. Create list of files to be indexed (cscope.files), using find command.

2. Create cscope database (using cscope.files file by default) for kernel (see -k option in man cscope), using next command:

   $ cscope -b -q -k
   

3. Create ctags file (tags), using cscope.files file (see man ctags):

   $ ctags -L cscope.files
   

4. Remove cscope.files file, as it's not needed anymore.

3. Configure vim for cscope

Download cscope_maps.vim file to your ~/.vim/plugin directory (see this link for details).

Now you can navigate your kernel code in vim using stuff like Ctrl+\, g, Ctrl+\, s, etc. See this for details. Also :help cscope-find command in vim can be helpful.

4. Configure vim for OmniCompletion

To use OmniCompletion you will need to edit your ~/.vimrc file. For example here is my OmniCompletion configuration (add these lines to your ~/.vimrc):

"-------------------------------------------------------------------------------
" OmniCppCompletion plugin
"-------------------------------------------------------------------------------

" Enable OmniCompletion
" http://vim.wikia.com/wiki/Omni_completion
filetype plugin on
set omnifunc=syntaxcomplete#Complete

" Configure menu behavior
" http://vim.wikia.com/wiki/VimTip1386
set completeopt=longest,menuone
inoremap <expr> <CR> pumvisible() ? "\<C-y>" : "\<C-g>u\<CR>"
inoremap <expr> <C-n> pumvisible() ? '<C-n>' :
  \ '<C-n><C-r>=pumvisible() ? "\<lt>Down>" : ""<CR>'
inoremap <expr> <M-,> pumvisible() ? '<C-n>' :
  \ '<C-x><C-o><C-n><C-p><C-r>=pumvisible() ? "\<lt>Down>" : ""<CR>'

" Use Ctrl+Space for omni-completion
" https://mcmap.net/q/427171/-ctrl-space-for-omni-and-keyword-completion-in-vim
inoremap <expr> <C-Space> pumvisible() \|\| &omnifunc == '' ?
  \ "\<lt>C-n>" :
  \ "\<lt>C-x>\<lt>C-o><c-r>=pumvisible() ?" .
  \ "\"\\<lt>c-n>\\<lt>c-p>\\<lt>c-n>\" :" .
  \ "\" \\<lt>bs>\\<lt>C-n>\"\<CR>"
imap <C-@> <C-Space>

" Popup menu hightLight Group
highlight Pmenu ctermbg=13 guibg=LightGray
highlight PmenuSel ctermbg=7 guibg=DarkBlue guifg=White
highlight PmenuSbar ctermbg=7 guibg=DarkGray
highlight PmenuThumb guibg=Black

" enable global scope search
let OmniCpp_GlobalScopeSearch = 1
" show function parameters
let OmniCpp_ShowPrototypeInAbbr = 1
" show access information in pop-up menu
let OmniCpp_ShowAccess = 1
" auto complete after '.'
let OmniCpp_MayCompleteDot = 1
" auto complete after '->'
let OmniCpp_MayCompleteArrow = 1
" auto complete after '::'
let OmniCpp_MayCompleteScope = 0
" don't select first item in pop-up menu
let OmniCpp_SelectFirstItem = 0

OmniCompletion will use your ctags file (tags). You can use completion now using Ctrl+Space (added by vim config above).

5. Some extra stuff

Check out also next plugins:

• NERDTree
• Tagbar
• linuxsty
• vim-airline
• vim-pathogen

Krucik

Also see this answer for more detailed overview. – Krucik

Answer 2

Forgive me for waking this long dead post from its eternal rest, but I've been fighting this same problem and the accepted answer does not answer the question. The question was setting up Vim with the YouCompleteMe plugin for kernel and device driver development. And while the accepted answer is helpful, it sent me on a wrong path because I wanted to make YouCompleteMe work.

This is how I solved it

I'll outline below how I've solved it in case others come across this and hopefully it helps them. I am not an expert in any of these technologies so it may not be the best way, but I've failed to find any other solution to the problem.

Tools

1. Install Vim
2. Install YouCompleteMe
3. Install BEAR

The code

I've followed along with Linux Device Drivers, Third Edition and I have no doubt that many stumbling on this question will likely be trying the same. In chapter 2, the author shows a simple "Hello, world!" for device drivers, which I'll copy below:

// hello.c
#include <linux/init.h>
#include <linux/module.h>

MODULE_LICENSE("Dual BSD/GPL");

static int hello_init(void){
    printk(KERN_ALERT "Hello, world\n");
    return 0;
}

static void hello_exit(void){
    printk(KERN_ALERT "Goodbye, cruel world\n");
}

module_init(hello_init);
module_exit(hello_exit);

The problem

Opening this in Vim will probably get YCM to complain about #include <linux/init.h>.

Get the linux headers

Apparently kernel compilation is a bit tougher than your average gcc hello.c and you'll need to have the linux header files on your machine. I use arch (btw) so I got them using pacman -S linux-headers but other distros should have similar ways of getting them using the package managers. If all else fails, you can download the Linux kernel source, including header files from their Website. If you installed using a package manager, you should have the headers in your /lib/modules/$(uname -r)/build/include directory. If you've installed the kernel source, the headers are in /path/to/source/include/.

Get a Makefile

Additionally you need a Makefile that can build your hello.c and link together all dependencies. For this I've used the Makefile for the book. Note however, that this Makefile points to /lib/modules/$(uname -r)/build so you'll have to change this if you're building using the kernel source. Additionally the Makefile will try to build a bunch of other examples so you can just remove these. obj-m := hello.o should be all that remains for this example.

Test Makefile

If you run make at this point, it should compile and you can insmod and rmmod the newly created hello.ko module. However your Vim will still complain.

Create the compilation database

In other to make this go away, first run make clean because BEAR demands a clean directory. Then run bear make and it should generate a 'compile_commands.json' for you. This is what the YCM guide was referring to here.

The finish line

Now if you reopen hello.c in vim, it should not complain anymore about init.h and everything should work as expected. You might run into this issue though for a bunch of YCM complaints which I've solved by adding the following to my .vimrc file:

let g:ycm_filter_diagnostics = {                                      
  \ "c": {                                                            
  \   "regex": [                                                      
  \     "-mno-fp-ret-in-387",                                         
  \     "-mpreferred-stack-boundary=3",                               
  \     "-mskip-rax-setup",                                           
  \     "-mindirect-branch=thunk-extern",                             
  \     "-mindirect-branch-register",                                 
  \     "-fno-allow-store-data-races",                                
  \     "-fplugin-arg-structleak_plugin-byref-all",                   
  \     "-fno-var-tracking-assignments",                              
  \     "-fconserve-stack",                                           
  \     "-mrecord-mcount"                                             
  \   ]                                                               
  \ }                                                                 
  \}       

It will take someone more qualified than me to explain why these have to be filtered out. I was content with the explanation in the github issue I linked above. But in any case, now you should have no more errors or warnings in Vim, with YouCompleteMe offering its full range of services for kernel and device driver development.

I hope this answer finds and helps anyone still struggling with this problem!

Alleviate

This could be a useful link for somebody. As regards me, I still get some error from clangd about unknown options such as -mpreferred-stack-boundary=3. Did you get anything like this? You last section seems to have to do with that, but it doesn't solve the problem for me... – Whitewood

Answer 3

You want cscope. See here for setup instructions:

http://cscope.sourceforge.net/large_projects.html

The kernel is pretty large and complex. Cscope performs pretty well for searching it in reasonable amounts of time.

It is still a hard problem, though, because the kernel makes heavy use of C macros in a way that isn't always easy for tools to follow. Tracing through kernel code can be a bit of an art, and takes practice.

There are more tips for a few other tools here:

http://kernelnewbies.org/FAQ/CodeBrowsing

Ats

And how exactly cscope helps does auto completion? Maybe you mean ctags + OmniCompletion? – Krucik

The secaond link mostly centers around ctags. Code completion isn't the time consuming part, though -- it's actually searching and finding the function -- he's talking about generally hacking on kernel code. Savinng keystrokes typing function names isn't the main thing -- figuring out functions and signitures is. +1 for mentioning omnicomplete, though, that IS an excellent tool to add. – Ats

@BarryGackle YouCompleteMe sucks. It's actually next to impossible to configure it for large and complex projects. I have installed cscope and it's excellent but aren't the cscope and ctags the same thing? Do I need them both? I'm upvoting your answer for the second link. I'll take a closer look when I can and probably accept the answer if it resolves my issue. – Hypotrachelium

They do similar things, but many people use both. Ctags is often the better tool for finding definitions, and it supports a wider range of languages. Cscope is better for finding all the places a function was actually called (awesome for refactoring). Most of my projects have a script somewhere that builds both databases. – Ats

@Barracuda you need to use OmniCompletion for code completion in vim. OmniCompletion is now integrated in vim. For using OmniCompletion you need to create ctags file first. cscope and ctags play nice together: first, create cscope.files file (the easiest way is find . -name '*.[ch]' >cscope.files); then create tags file using ctags -L cscope.files command; after this create actual cscope database using cscope -b -q -k command; now you can delete cscope.files file. Also in order to use OmniCompletion you need to configure it in your ~/.vimrc. – Krucik

@SamProtsenko I wish you had written that as an answer to this question. – Hypotrachelium