On April 10, 2013, “開発ツール徹底攻略” (lit. The Definitive Guide to Development Tools), the latest book from WEB+DB PRESS plus series will be released. The book is a collection of excellent articles which were contributed to WEB+DB PRESS, one of the most famous tech magazines in Japan. Each article is a great guide to use important tools for software development, such as Git, GitHub, Jenkins, Vim, Emacs, and Linux.
And I contributed an article to the book. The title is “Vimの流儀” (lit. The Style of Vim). It’s a whirlwind guide to master Vim. It covers various topics, for example:
How to effectively edit text with the standard features, such as operators, text objects, completion, and more.
How to customize various aspects of Vim, such as key bindings, Ex commands, filetype-specific settings.
How to boost productivity with useful plugins.
How to write and to share your own plugins.
Originally the article was contributed to WEB+DB PRESS Vol.52, released on August 2009. It’s still an effective guide to master Vim, because I wrote only essential aspects on Vim for the article. But a few parts are getting obsolete due to the latest trend. So that I revised such parts, and the book contains a revised version of the article.
Previously, I introduced how to "Use Travis CI for Vim plugin development". CI requires well-developed test suites to achieve advantages. But I did not describe much about how to write unit tests for Vim plugin development in the article. If you try to write unit tests for a Vim plugin, you will be faced with problems like the following:
Run a Vim process without any customization to reproduce same results.
For example, plugins installed by users should not be loaded
even if unit tests try to load them by :runtime! plugin/**/*.vim
or other ways.
Resolve dependencies of a plugin to be tested. It requires to fetch proper versions of dependencies and to make them available for a Vim process to run unit tests.
It is hard to resolve each problem. So that there are many frameworks for unit testing. But all of them have one or more problems as follows:
Runtime environment is not considered well for reproducibility.
It is hard to automate run unit tests for CI or other usage.
It is not easy to read and to write unit tests.
Therefore I wrote another framework to resolve all problems. It is called vim-vspec. Though I wrote it 4 years ago, I did not talk much about it. Because the ecosystem including vim-vspec was not mature. But now the ecosystem has been mature and ready to use anyone. So let me describe how I write unit tests for Vim plugin development.
Follow the step 1 to 4 in "Use Travis CI for Vim plugin development".
Install vim-vspec.
As described later, vim-vspec introduces a custom syntax for Vim script to write readable unit tests. vim-vspec includes additional configuration for syntax highlighting and automatic indentation for the custom syntax. So that it is recommended to install vim-vspec for the configuration. But you may choose not to install vim-vspec, because it is automatically downloaded to run unit tests,
Create a directory called t.
Save test scripts into the t directory with extension .vim.
It often requires to describe detailed operations on Vim to test a Vim plugin. So that the best language to write unit tests for Vim plugins is — Vim script. (Note that the ecosystem allows writing tests in an arbitrary language, but it is not necessary in most cases.)
The structure of a test script is similar to RSpec. In short,
Write a describe block
for each target to be tested.
Write an it block
into a describe block for each behavior to be tested.
Write expectations about behavior into an it block.
For example:
describe 'math#round_to_zero'
it 'returns 0 as is'
Expect math#round_to_zero(0) == 0
end
it 'returns a floor of a positive number'
Expect math#round_to_zero(0.1) == 0
Expect math#round_to_zero(1) == 1
Expect math#round_to_zero(1.23) == 1
Expect math#round_to_zero(123.456) == 123
end
it 'returns a ceiling of a negative number'
Expect math#round_to_zero(-0.1) == 0
Expect math#round_to_zero(-1) == -1
Expect math#round_to_zero(-1.23) == -1
Expect math#round_to_zero(-123.456) == -123
end
endTo compare actual results with expected results, use
Expect.
For example:
Expect foo#bar#baz() == 'qux'Write an expression to get an actual result into the left side.
Write an expression for an expected result into the right side.
All comparison operators of Vim script are available. Comparison operators in this context are called "matchers".
An expectation is passed when its actual value is matched to its expected value. Otherwise an expectation is failed.
You may write like Expect X not == Y to inverse the meaning of
a matcher.
You may add your own mathcers to improve readability of expectations.
It is fairly simple; just run rake test.
You will see results of unit tests like the following:
$ rake test bundle exec vim-flavor test -------- Preparing dependencies Checking versions... Use kana/vim-textobj-user ... 0.3.12 Use kana/vim-vspec ... 1.1.0 Deploying plugins... kana/vim-textobj-user 0.3.12 ... skipped (already deployed) kana/vim-vspec 1.1.0 ... skipped (already deployed) Completed. -------- Testing a Vim plugin Files=0, Tests=0, 0 wallclock secs ( 0.00 usr + 0.11 sys = 0.11 CPU) Result: NOTESTS t/c.vim .... ok t/vim.vim .. ok All tests successful. Files=2, Tests=12, 2 wallclock secs ( 0.08 usr 0.72 sys + 0.21 cusr 0.97 csys = 1.98 CPU) Result: PASS
If one or more unit tests are failed, you will see results like the following:
$ rake test
bundle exec vim-flavor test
-------- Preparing dependencies
Checking versions...
Use kana/vim-textobj-user ... 0.3.12
Use kana/vim-vspec ... 1.1.0
Deploying plugins...
kana/vim-textobj-user 0.3.12 ... skipped (already deployed)
kana/vim-vspec 1.1.0 ... skipped (already deployed)
Completed.
-------- Testing a Vim plugin
Files=0, Tests=0, 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU)
Result: NOTESTS
t/c.vim .... 1/?
not ok 1 - <Plug>(textobj-function-a) selects the next function if there is no function under the cursor
# Expected line("'<") == 3
# Actual value: 2
# Expected value: 3
t/c.vim .... Failed 1/6 subtests
t/vim.vim .. ok
Test Summary Report
-------------------
t/c.vim (Wstat: 0 Tests: 6 Failed: 1)
Failed test: 1
Files=2, Tests=12, 1 wallclock secs ( 0.03 usr 0.02 sys + 0.11 cusr 0.06 csys = 0.22 CPU)
Result: FAIL
rake aborted!
Command failed with status (1): [bundle exec vim-flavor test...]
Tasks: TOP => test
(See full trace by running task with --trace)Note that actual output is colored.
Suppose that you define a new operator to edit text. To test its behavior, it is necessary to
Set up a new buffer with sample text before running a test case, and
Discard the buffer after running a test case.
Such initializations and finalizations can be simplified with
before/after.
before block will be executed before running each it block, and
after block will be executed after running each it block.
For example:
describe '...'
before
new
put =[
\ 'foo',
\ 'bar',
\ 'baz',
\ '...',
\ ]
end
after
close!
end
it '...'
...
end
it '...'
...
end
endIt takes a long time to write complete unit tests. So that it is common to
List targets or use cases to be tested, but
Defer writing the contents of the tests.
To indicate such "not implemented yet" tests, use
TODO
as follows:
it '...'
TODO
endTODO tests are always treated as failed ones, and they are highlighted
like the following:
$ rake test ... -------- Testing a Vim plugin Files=0, Tests=0, 0 wallclock secs ( 0.00 usr + 0.04 sys = 0.04 CPU) Result: NOTESTS t/c.vim .... 1/? not ok 1 - # TODO <Plug>(textobj-function-a) selects the next function if there is no function under the cursor t/c.vim .... ok t/vim.vim .. ok All tests successful. Files=2, Tests=12, 1 wallclock secs ( 0.05 usr 0.46 sys + 0.13 cusr 0.40 csys = 1.04 CPU) Result: PASS
Sometimes you have to write tests for a specific environment.
Such tests should be skipped for other environments.
You can use
SKIP
to indicate such tests:
it '...'
if executable('git') < 1
SKIP 'Git is not available.'
endif
...
endSKIP tests are always treated as passed ones, and they are highlighted
like the following:
$ rake test ... -------- Testing a Vim plugin Files=0, Tests=0, 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU) Result: NOTESTS t/c.vim .... 1/? ok 1 - # SKIP <Plug>(textobj-function-a) selects the next function if there is no function under the cursor - 'Git is not available.' t/c.vim .... ok t/vim.vim .. ok All tests successful. Files=2, Tests=12, 0 wallclock secs ( 0.03 usr 0.01 sys + 0.08 cusr 0.02 csys = 0.14 CPU) Result: PASS
It is also possible to:
Call script-local functions and read/write script-local variables
Write unit tests in an arbitrary language (note that the output format must be Test Anything Protocol)
Travis CI provides a hosted continuous integration (CI) service for the open source community. It is integrated with GitHub and supports various languages. Since Travis CI is flexible, it is not difficult to use Travis CI for projects written in languages for which Travis CI does not offer first class support. So that it is possible to introduce CI for Vim plugins.
But there are problems to start CI for Vim plugins. For example:
How do I write tests for Vim plugins?
How do I run tests in a clean environment?
How do I resolve dependencies for Vim plugins to run tests?
Fortunately, there is a toolchain to solve the above problems. The toolchain consists of two parts; vim-vspec and vim-flavor. So that it is easy to start CI for Vim plugins with the following steps.
Write a Gemfile
to install the toolchain,
then commit it:
source 'https://rubygems.org'
gem 'vim-flavor', '~> 1.1'Then run bundle install to install the toolchain.
Write
a Rakefile
to automate testing,
then commit it:
#!/usr/bin/env rake
task :ci => [:dump, :test]
task :dump do
sh 'vim --version'
end
task :test do
sh 'bundle exec vim-flavor test'
endNow you can run tests with rake test.
Note that tasks except test are to output extra information
for tests run on Travis CI.
Write
a .travis.yml
to configure how to run tests on Travis CI,
then commit it:
language: ruby
rvm:
- 1.9.3
script: rake ci(You may skip this step if a target Vim plugin does not require any other plugins.)
Some Vim plugins require other plugins. For example, vim-textobj-function provides text objects to edit text by a function. It is not easy to implement text objects because of many edge cases. So that vim-textobj-function uses vim-textobj-user to hide such details.
And dependencies can be complex. For example,
Plugin A 1.0.0 requires plugin B 2.2 or later.
Plugin A 3.0.0 requires plugin B 4.4 or later.
But plugin B 2.x and 4.x are incompatible; so that "2.2 or later" implies "2.2 or later versions compatible with 2.x".
It is hard to maintain such dependencies and compatibility problems by hand.
But vim-flavor automates complex tasks.
All you have to do is to write
a VimFlavor
to declare dependencies of a Vim plugin.
For example, vim-textobj-function requires vim-textobj-user 0.3 or later (except 1.x or later).
Such a dependency is declared as follows:
flavor 'kana/vim-textobj-user', '~> 0.3'See also the document about dependency declaration of a Vim plugin.
Vim plugins are run on Vim. Tests for Vim plugins often require to specify complex operations of Vim. So that the best language to write tests for Vim plugins is — Vim script.
So,
Create a directory called t.
Write test scripts in the vim-vspec format.
Save test scripts into the t directory. Each script name must be end with .vim.
With vim-vspec, you can write tests like RSpec:
describe 'math#round_to_zero'
it 'returns a floor for a positive number'
Expect math#round_to_zero(1.2) == 1
Expect math#round_to_zero(34.5) == 34
end
endSee also tests for vim-textobj-function and tests for vim-vspec for real world examples.
Example: a result of tests for vim-textobj-function run on Travis CI
Vim plugins sometimes require other plugins as libraries. For example, vim-textobj-entire provides text objects to deal with the entire text in the current buffer. But it is hard to properly implement text objects because of many pitfalls and repetitive routines. So that vim-textobj-entire uses vim-textobj-user to define text objects in a simplified and declarative way. Therefore, if user wants to use vim-textobj-entire, he or she must install both vim-textobj-entire and vim-textobj-user.
But it is a boring task to install dependencies by hand. Even if the authors of a plugin noted about dependencies in its document, such notes are often overlooked.
So that I’ve released
vim-flavor 1.1.0.
Now vim-flavor
automatically resolves dependencies of Vim plugins.
If a plugin declares its dependencies with a
flavorfile
and saves it as VimFlavor, vim-flavor reads the file and automatically
installs dependencies according to the file.
vim-flavor also takes care about versions of Vim plugins. If two plugins require the same plugin but required versions are not compatible to others, installation will be stopped to avoid using Vim with a broken configuration.
I’ll update my plugins, especially ones using vim-textobj-user, to declare dependencies for vim-flavor.
There are several outputs as follows. But I’m not satisfied with the result.
Wrote hatokurandom, a tool for Heart of Crown. It was the first time for me to write a mobile Web application with jQuery Mobile.
Read The RSpec Book and tried behavior-driven development with Cucumber.
Wrote vim-flavor, a Vim plugin manager based on my own philosophy.
Created more Vim plugins such as vim-niceblock, vim-smartinput and vim-tabpagecd. Most of them were extracted as plugins from my vimrc.
Refined existing Vim plugins such as vim-altr, vim-arpeggio, vim-operator-replace, vim-operator-user and vim-vspec. Especially vim-vspec was completely rewritten and makes it easy to write more readable tests.
Held TokyoVim, a workshop/hackathon on Vim many times.
Worked as a technical reviewer of Practical Vim. I’m acknowledged a few times.
But didn’t log such activities and progress of currently working projects. It’s good to write much code, but I should write much text too.
Completely failed. Though MacBook Pro with Retina display was a must tool as a developer, I bought many stuffs, especially
And… a one-off model who was unveiled last December at Tenshi-no-Mado.
More output with a new language.
Dispose loots.