The utility was removed from this page. You can get the code at MizarParser

-- MichaelNedzelsky - 09 Oct 2006

The following simple utility (written on OCaml) splits given Mizar article into sequence of tokens and stores the result as XML file.

Compile:
  ocamlc voc.ml lexer.ml -o lexer

Usage:
  lexer [dict_dir] article_name

Example:
  lexer dict text/test.miz

The result is the file text/test.tokens.xml