OPUS - an open source parallel corpus

Tools

The following tools have been used for annotation (not including standard GNU-tools):

• align - sentence aligner (based on Gale&Church, 1993) integrated in Uplug
• Uplug - tokenizer, sentence-splitter, XML-tools
  http://www.sourceforge.net/projects/uplug
• OpenNLP & Grok, Jason Baldridge and Gann Bierner
  http://grok.sourceforge.net/
  
  

  tool	language	trained on	trained by
  tagger	English	WSJ+Brown	Gann Bierner
  chunker	English	Penn Tree Bank	Jörg Tiedemann

  
  
• TnT - Statistical Part-of-Speech Tagging, Thorsten Brants
  http://www.coli.uni-sb.de/~thorsten/tnt/
  
  

  tool	language	trained on	trained by
  tagger	German	NEGRA	Thorsten Brants
  tagger	English	WSJ	Thorsten Brants
  tagger	Swedish	SUC	Beáta Megyesi

  
  
• TreeTagger - Decision Tree Tagger, Helmut Schmid
  http://www.ims.uni-stuttgart.de/projekte/corplex/TreeTagger/
  (tokenizer, tagger, lemmatizer)
  
  

  language	trained on	trained by
  German	NEGRA	Helmut Schmid
  English	WSJ	Helmut Schmid
  French		Achim Stein
  Italian		Achim Stein

  
  
• ChaSen - japanese tokenizer + tagger
  http://chasen.aist-nara.ac.jp/
  (tokenizer POS-tagger lemmatizer sentence splitter)


• QTag - a probabilistic parts-of-speech tagger, Oliver Mason
  http://web.bham.ac.uk/o.mason/software/tagger/
  
  

  language	trained on	trained by
  English	1 Mio words	Oliver Mason
  Portuguese	500 Mio words	Tony Berber Sardinha and Rod de Lima-Lopes

  
  
• recode - convert between various character encodings
  http://www.iro.umontreal.ca/contrib/recode/HTML/
• tidy - validate, correct, and pretty-print XML-files
  http://www.w3.org/People/Raggett/tidy/

The following tools are used for data management:

• IMS Corpus Workbench (CWB)
  http://www.ims.uni-stuttgart.de/projekte/CorpusWorkbench/index.html
• The Concurrent Versions System (CVS)
  http://www.cvshome.org/
• ViewCVS for viewing CVS repositories
  http://viewcvs.sourceforge.net/