AnyText: Incremental, left-recursive Parsing and Pretty-Printing from a single Grammar Definition with first-class LSP support

Abstract

Many current language workbenches do not support incremental parsing or left recursion, lack support for resolving references and pretty-printing or make it difficult to implement the Language Server Protocol (LSP) based on the generated parser. In this paper, we present AnyText, a new language workbench that solves these challenges by implementing the LSP using a scannerless, incremental packrat parser with support for left recursion, based on a grammar description in an extended EBNF-like grammar similar to Xtext/Langium. Furthermore, we show how an EBNF extension with formatting instructions can be used to obtain a pretty-printer. We use a combination of an internal and an external DSL to simplify the process of creating a new language. We demonstrate our approach using two new DSLs and evaluate performance and the amount of manual code necessary to support different editor features.