Prism Ruby parser

This is a parser for the Ruby programming language. It is designed to be portable, error tolerant, and maintainable. It is written in C99 and has no dependencies.

Overview

The repository contains the infrastructure for both a shared library (libprism) and a native CRuby extension. The shared library has no bindings to CRuby itself, and so can be used by other projects. The native CRuby extension links against ruby.h, and so is suitable in the context of CRuby.

.
├── Makefile              configuration to compile the shared library and native tests
├── Rakefile              configuration to compile the native extension and run the Ruby tests
├── bin
│   ├── lex               runs the lexer on a file or string, prints the tokens, and compares to ripper
│   ├── parse             runs the parse on a file or string and prints the AST
│   └── prism             a CLI for development and debugging
├── config.yml            specification for tokens and nodes in the tree
├── doc                   documentation website
├── docs                  markdown documentation about the project
├── ext
│   └── prism
│       ├── extconf.rb    configuration to generate the Makefile for the native extension
│       └── extension.c   the native extension that interacts with libprism
├── fuzz                  files related to fuzz testing
├── gemfiles              gemfiles used by different Ruby versions in CI
├── include
│   ├── prism             header files for the shared library
│   └── prism.h           main header file for the shared library
├── java
|   ├── api               Java API for Prism
|   ├── native            Java native bindings for the shared library
|   └── wasm              Java WASM bindings for the shared library
├── javascript            JavaScript WASM bindings for the shared library
├── lib
│   ├── prism             Ruby library files
│   └── prism.rb          main entrypoint for the Ruby library
├── rakelib               various Rake tasks for the project
├── rbi                   RBI type signatures for the Ruby library
├── rust
│   ├── ruby-prism        Rustified crate for the shared library
│   └── ruby-prism-sys    FFI binding for Rust
├── sample
│   └── prism             Sample code that uses the Ruby API for documentation purposes
├── sig                   RBS type signatures for the Ruby library
├── src
│   └── prism.c           main entrypoint for the shared library
├── templates             contains ERB templates generated by templates/template.rb
│   └── template.rb       generates code from the nodes and tokens configured by config.yml
└── test
    └── prism
        ├── fixtures      Ruby code used for testing
        └── snapshots     snapshots of generated syntax trees corresponding to fixtures

Getting started

To compile the shared library, you will need:

• C99 compiler
• GNU make
• Ruby 2.7.0 or later

Once you have these dependencies, run:

bundle install

to fetch the Ruby dependencies. Finally, run:

bundle exec rake compile

to compile the shared library. It will be built in the build directory. To test that everything is working, run:

bin/parse -e "1 + 2"

to see the syntax tree for the expression 1 + 2.

Contributing

See the CONTRIBUTING.md file for more information. We additionally have documentation about the overall design of the project as well as various subtopics.

• Build system
• Configuration
• CRuby compilation
• Design
• Encoding
• Fuzzing
• Heredocs
• JavaScript
• Local variable depth
• Mapping
• Parser translation
• Parsing rules
• Releasing
• Ripper translation
• Ruby API
• RubyParser translation
• Serialization
• Testing

Examples

Prism has been integrated into the majority of Ruby runtimes, many libraries, and some applications. Below is a list of some of the projects that use Prism:

Runtimes

• CRuby (via C)
• Garnet (via WASM)
• JRuby (via Java)
• MRuby (via C)
• Natalie (via C++ and Ruby)
• Opal (via Ruby and WASM)
• TruffleRuby (via Java)

Libraries

• dispersion
• fast
• minifyrb
• packwerk (via parser translator)
• rbi
• rails
  • parsing renders
  • parsing rdoc
  • parsing tests
• repl_type_completor
• rubocop (via parser translator)
• ruby-lsp
• smart_todo
• sorbet-eraser
• synvert
• typeprof
• unparser (via parser translator)

Applications

• gem.sh
• Sorbet