mirror of https://github.com/Wilfred/difftastic/
Merge commit 'be514eec2c86d560c18fab146e9298e21b8eab62'
Wilfred Hughes
commit
092817a046
@ -1,25 +0,0 @@
|
|||||||
[package]
|
|
||||||
name = "tree-sitter-clojure"
|
|
||||||
description = "clojure grammar for the tree-sitter parsing library"
|
|
||||||
version = "0.0.9"
|
|
||||||
keywords = ["incremental", "parsing", "clojure"]
|
|
||||||
categories = ["parsing", "text-editors"]
|
|
||||||
repository = "https://github.com/sogaiu/tree-sitter-clojure"
|
|
||||||
edition = "2018"
|
|
||||||
|
|
||||||
build = "bindings/rust/build.rs"
|
|
||||||
include = [
|
|
||||||
"bindings/rust/*",
|
|
||||||
"grammar.js",
|
|
||||||
"queries/*",
|
|
||||||
"src/*",
|
|
||||||
]
|
|
||||||
|
|
||||||
[lib]
|
|
||||||
path = "bindings/rust/lib.rs"
|
|
||||||
|
|
||||||
[dependencies]
|
|
||||||
tree-sitter = "0.19.3"
|
|
||||||
|
|
||||||
[build-dependencies]
|
|
||||||
cc = "1.0"
|
|
||||||
@ -1,156 +1,46 @@
|
|||||||
# tree-sitter-clojure
|
# tree-sitter-clojure
|
||||||
|
|
||||||
## Notice
|
A tree-sitter grammar for Clojure and ClojureScript
|
||||||
|
|
||||||
Although no major changes are anticipated at this point, there are no
|
## What the Repository Provides
|
||||||
guarantees. To get a heads-up before such changes occur, please
|
|
||||||
consider subscribing to the [Potential Changes Announcements
|
|
||||||
issue](https://github.com/sogaiu/tree-sitter-clojure/issues/33) to be
|
|
||||||
notified beforehand. The hope is that by commnuicating early enough
|
|
||||||
about these sorts of things, unnecessary breakage can be avoided
|
|
||||||
and/or mitigated.
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
tree-sitter-clojure has been:
|
|
||||||
|
|
||||||
* [Tested in various ways](doc/testing.md)
|
|
||||||
* [Used in some ways](doc/use.md)
|
|
||||||
* [Scoped for better behavior](doc/scope.md)
|
|
||||||
* [Brought about through cooperation](doc/credits.md)
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
Unfortunately, the short of it is that it may be a bit complicated depending on what you want to do.
|
|
||||||
|
|
||||||
* If you don't use any of the wasm-related functionality (e.g. previewing parse results in your web browser or you want to build a `.wasm` file for use in a plugin or extension), you probably just need:
|
|
||||||
* an appropriate version of node (I've tested with various versions >= 12, 14) and
|
|
||||||
* other typical development-related bits (e.g. git, appropriate c compiler, etc.)
|
|
||||||
|
|
||||||
* If you want wasm-related functionality, you get to have fun figuring out which version of [emsdk](https://emscripten.org/docs/getting_started/downloads.html#installation-instructions) currently works with tree-sitter. At the time of this writing, [this file](https://github.com/tree-sitter/tree-sitter/blob/master/cli/emscripten-version) indicates a version that might be appropriate. That may depend on precisely what the versions of other bits (e.g. tree-sitter-cli, web-tree-sitter, etc.) might be though, so if something doesn't work right away, you might consider trying [different versions that have been recorded](https://github.com/tree-sitter/tree-sitter/commits/master/emscripten-version).
|
|
||||||
|
|
||||||
Note that there may be an upside to using emsdk though -- it may figure out and arrange for an appropriate version of node, making a separate installation of node unnecessary. I don't use such a setup on a day-to-day basis, but it did work for me at least once.
|
|
||||||
|
|
||||||
## Fine Print
|
|
||||||
|
|
||||||
* The instructions below assume emsdk has been installed, but `emcc` (tool that can be used to compile to wasm) is not necessarily on one's `PATH`. If an appropriate `emcc` is on one's `PATH` (e.g. emscripten installed via homebrew), the emsdk steps (e.g. `source ~/src/emsdk/emsdk_env.sh`) below may be ignored.
|
|
||||||
|
|
||||||
* `node-gyp` (tool for compiling native addon modules for Node.js) may fail on machines upgraded to macos Catalina. [This document](https://github.com/nodejs/node-gyp/blob/master/macOS_Catalina.md) may help cope with such a situation.
|
|
||||||
|
|
||||||
## Initial Setup
|
|
||||||
|
|
||||||
Suppose typical development sources are stored under `~/src`.
|
|
||||||
|
|
||||||
### Short Version
|
|
||||||
|
|
||||||
```
|
|
||||||
# clone repository
|
|
||||||
cd ~/src
|
|
||||||
git clone https://github.com/sogaiu/tree-sitter-clojure
|
|
||||||
cd tree-sitter-clojure
|
|
||||||
|
|
||||||
# install tree-sitter-cli and dependencies, then build
|
|
||||||
npm ci
|
|
||||||
```
|
|
||||||
|
|
||||||
### Long Version
|
|
||||||
|
|
||||||
```
|
|
||||||
# clone repository
|
|
||||||
cd ~/src
|
|
||||||
git clone https://github.com/sogaiu/tree-sitter-clojure
|
|
||||||
cd tree-sitter-clojure
|
|
||||||
|
|
||||||
# ensure tree-sitter-cli is avaliable as a dev dependency
|
|
||||||
npm install --save-dev --save-exact tree-sitter-cli
|
|
||||||
|
|
||||||
# create `src` and populate with tree-sitter `.c` goodness
|
This repository provides some files used to create various artifacts
|
||||||
npx tree-sitter generate
|
(e.g. dynamic libraries) used for handling Clojure and ClojureScript
|
||||||
|
source code via tree-sitter.
|
||||||
|
|
||||||
# populate `node_modules` with dependencies
|
Please see the [what and why document](doc/what-and-why.md) for
|
||||||
npm install
|
detailed information.
|
||||||
|
|
||||||
# create `build` and populate appropriately
|
## Potential Changes Announcements
|
||||||
npx node-gyp configure
|
|
||||||
|
|
||||||
# create `build/Release` and build `tree_sitter_clojure_binding.node`
|
Changes may occur because:
|
||||||
npx node-gyp rebuild
|
|
||||||
```
|
|
||||||
|
|
||||||
## Grammar Development
|
1. There may be unanticipated important use cases we may want to
|
||||||
|
account for
|
||||||
|
2. The grammar depends on tree-sitter which remains in flux (and is
|
||||||
|
still pre 1.0)
|
||||||
|
3. It's possible we missed something or got something wrong about
|
||||||
|
Clojure and we might want to remedy that
|
||||||
|
|
||||||
Hack on grammar.
|
To get a heads-up before such changes occur, please consider
|
||||||
|
subscribing to the [Potential Changes Announcements
|
||||||
```
|
issue](https://github.com/sogaiu/tree-sitter-clojure/issues/33) to be
|
||||||
# edit grammar.js using some editor
|
notified beforehand.
|
||||||
|
|
||||||
# rebuild tree-sitter stuff
|
|
||||||
npx tree-sitter generate && \
|
|
||||||
npx node-gyp rebuild
|
|
||||||
```
|
|
||||||
|
|
||||||
Parse individual files.
|
|
||||||
|
|
||||||
```
|
|
||||||
# create and populate sample code file for parsing named `sample.clj`
|
|
||||||
|
|
||||||
# parse sample file
|
|
||||||
npx tree-sitter parse sample.clj
|
|
||||||
|
|
||||||
# if output has errors, figure out what's wrong
|
|
||||||
```
|
|
||||||
|
|
||||||
Interactively test in the browser (requires emsdk).
|
|
||||||
|
|
||||||
```
|
|
||||||
# prepare emsdk (specifically emcc) for building .wasm
|
|
||||||
source ~/src/emsdk/emsdk_env.sh
|
|
||||||
|
|
||||||
# build .wasm bits and invoke web-ui for interactive testing
|
|
||||||
npx tree-sitter build-wasm && \
|
|
||||||
npx tree-sitter web-ui
|
|
||||||
|
|
||||||
# in appropriate browser window, paste code in left pane
|
|
||||||
|
|
||||||
# examine results in right pane -- can even click on nodes
|
|
||||||
|
|
||||||
# if output has errors, figure out what's wrong
|
|
||||||
```
|
|
||||||
|
|
||||||
## Measure Performance
|
|
||||||
|
|
||||||
```
|
|
||||||
# single measurement
|
|
||||||
npx tree-sitter parse --time sample.clj
|
|
||||||
|
|
||||||
# mutliple measurements with `multitime`
|
|
||||||
multitime -n10 -s1 npx tree-sitter parse --time --quiet sample.clj
|
|
||||||
```
|
|
||||||
|
|
||||||
## Build .wasm
|
|
||||||
|
|
||||||
Assuming emsdk is installed appropriately under `~/src/emsdk`.
|
|
||||||
|
|
||||||
```
|
Note that previously tagged versions may work fine depending on the
|
||||||
# prepare emsdk (specifically emcc) for use
|
use case. See the [changelog](CHANGELOG.md) for details.
|
||||||
source ~/src/emsdk/emsdk_env.sh
|
|
||||||
|
|
||||||
# create `tree-sitter-clojure.wasm`
|
## Other Documents
|
||||||
npx tree-sitter build-wasm
|
|
||||||
```
|
|
||||||
|
|
||||||
## Resources
|
There are some documents in the [`doc` directory](doc/) covering
|
||||||
|
topics such as:
|
||||||
|
|
||||||
* [Guide to your first Tree-sitter grammar](https://gist.github.com/Aerijo/df27228d70c633e088b0591b8857eeef)
|
* [Scope](doc/scope.md)
|
||||||
* [sublime-clojure](https://github.com/tonsky/sublime-clojure)
|
* [Limits](doc/limits.md)
|
||||||
* [syntax-highlighter](https://github.com/EvgeniyPeshkov/syntax-highlighter)
|
* [Testing](doc/testing.md)
|
||||||
* [tree-sitter](http://tree-sitter.github.io/tree-sitter/)
|
* [Uses](doc/use.md)
|
||||||
* [tree-sitter-clojure.oakmac](https://github.com/oakmac/tree-sitter-clojure)
|
|
||||||
* [tree-sitter-clojure.SergeevPavel](https://github.com/SergeevPavel/tree-sitter-clojure)
|
|
||||||
* [tree-sitter-clojure.Tavistock](https://github.com/Tavistock/tree-sitter-clojure)
|
|
||||||
* [vscode-tree-sitter](https://github.com/georgewfraser/vscode-tree-sitter)
|
|
||||||
* [web-tree-sitter API](https://github.com/tree-sitter/tree-sitter/blob/master/lib/binding_web/tree-sitter-web.d.ts)
|
|
||||||
|
|
||||||
## Acknowledgments
|
## Acknowledgments
|
||||||
|
|
||||||
Please see the [credits](doc/credits.md).
|
Please see the [credits](doc/credits.md).
|
||||||
|
|
||||||
|
|||||||
@ -1,18 +0,0 @@
|
|||||||
{
|
|
||||||
"targets": [
|
|
||||||
{
|
|
||||||
"target_name": "tree_sitter_clojure_binding",
|
|
||||||
"include_dirs": [
|
|
||||||
"<!(node -e \"require('nan')\")",
|
|
||||||
"src"
|
|
||||||
],
|
|
||||||
"sources": [
|
|
||||||
"src/parser.c",
|
|
||||||
"bindings/node/binding.cc"
|
|
||||||
],
|
|
||||||
"cflags_c": [
|
|
||||||
"-std=c99",
|
|
||||||
]
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
@ -1,28 +0,0 @@
|
|||||||
#include "tree_sitter/parser.h"
|
|
||||||
#include <node.h>
|
|
||||||
#include "nan.h"
|
|
||||||
|
|
||||||
using namespace v8;
|
|
||||||
|
|
||||||
extern "C" TSLanguage * tree_sitter_clojure();
|
|
||||||
|
|
||||||
namespace {
|
|
||||||
|
|
||||||
NAN_METHOD(New) {}
|
|
||||||
|
|
||||||
void Init(Local<Object> exports, Local<Object> module) {
|
|
||||||
Local<FunctionTemplate> tpl = Nan::New<FunctionTemplate>(New);
|
|
||||||
tpl->SetClassName(Nan::New("Language").ToLocalChecked());
|
|
||||||
tpl->InstanceTemplate()->SetInternalFieldCount(1);
|
|
||||||
|
|
||||||
Local<Function> constructor = Nan::GetFunction(tpl).ToLocalChecked();
|
|
||||||
Local<Object> instance = constructor->NewInstance(Nan::GetCurrentContext()).ToLocalChecked();
|
|
||||||
Nan::SetInternalFieldPointer(instance, 0, tree_sitter_clojure());
|
|
||||||
|
|
||||||
Nan::Set(instance, Nan::New("name").ToLocalChecked(), Nan::New("clojure").ToLocalChecked());
|
|
||||||
Nan::Set(module, Nan::New("exports").ToLocalChecked(), instance);
|
|
||||||
}
|
|
||||||
|
|
||||||
NODE_MODULE(tree_sitter_clojure_binding, Init)
|
|
||||||
|
|
||||||
} // namespace
|
|
||||||
@ -1,19 +0,0 @@
|
|||||||
try {
|
|
||||||
module.exports = require("../../build/Release/tree_sitter_clojure_binding");
|
|
||||||
} catch (error1) {
|
|
||||||
if (error1.code !== 'MODULE_NOT_FOUND') {
|
|
||||||
throw error1;
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
module.exports = require("../../build/Debug/tree_sitter_clojure_binding");
|
|
||||||
} catch (error2) {
|
|
||||||
if (error2.code !== 'MODULE_NOT_FOUND') {
|
|
||||||
throw error2;
|
|
||||||
}
|
|
||||||
throw error1
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
module.exports.nodeTypeInfo = require("../../src/node-types.json");
|
|
||||||
} catch (_) {}
|
|
||||||
@ -1,40 +0,0 @@
|
|||||||
fn main() {
|
|
||||||
let src_dir = std::path::Path::new("src");
|
|
||||||
|
|
||||||
let mut c_config = cc::Build::new();
|
|
||||||
c_config.include(&src_dir);
|
|
||||||
c_config
|
|
||||||
.flag_if_supported("-Wno-unused-parameter")
|
|
||||||
.flag_if_supported("-Wno-unused-but-set-variable")
|
|
||||||
.flag_if_supported("-Wno-trigraphs");
|
|
||||||
let parser_path = src_dir.join("parser.c");
|
|
||||||
c_config.file(&parser_path);
|
|
||||||
|
|
||||||
// If your language uses an external scanner written in C,
|
|
||||||
// then include this block of code:
|
|
||||||
|
|
||||||
/*
|
|
||||||
let scanner_path = src_dir.join("scanner.c");
|
|
||||||
c_config.file(&scanner_path);
|
|
||||||
println!("cargo:rerun-if-changed={}", scanner_path.to_str().unwrap());
|
|
||||||
*/
|
|
||||||
|
|
||||||
println!("cargo:rerun-if-changed={}", parser_path.to_str().unwrap());
|
|
||||||
c_config.compile("parser");
|
|
||||||
|
|
||||||
// If your language uses an external scanner written in C++,
|
|
||||||
// then include this block of code:
|
|
||||||
|
|
||||||
/*
|
|
||||||
let mut cpp_config = cc::Build::new();
|
|
||||||
cpp_config.cpp(true);
|
|
||||||
cpp_config.include(&src_dir);
|
|
||||||
cpp_config
|
|
||||||
.flag_if_supported("-Wno-unused-parameter")
|
|
||||||
.flag_if_supported("-Wno-unused-but-set-variable");
|
|
||||||
let scanner_path = src_dir.join("scanner.cc");
|
|
||||||
cpp_config.file(&scanner_path);
|
|
||||||
println!("cargo:rerun-if-changed={}", scanner_path.to_str().unwrap());
|
|
||||||
cpp_config.compile("scanner");
|
|
||||||
*/
|
|
||||||
}
|
|
||||||
@ -1,52 +0,0 @@
|
|||||||
//! This crate provides clojure language support for the [tree-sitter][] parsing library.
|
|
||||||
//!
|
|
||||||
//! Typically, you will use the [language][language func] function to add this language to a
|
|
||||||
//! tree-sitter [Parser][], and then use the parser to parse some code:
|
|
||||||
//!
|
|
||||||
//! ```
|
|
||||||
//! let code = "";
|
|
||||||
//! let mut parser = tree_sitter::Parser::new();
|
|
||||||
//! parser.set_language(tree_sitter_javascript::language()).expect("Error loading clojure grammar");
|
|
||||||
//! let tree = parser.parse(code, None).unwrap();
|
|
||||||
//! ```
|
|
||||||
//!
|
|
||||||
//! [Language]: https://docs.rs/tree-sitter/*/tree_sitter/struct.Language.html
|
|
||||||
//! [language func]: fn.language.html
|
|
||||||
//! [Parser]: https://docs.rs/tree-sitter/*/tree_sitter/struct.Parser.html
|
|
||||||
//! [tree-sitter]: https://tree-sitter.github.io/
|
|
||||||
|
|
||||||
use tree_sitter::Language;
|
|
||||||
|
|
||||||
extern "C" {
|
|
||||||
fn tree_sitter_clojure() -> Language;
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Get the tree-sitter [Language][] for this grammar.
|
|
||||||
///
|
|
||||||
/// [Language]: https://docs.rs/tree-sitter/*/tree_sitter/struct.Language.html
|
|
||||||
pub fn language() -> Language {
|
|
||||||
unsafe { tree_sitter_clojure() }
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The content of the [`node-types.json`][] file for this grammar.
|
|
||||||
///
|
|
||||||
/// [`node-types.json`]: https://tree-sitter.github.io/tree-sitter/using-parsers#static-node-types
|
|
||||||
pub const NODE_TYPES: &'static str = include_str!("../../src/node-types.json");
|
|
||||||
|
|
||||||
// Uncomment these to include any queries that this grammar contains
|
|
||||||
|
|
||||||
// pub const HIGHLIGHTS_QUERY: &'static str = include_str!("../../queries/highlights.scm");
|
|
||||||
// pub const INJECTIONS_QUERY: &'static str = include_str!("../../queries/injections.scm");
|
|
||||||
// pub const LOCALS_QUERY: &'static str = include_str!("../../queries/locals.scm");
|
|
||||||
// pub const TAGS_QUERY: &'static str = include_str!("../../queries/tags.scm");
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
#[test]
|
|
||||||
fn test_can_load_grammar() {
|
|
||||||
let mut parser = tree_sitter::Parser::new();
|
|
||||||
parser
|
|
||||||
.set_language(super::language())
|
|
||||||
.expect("Error loading clojure language");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -0,0 +1,28 @@
|
|||||||
|
# Limits
|
||||||
|
|
||||||
|
The following items are known to not necessarily work:
|
||||||
|
|
||||||
|
* [Some template
|
||||||
|
files](https://github.com/sogaiu/tree-sitter-clojure/issues/42#issuecomment-1426727973) -
|
||||||
|
these are often not strictly speaking Clojure, though they look pretty close
|
||||||
|
* Other code that is not standard Clojure
|
||||||
|
[1](https://github.com/fjarri/clojure-scribble#basic-usage)
|
||||||
|
[2](https://github.com/dgrnbrg/piplin/blob/4c39386413d62ec9c2d679fa4c742313d97f75ef/src/piplin/mips.clj#L12)
|
||||||
|
because it uses functionality that modifies Clojure's reader behavior
|
||||||
|
in certain ways [1](https://github.com/jwymanm/chiara#the-syntax)
|
||||||
|
[2](https://github.com/dgrnbrg/piplin/blob/4c39386413d62ec9c2d679fa4c742313d97f75ef/src/piplin/types/bits.clj#L231-L251)
|
||||||
|
* Some older Clojure code - for example, `^` used to mean "wrap the
|
||||||
|
following thing in `(meta ...)`"
|
||||||
|
[1](https://github.com/clojure/clojure/blob/1.0.x/src/jvm/clojure/lang/LispReader.java#L71)
|
||||||
|
[2](https://github.com/clojure/clojure/blob/1.0.x/src/clj/clojure/zip.clj#L58)
|
||||||
|
* [ClojureCLR's pipe syntax for
|
||||||
|
symbols](https://github.com/sogaiu/tree-sitter-clojure/issues/35#issuecomment-1407320526)
|
||||||
|
([comment at #42](https://github.com/sogaiu/tree-sitter-clojure/issues/42#issuecomment-1450290140))
|
||||||
|
* [Files that contain one or more
|
||||||
|
zero-bytes](https://github.com/sogaiu/tree-sitter-clojure/issues/42#issuecomment-1430546851)
|
||||||
|
[1](https://github.com/santifa/clj-dbase/blob/a269ca62d529cf82cec7bffce2e38b71458c6087/src/clj_dbase/core.clj#L121)
|
||||||
|
[2](https://github.com/ont-app/vocabulary/blob/5929b9b1a16b07dc60f1012070da684e8f073326/resources/uri-escapes.edn) -
|
||||||
|
this might be a tree-sitter limitation
|
||||||
|
|
||||||
|
See [#42](https://github.com/sogaiu/tree-sitter-clojure/issues/42) for
|
||||||
|
more details.
|
||||||
@ -0,0 +1,17 @@
|
|||||||
|
# Resources
|
||||||
|
|
||||||
|
Below is a list of resources related to tree-sitter and/or Clojure.
|
||||||
|
|
||||||
|
Some may be a bit dated at this point.
|
||||||
|
|
||||||
|
* [Guide to your first Tree-sitter
|
||||||
|
grammar](https://gist.github.com/Aerijo/df27228d70c633e088b0591b8857eeef)
|
||||||
|
* [sublime-clojure](https://github.com/tonsky/sublime-clojure)
|
||||||
|
* [syntax-highlighter](https://github.com/EvgeniyPeshkov/syntax-highlighter)
|
||||||
|
* [tree-sitter](http://tree-sitter.github.io/tree-sitter/)
|
||||||
|
* [tree-sitter-clojure.oakmac](https://github.com/oakmac/tree-sitter-clojure)
|
||||||
|
* [tree-sitter-clojure.SergeevPavel](https://github.com/SergeevPavel/tree-sitter-clojure)
|
||||||
|
* [tree-sitter-clojure.Tavistock](https://github.com/Tavistock/tree-sitter-clojure)
|
||||||
|
* [vscode-tree-sitter](https://github.com/georgewfraser/vscode-tree-sitter)
|
||||||
|
* [web-tree-sitter
|
||||||
|
API](https://github.com/tree-sitter/tree-sitter/blob/master/lib/binding_web/tree-sitter-web.d.ts)
|
||||||
@ -0,0 +1,101 @@
|
|||||||
|
# What the Repository Provides and Why
|
||||||
|
|
||||||
|
This document describes what files and directories the repository
|
||||||
|
provides and associated reasoning. First it covers things which are
|
||||||
|
likely to remain in place for some time (except perhaps the `src`
|
||||||
|
directory). This is followed by a description of things that are more
|
||||||
|
likely to change or be removed.
|
||||||
|
|
||||||
|
One might be interested in this content out of academic curiosity but
|
||||||
|
more likely it might be because one is thinking of depending on the
|
||||||
|
repository in some way.
|
||||||
|
|
||||||
|
## What and Why
|
||||||
|
|
||||||
|
The order of the following files and directories is alphabetical and
|
||||||
|
not meant to reflect relative importance.
|
||||||
|
|
||||||
|
* `CHANGELOG.md` - this file contains a changelog.
|
||||||
|
|
||||||
|
* `COPYING.txt` - this file contains license information for the
|
||||||
|
repository.
|
||||||
|
|
||||||
|
* `grammar.js` - this file contains a grammar description and is used
|
||||||
|
in the process of generating parser source code that lives in `src`.
|
||||||
|
It's likely that this (or something comparable) will continue to be
|
||||||
|
provided assuming tree-sitter doesn't change the way it works.
|
||||||
|
|
||||||
|
* `package.json` - this file is needed by a
|
||||||
|
[component](https://github.com/cursorless-dev/vscode-parse-tree/) of
|
||||||
|
[Cursorless](https://www.cursorless.org/). It uses our grammar via
|
||||||
|
yarn and `package.json` seems to be essential.
|
||||||
|
|
||||||
|
* `queries` - this directory and the simple file it contains are
|
||||||
|
provided on request from
|
||||||
|
[`difftastic`](https://github.com/Wilfred/difftastic) folks. The
|
||||||
|
file it contains doesn't contain much and is not likely to be the
|
||||||
|
sort of thing one expects to be used in an editor.
|
||||||
|
|
||||||
|
* `README.md` - this file contains the repository's README content.
|
||||||
|
|
||||||
|
* `src` - this directory contains source files that are generated [1]
|
||||||
|
from `grammar.js`. The files are typically used to generate a
|
||||||
|
dynamic library / shared object that can be used by the tree-sitter
|
||||||
|
library to handle Clojure / ClojureScript source code. Although the
|
||||||
|
content of this directory is generated, the files are provided
|
||||||
|
because in practice, multiple parties have already become dependant
|
||||||
|
on them. There have been opinions voiced that this should not
|
||||||
|
remain so, but change in that direction has not been widespread. We
|
||||||
|
would prefer not to be hosting this directory and its content, but
|
||||||
|
are leaving it in place for the time being. See
|
||||||
|
[here](https://github.com/sogaiu/ts-questions/blob/master/questions/should-parser-source-be-committed/README.md)
|
||||||
|
for more on the topic if interested.
|
||||||
|
|
||||||
|
* `test/corpus` - this directory contains tree-sitter's corpus
|
||||||
|
test-related files.
|
||||||
|
|
||||||
|
## Other Content
|
||||||
|
|
||||||
|
The rest of the content of the repository is currently documentation
|
||||||
|
that lives in the `doc` directory.
|
||||||
|
|
||||||
|
## About Bindings
|
||||||
|
|
||||||
|
The repository does not host any bindings (e.g. for Rust, Node, or
|
||||||
|
other programming language).
|
||||||
|
|
||||||
|
They should be straight-forward to generate as long as one has a
|
||||||
|
suitable `tree-sitter` cli and the `grammar.js` file mentioned above.
|
||||||
|
|
||||||
|
Binding code used to be created by the `generate` subcommand, but this
|
||||||
|
appears to have [changed from version 0.24.0 of the `tree-sitter`
|
||||||
|
cli](https://github.com/tree-sitter/tree-sitter/releases/tag/v0.24.0):
|
||||||
|
|
||||||
|
> Move generation of grammar files to an init command ([#3694](https://github.com/tree-sitter/tree-sitter/pull/3694))
|
||||||
|
|
||||||
|
Note that "grammar files" here seems to refer to "bindings" files.
|
||||||
|
|
||||||
|
Further evidence in support of this change is [this
|
||||||
|
documentation](https://tree-sitter.github.io/tree-sitter/cli/init.html#binding-files):
|
||||||
|
|
||||||
|
> When you run tree-sitter init, the CLI will also generate a number
|
||||||
|
> of files in your repository that allow for your parser to be used
|
||||||
|
> from different language.
|
||||||
|
|
||||||
|
Which languages bindings files are generated for is affected by [the
|
||||||
|
`bindings` field in
|
||||||
|
`tree-sitter.json`](https://tree-sitter.github.io/tree-sitter/cli/init.html#the-bindings-field).
|
||||||
|
(It appears that omitting the field means "don't generate any
|
||||||
|
bindings".)
|
||||||
|
|
||||||
|
Probably it's better to consult the official documentation and/or ask
|
||||||
|
around about what the latest procedure is rather than rely on these
|
||||||
|
brief notes though.
|
||||||
|
|
||||||
|
## Footnotes
|
||||||
|
|
||||||
|
[1] If the grammar uses an external scanner, `src` may contain
|
||||||
|
non-generated files such as `scanner.c`, `scanner.cc`, etc. In the
|
||||||
|
current case, no external scanner is used and the `src` directory
|
||||||
|
content is entirely generated.
|
||||||
|
|
||||||
@ -1,278 +0,0 @@
|
|||||||
// NOTES
|
|
||||||
//
|
|
||||||
// - possibilities (may be as separate grammars?)
|
|
||||||
// - no fields (but likely that means metadata lives "outside")
|
|
||||||
// - retain whitespace and comments (for round-tripping)
|
|
||||||
// - clojure clr's pipe-escaping:
|
|
||||||
// https://github.com/clojure/clojure-clr/wiki/Specifying-types
|
|
||||||
//
|
|
||||||
// - traveral issues
|
|
||||||
// - use of fields (e.g. value, prefix, tag, metadata)
|
|
||||||
// - allows skipping certain nodes such as:
|
|
||||||
// - metadata
|
|
||||||
// - comment
|
|
||||||
// - discard-related
|
|
||||||
// - allows targeted navigation without having to know the
|
|
||||||
// node type (e.g. field value vs node type map)
|
|
||||||
// - limitations
|
|
||||||
// - a bit slower?
|
|
||||||
// - cannot use fields for things without names, e.g.
|
|
||||||
// - seq(...) cannot be the 2nd arg to field()
|
|
||||||
// - $._foo won't work unless it "resolves" to $.bar (non underscore)
|
|
||||||
// - for a given node, examine child nodes in reverse, that is,
|
|
||||||
// starting at the end and working backwards
|
|
||||||
//
|
|
||||||
// - probably won't do
|
|
||||||
// - support def, if, and other "primitives"
|
|
||||||
// - support for {{}} template constructs
|
|
||||||
//
|
|
||||||
// - testing
|
|
||||||
// - clj, cljc, cljs
|
|
||||||
// - what about edn?
|
|
||||||
// - approaches
|
|
||||||
// - "port" hand-written tests
|
|
||||||
// - oakmac (done)
|
|
||||||
// - Tavistock (done)
|
|
||||||
// - tonsky
|
|
||||||
// - generative testing for token testing (done via hypothesis and py-tree-sitter)
|
|
||||||
// - look for parsing errors across large sample (e.g. clojars) (done)
|
|
||||||
// - how to "package" testing facilities
|
|
||||||
// - currently each approach has its own project directory
|
|
||||||
//
|
|
||||||
// - debugging
|
|
||||||
// - npx tree-sitter parse filepath + look for ERROR in console output
|
|
||||||
// - npx tree-sitter parse --debug-graph filepath + view log.html
|
|
||||||
// - npx tree-sitter parse --debug filepath + view console output
|
|
||||||
//
|
|
||||||
// - loosening ideas:
|
|
||||||
// - allow ##Other (not just ##Inf, -##Inf, ##NaN)
|
|
||||||
// - allow # in keywords
|
|
||||||
// - allow ::/
|
|
||||||
// - don't handle "no repeating colons" in symbols and in non-leading
|
|
||||||
// portions of keywords (currently unimplemented anyway)
|
|
||||||
//
|
|
||||||
// - can strings have unicode escapes in them?
|
|
||||||
//
|
|
||||||
// - tree-sitter
|
|
||||||
// - parse subcommand
|
|
||||||
// - parse from stdin
|
|
||||||
// - recursively traverse multiple directories (globbing exists)
|
|
||||||
// - parsing within zips/jars
|
|
||||||
// - more flexible file type specification
|
|
||||||
// - custom parsing / processing per "file"
|
|
||||||
// - web-ui subcommand
|
|
||||||
// - didn't work when grammar used externals
|
|
||||||
// - file browsing + loading better than copy-paste
|
|
||||||
// - indiciate error via color
|
|
||||||
// - jump to error
|
|
||||||
// - somehow searching for error doesn't seem to work sometimes
|
|
||||||
// - ~/.tree-sitter
|
|
||||||
// - bin
|
|
||||||
// - contains shared libraries for each grammar
|
|
||||||
// - parse command seems to install stuff here
|
|
||||||
// - config.json
|
|
||||||
// - parser-directories used to customize "scan" for grammars
|
|
||||||
// - theme used for highlight subcommand
|
|
||||||
|
|
||||||
// symbolPat from LispReader.java (for keywords and symbols?)
|
|
||||||
// "[:]?([\\D&&[^/]].*/)?(/|[\\D&&[^/]][^/]*)"
|
|
||||||
//
|
|
||||||
// https://clojure.org/reference/reader#_symbols
|
|
||||||
// 1. Symbols begin with a non-numeric char -- XXX: see 2 for limits?
|
|
||||||
// 2. Can contain alphanumeric chars and *, +, !, -, _, ', ?, <, > and =
|
|
||||||
// 3. / can be used once in the middle of a symbol (sep the ns from the name)
|
|
||||||
// 4. / by itself names the division function
|
|
||||||
// 5. . special meaning can be used >= 1 times in the middle of a symbol
|
|
||||||
// to designate a fully-qualified class name, e.g. java.util.BitSet,
|
|
||||||
// or in namespace names.
|
|
||||||
// 6. Symbols beginning or ending with . are reserved by Clojure
|
|
||||||
// 7. Symbols beginning or ending with : are reserved by Clojure
|
|
||||||
// 8. A symbol can contain one or more non-repeating ':'s
|
|
||||||
//
|
|
||||||
// missing
|
|
||||||
// 9. $, &, % -- in body and end of symbol
|
|
||||||
//
|
|
||||||
// undocumented
|
|
||||||
// -1a can be made a symbol, but reader will reject? repl rejects
|
|
||||||
// => number parsing takes priority?
|
|
||||||
// 'a can be made a symbol, but reader will reject? repl -> quote
|
|
||||||
//
|
|
||||||
// implied?
|
|
||||||
// doesn't start with ,
|
|
||||||
// doesn't start with '
|
|
||||||
// doesn't start with #
|
|
||||||
// doesn't start with `
|
|
||||||
// doesn't start with @
|
|
||||||
// doesn't start with ^
|
|
||||||
// doesn't start with \
|
|
||||||
// doesn't start with ;
|
|
||||||
// doesn't start with ~
|
|
||||||
// doesn't start with "
|
|
||||||
// doesn't start with ( )
|
|
||||||
// doesn't start with { }
|
|
||||||
// doesn't start with [ ]
|
|
||||||
//
|
|
||||||
// extra:
|
|
||||||
//
|
|
||||||
// is my-ns// valid?
|
|
||||||
//
|
|
||||||
// "Consistency of symbols between different readers/edn"
|
|
||||||
//
|
|
||||||
// foo// should be valid.
|
|
||||||
//
|
|
||||||
// 2014-09-16 clojure-dev google group alex miller
|
|
||||||
//
|
|
||||||
// https://groups.google.com/d/msg/clojure-dev/b09WvRR90Zc/c3zzMFqDsRYJ
|
|
||||||
//
|
|
||||||
// "CLJ-1238 Allow EdnReader to read foo// (matches LispReader behavior)"
|
|
||||||
//
|
|
||||||
// changelog for clojure 1.6
|
|
||||||
//
|
|
||||||
// is # allowed as a constituent character in keywords?
|
|
||||||
//
|
|
||||||
// following points are reasoning based on edn docs
|
|
||||||
//
|
|
||||||
// "Bug in reader or repl? reading keyword :#abc"
|
|
||||||
//
|
|
||||||
// Symbols begin with a non-numeric character and can contain
|
|
||||||
// alphanumeric characters and . * + ! - _ ? $ % & =. If -, + or
|
|
||||||
// . are the first character, the second character must be
|
|
||||||
// non-numeric. Additionally, : # are allowed as constituent
|
|
||||||
// characters in symbols other than as the first character.
|
|
||||||
//
|
|
||||||
// 2013-05-02 clojure google group colin jones (fwd by dave sann)
|
|
||||||
//
|
|
||||||
// https://groups.google.com/d/msg/clojure/lK7juHxsPCc/TeYjxoW_3csJ
|
|
||||||
//
|
|
||||||
// Keywords are identifiers that typically designate
|
|
||||||
// themselves. They are semantically akin to enumeration
|
|
||||||
// values. Keywords follow the rules of symbols, except they can
|
|
||||||
// (and must) begin with :, e.g. :fred or :my/fred. If the target
|
|
||||||
// platform does not have a keyword type distinct from a symbol
|
|
||||||
// type, the same type can be used without conflict, since the
|
|
||||||
// mandatory leading : of keywords is disallowed for symbols.
|
|
||||||
//
|
|
||||||
// https://github.com/edn-format/edn#symbols
|
|
||||||
//
|
|
||||||
// https://clojure.org/reference/reader#_literals
|
|
||||||
// 0. Keywords are like symbols, except:
|
|
||||||
// 1. They can and must begin with a colon, e.g. :fred.
|
|
||||||
// ~~2. They cannot contain '.' in the name part, or name classes.~~
|
|
||||||
// 3. They can contain a namespace, :person/name, which may contain '.'s.
|
|
||||||
// 4. A keyword that begins with two colons is auto-resolved in the current
|
|
||||||
// namespace to a qualified keyword:
|
|
||||||
// - If the keyword is unqualified, the namespace will be the current
|
|
||||||
// namespace. In user, ::rect is read as :user/rect.
|
|
||||||
// - If the keyword is qualified, the namespace will be resolved using
|
|
||||||
// aliases in the current namespace. In a namespace where x is aliased
|
|
||||||
// to example, ::x/foo resolves to :example/foo.
|
|
||||||
//
|
|
||||||
// extra:
|
|
||||||
//
|
|
||||||
// :/ is a legal keyword(?):
|
|
||||||
//
|
|
||||||
// alexmiller: @gfredericks :/ is "open for the language to start
|
|
||||||
// interpreting" and not an invalid keyword so should be ok to generate.
|
|
||||||
// and cljs should fix it's weirdness. (#clojure-dev 2019-06-07)
|
|
||||||
//
|
|
||||||
// https://clojurians-log.clojureverse.org/clojure-dev/2019-06-07
|
|
||||||
//
|
|
||||||
// It is undefined/left for future expansion.
|
|
||||||
//
|
|
||||||
// Clojurescript's reading seems weird but given that this is undefined
|
|
||||||
// it's hard to say it's wrong. :)
|
|
||||||
//
|
|
||||||
// 2020-07-10 (or so) alexmiller
|
|
||||||
//
|
|
||||||
// https://ask.clojure.org/index.php/9427/clarify-the-position-of-as-a-keyword
|
|
||||||
// https://clojure.atlassian.net/browse/TCHECK-155
|
|
||||||
//
|
|
||||||
// . CAN be in the name part:
|
|
||||||
//
|
|
||||||
// "[Bug?] Keyword constraints not enforced"
|
|
||||||
//
|
|
||||||
// I think you've both misread "they cannot name classes" to be - "They
|
|
||||||
// cannot contain class names".
|
|
||||||
//
|
|
||||||
// The symbol String can name a class but the keyword :String can't,
|
|
||||||
// that's all I meant there.
|
|
||||||
//
|
|
||||||
// As far as '.', that restriction has been relaxed. I'll try to touch
|
|
||||||
// up the docs for the next release.
|
|
||||||
//
|
|
||||||
// 2008-11-25 clojure google group rich hickey
|
|
||||||
//
|
|
||||||
// https://groups.google.com/d/msg/clojure/CCuIp_bZ-ZM/THea7NF91Z4J
|
|
||||||
//
|
|
||||||
// Whether keywords can start with numbers:
|
|
||||||
//
|
|
||||||
// "puzzled by RuntimeException"
|
|
||||||
//
|
|
||||||
// we currently allow keywords starting with numbers and seem to have
|
|
||||||
// decided this is ok. I would like to get Rich to approve a change to
|
|
||||||
// the page and do so.
|
|
||||||
//
|
|
||||||
// 2014-04-25 clojure google group alex miller
|
|
||||||
//
|
|
||||||
// https://groups.google.com/forum/#!msg/clojure/XP1XAaDdKLY/kodfZTk8eeoJ
|
|
||||||
//
|
|
||||||
// From a discussion in #clojure, it emerged that while :foo/1 is
|
|
||||||
// currently not allowed, ::1 is.
|
|
||||||
//
|
|
||||||
// 2014-12-10 nicola mometto
|
|
||||||
//
|
|
||||||
// https://clojure.atlassian.net/browse/CLJ-1286
|
|
||||||
//
|
|
||||||
// "Clarify and align valid symbol and keyword rules for Clojure (and edn)"
|
|
||||||
//
|
|
||||||
// https://clojure.atlassian.net/browse/CLJ-1527
|
|
||||||
//
|
|
||||||
// consistency of symbols between different readers/edn
|
|
||||||
//
|
|
||||||
// https://groups.google.com/forum/#!topic/clojure-dev/b09WvRR90Zc
|
|
||||||
//
|
|
||||||
// :1 is accepted because it once accidentally worked and they
|
|
||||||
// don't like breaking existing code
|
|
||||||
//
|
|
||||||
// it was never meant to
|
|
||||||
//
|
|
||||||
// 2020-06-14 ish noisesmith on #clojure (slack)
|
|
||||||
//
|
|
||||||
// There are libraries out there that assume :1 works. They changed
|
|
||||||
// Clojure at one point in an alpha to disallow such keywords and it broke
|
|
||||||
// code so they decided to continue allowing them (even tho' they are
|
|
||||||
// not "legal").
|
|
||||||
//
|
|
||||||
// 2020-06-14 ish seancorfield on #clojure (slack)
|
|
||||||
//
|
|
||||||
// Whether # is allowed in a keyword:
|
|
||||||
//
|
|
||||||
// "Clarification on # as valid symbol character"
|
|
||||||
//
|
|
||||||
// this works now, but is not guaranteed to always be valid
|
|
||||||
//
|
|
||||||
// 2016-11-08 clojure google group alex miller
|
|
||||||
//
|
|
||||||
// https://groups.google.com/forum/#!topic/clojure/CwZHu1Eszbk
|
|
||||||
|
|
||||||
// https://clojure.org/reference/reader#_literals
|
|
||||||
// 1. Integers can be indefinitely long and will be read as Longs when
|
|
||||||
// in range and clojure.lang.BigInts otherwise.
|
|
||||||
// 2. Integers with an N suffix are always read as BigInts.
|
|
||||||
// 3. When possible, they can be specified in any base with radix from
|
|
||||||
// 2 to 36 (see Long.parseLong()); for example 2r101010, 8r52, 36r16,
|
|
||||||
// and 42 are all the same Long.
|
|
||||||
// 4. Floating point numbers are read as Doubles; with M suffix they are
|
|
||||||
// read as BigDecimals.
|
|
||||||
// 5. Ratios are supported, e.g. 22/7.
|
|
||||||
|
|
||||||
// intPat
|
|
||||||
// "([-+]?)(?:(0)|([1-9][0-9]*)|0[xX]([0-9A-Fa-f]+)|0([0-7]+)|([1-9][0-9]?)[rR]([0-9A-Za-z]+)|0[0-9]+)(N)?"
|
|
||||||
|
|
||||||
// 0[0-9]+ is for better errors -- thanks seancorfield and andyfingerhut
|
|
||||||
|
|
||||||
// ratioPat
|
|
||||||
// "([-+]?[0-9]+)/([0-9]+)"
|
|
||||||
|
|
||||||
// floatPat
|
|
||||||
// "([-+]?[0-9]+(\\.[0-9]*)?([eE][-+]?[0-9]+)?)(M)?"
|
|
||||||
@ -1,19 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "tree-sitter-clojure",
|
|
||||||
"version": "0.0.11",
|
|
||||||
"lockfileVersion": 1,
|
|
||||||
"requires": true,
|
|
||||||
"dependencies": {
|
|
||||||
"nan": {
|
|
||||||
"version": "2.14.2",
|
|
||||||
"resolved": "https://registry.npmjs.org/nan/-/nan-2.14.2.tgz",
|
|
||||||
"integrity": "sha512-M2ufzIiINKCuDfBSAUr1vWQ+vuVcA9kqx8JJUsbQi6yf1uGRyb7HfpdfUr5qLXf3B/t8dPvcjhKMmlfnP47EzQ=="
|
|
||||||
},
|
|
||||||
"tree-sitter-cli": {
|
|
||||||
"version": "0.19.3",
|
|
||||||
"resolved": "https://registry.npmjs.org/tree-sitter-cli/-/tree-sitter-cli-0.19.3.tgz",
|
|
||||||
"integrity": "sha512-UlntGxLrlkQCKVrhm7guzfi+ovM4wDLVCCu3z5jmfDgFNoUoKa/23ddaQON5afD5jB9a02xv4N5MXJfCx+/mpw==",
|
|
||||||
"dev": true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
File diff suppressed because it is too large
Load Diff
Loading…
Reference in New Issue