mirror of https://github.com/Wilfred/difftastic/
Merge commit 'be514eec2c86d560c18fab146e9298e21b8eab62'
Wilfred Hughes
commit
092817a046
@ -1,25 +0,0 @@
|
||||
[package]
|
||||
name = "tree-sitter-clojure"
|
||||
description = "clojure grammar for the tree-sitter parsing library"
|
||||
version = "0.0.9"
|
||||
keywords = ["incremental", "parsing", "clojure"]
|
||||
categories = ["parsing", "text-editors"]
|
||||
repository = "https://github.com/sogaiu/tree-sitter-clojure"
|
||||
edition = "2018"
|
||||
|
||||
build = "bindings/rust/build.rs"
|
||||
include = [
|
||||
"bindings/rust/*",
|
||||
"grammar.js",
|
||||
"queries/*",
|
||||
"src/*",
|
||||
]
|
||||
|
||||
[lib]
|
||||
path = "bindings/rust/lib.rs"
|
||||
|
||||
[dependencies]
|
||||
tree-sitter = "0.19.3"
|
||||
|
||||
[build-dependencies]
|
||||
cc = "1.0"
|
||||
@ -1,156 +1,46 @@
|
||||
# tree-sitter-clojure
|
||||
|
||||
## Notice
|
||||
A tree-sitter grammar for Clojure and ClojureScript
|
||||
|
||||
Although no major changes are anticipated at this point, there are no
|
||||
guarantees. To get a heads-up before such changes occur, please
|
||||
consider subscribing to the [Potential Changes Announcements
|
||||
issue](https://github.com/sogaiu/tree-sitter-clojure/issues/33) to be
|
||||
notified beforehand. The hope is that by commnuicating early enough
|
||||
about these sorts of things, unnecessary breakage can be avoided
|
||||
and/or mitigated.
|
||||
|
||||
## Status
|
||||
|
||||
tree-sitter-clojure has been:
|
||||
|
||||
* [Tested in various ways](doc/testing.md)
|
||||
* [Used in some ways](doc/use.md)
|
||||
* [Scoped for better behavior](doc/scope.md)
|
||||
* [Brought about through cooperation](doc/credits.md)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Unfortunately, the short of it is that it may be a bit complicated depending on what you want to do.
|
||||
|
||||
* If you don't use any of the wasm-related functionality (e.g. previewing parse results in your web browser or you want to build a `.wasm` file for use in a plugin or extension), you probably just need:
|
||||
* an appropriate version of node (I've tested with various versions >= 12, 14) and
|
||||
* other typical development-related bits (e.g. git, appropriate c compiler, etc.)
|
||||
|
||||
* If you want wasm-related functionality, you get to have fun figuring out which version of [emsdk](https://emscripten.org/docs/getting_started/downloads.html#installation-instructions) currently works with tree-sitter. At the time of this writing, [this file](https://github.com/tree-sitter/tree-sitter/blob/master/cli/emscripten-version) indicates a version that might be appropriate. That may depend on precisely what the versions of other bits (e.g. tree-sitter-cli, web-tree-sitter, etc.) might be though, so if something doesn't work right away, you might consider trying [different versions that have been recorded](https://github.com/tree-sitter/tree-sitter/commits/master/emscripten-version).
|
||||
|
||||
Note that there may be an upside to using emsdk though -- it may figure out and arrange for an appropriate version of node, making a separate installation of node unnecessary. I don't use such a setup on a day-to-day basis, but it did work for me at least once.
|
||||
|
||||
## Fine Print
|
||||
|
||||
* The instructions below assume emsdk has been installed, but `emcc` (tool that can be used to compile to wasm) is not necessarily on one's `PATH`. If an appropriate `emcc` is on one's `PATH` (e.g. emscripten installed via homebrew), the emsdk steps (e.g. `source ~/src/emsdk/emsdk_env.sh`) below may be ignored.
|
||||
|
||||
* `node-gyp` (tool for compiling native addon modules for Node.js) may fail on machines upgraded to macos Catalina. [This document](https://github.com/nodejs/node-gyp/blob/master/macOS_Catalina.md) may help cope with such a situation.
|
||||
|
||||
## Initial Setup
|
||||
|
||||
Suppose typical development sources are stored under `~/src`.
|
||||
|
||||
### Short Version
|
||||
|
||||
```
|
||||
# clone repository
|
||||
cd ~/src
|
||||
git clone https://github.com/sogaiu/tree-sitter-clojure
|
||||
cd tree-sitter-clojure
|
||||
|
||||
# install tree-sitter-cli and dependencies, then build
|
||||
npm ci
|
||||
```
|
||||
|
||||
### Long Version
|
||||
|
||||
```
|
||||
# clone repository
|
||||
cd ~/src
|
||||
git clone https://github.com/sogaiu/tree-sitter-clojure
|
||||
cd tree-sitter-clojure
|
||||
|
||||
# ensure tree-sitter-cli is avaliable as a dev dependency
|
||||
npm install --save-dev --save-exact tree-sitter-cli
|
||||
## What the Repository Provides
|
||||
|
||||
# create `src` and populate with tree-sitter `.c` goodness
|
||||
npx tree-sitter generate
|
||||
This repository provides some files used to create various artifacts
|
||||
(e.g. dynamic libraries) used for handling Clojure and ClojureScript
|
||||
source code via tree-sitter.
|
||||
|
||||
# populate `node_modules` with dependencies
|
||||
npm install
|
||||
Please see the [what and why document](doc/what-and-why.md) for
|
||||
detailed information.
|
||||
|
||||
# create `build` and populate appropriately
|
||||
npx node-gyp configure
|
||||
## Potential Changes Announcements
|
||||
|
||||
# create `build/Release` and build `tree_sitter_clojure_binding.node`
|
||||
npx node-gyp rebuild
|
||||
```
|
||||
Changes may occur because:
|
||||
|
||||
## Grammar Development
|
||||
1. There may be unanticipated important use cases we may want to
|
||||
account for
|
||||
2. The grammar depends on tree-sitter which remains in flux (and is
|
||||
still pre 1.0)
|
||||
3. It's possible we missed something or got something wrong about
|
||||
Clojure and we might want to remedy that
|
||||
|
||||
Hack on grammar.
|
||||
|
||||
```
|
||||
# edit grammar.js using some editor
|
||||
|
||||
# rebuild tree-sitter stuff
|
||||
npx tree-sitter generate && \
|
||||
npx node-gyp rebuild
|
||||
```
|
||||
|
||||
Parse individual files.
|
||||
|
||||
```
|
||||
# create and populate sample code file for parsing named `sample.clj`
|
||||
|
||||
# parse sample file
|
||||
npx tree-sitter parse sample.clj
|
||||
|
||||
# if output has errors, figure out what's wrong
|
||||
```
|
||||
|
||||
Interactively test in the browser (requires emsdk).
|
||||
|
||||
```
|
||||
# prepare emsdk (specifically emcc) for building .wasm
|
||||
source ~/src/emsdk/emsdk_env.sh
|
||||
|
||||
# build .wasm bits and invoke web-ui for interactive testing
|
||||
npx tree-sitter build-wasm && \
|
||||
npx tree-sitter web-ui
|
||||
|
||||
# in appropriate browser window, paste code in left pane
|
||||
|
||||
# examine results in right pane -- can even click on nodes
|
||||
|
||||
# if output has errors, figure out what's wrong
|
||||
```
|
||||
|
||||
## Measure Performance
|
||||
|
||||
```
|
||||
# single measurement
|
||||
npx tree-sitter parse --time sample.clj
|
||||
|
||||
# mutliple measurements with `multitime`
|
||||
multitime -n10 -s1 npx tree-sitter parse --time --quiet sample.clj
|
||||
```
|
||||
|
||||
## Build .wasm
|
||||
|
||||
Assuming emsdk is installed appropriately under `~/src/emsdk`.
|
||||
To get a heads-up before such changes occur, please consider
|
||||
subscribing to the [Potential Changes Announcements
|
||||
issue](https://github.com/sogaiu/tree-sitter-clojure/issues/33) to be
|
||||
notified beforehand.
|
||||
|
||||
```
|
||||
# prepare emsdk (specifically emcc) for use
|
||||
source ~/src/emsdk/emsdk_env.sh
|
||||
Note that previously tagged versions may work fine depending on the
|
||||
use case. See the [changelog](CHANGELOG.md) for details.
|
||||
|
||||
# create `tree-sitter-clojure.wasm`
|
||||
npx tree-sitter build-wasm
|
||||
```
|
||||
## Other Documents
|
||||
|
||||
## Resources
|
||||
There are some documents in the [`doc` directory](doc/) covering
|
||||
topics such as:
|
||||
|
||||
* [Guide to your first Tree-sitter grammar](https://gist.github.com/Aerijo/df27228d70c633e088b0591b8857eeef)
|
||||
* [sublime-clojure](https://github.com/tonsky/sublime-clojure)
|
||||
* [syntax-highlighter](https://github.com/EvgeniyPeshkov/syntax-highlighter)
|
||||
* [tree-sitter](http://tree-sitter.github.io/tree-sitter/)
|
||||
* [tree-sitter-clojure.oakmac](https://github.com/oakmac/tree-sitter-clojure)
|
||||
* [tree-sitter-clojure.SergeevPavel](https://github.com/SergeevPavel/tree-sitter-clojure)
|
||||
* [tree-sitter-clojure.Tavistock](https://github.com/Tavistock/tree-sitter-clojure)
|
||||
* [vscode-tree-sitter](https://github.com/georgewfraser/vscode-tree-sitter)
|
||||
* [web-tree-sitter API](https://github.com/tree-sitter/tree-sitter/blob/master/lib/binding_web/tree-sitter-web.d.ts)
|
||||
* [Scope](doc/scope.md)
|
||||
* [Limits](doc/limits.md)
|
||||
* [Testing](doc/testing.md)
|
||||
* [Uses](doc/use.md)
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
Please see the [credits](doc/credits.md).
|
||||
|
||||
|
||||
@ -1,18 +0,0 @@
|
||||
{
|
||||
"targets": [
|
||||
{
|
||||
"target_name": "tree_sitter_clojure_binding",
|
||||
"include_dirs": [
|
||||
"<!(node -e \"require('nan')\")",
|
||||
"src"
|
||||
],
|
||||
"sources": [
|
||||
"src/parser.c",
|
||||
"bindings/node/binding.cc"
|
||||
],
|
||||
"cflags_c": [
|
||||
"-std=c99",
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -1,28 +0,0 @@
|
||||
#include "tree_sitter/parser.h"
|
||||
#include <node.h>
|
||||
#include "nan.h"
|
||||
|
||||
using namespace v8;
|
||||
|
||||
extern "C" TSLanguage * tree_sitter_clojure();
|
||||
|
||||
namespace {
|
||||
|
||||
NAN_METHOD(New) {}
|
||||
|
||||
void Init(Local<Object> exports, Local<Object> module) {
|
||||
Local<FunctionTemplate> tpl = Nan::New<FunctionTemplate>(New);
|
||||
tpl->SetClassName(Nan::New("Language").ToLocalChecked());
|
||||
tpl->InstanceTemplate()->SetInternalFieldCount(1);
|
||||
|
||||
Local<Function> constructor = Nan::GetFunction(tpl).ToLocalChecked();
|
||||
Local<Object> instance = constructor->NewInstance(Nan::GetCurrentContext()).ToLocalChecked();
|
||||
Nan::SetInternalFieldPointer(instance, 0, tree_sitter_clojure());
|
||||
|
||||
Nan::Set(instance, Nan::New("name").ToLocalChecked(), Nan::New("clojure").ToLocalChecked());
|
||||
Nan::Set(module, Nan::New("exports").ToLocalChecked(), instance);
|
||||
}
|
||||
|
||||
NODE_MODULE(tree_sitter_clojure_binding, Init)
|
||||
|
||||
} // namespace
|
||||
@ -1,19 +0,0 @@
|
||||
try {
|
||||
module.exports = require("../../build/Release/tree_sitter_clojure_binding");
|
||||
} catch (error1) {
|
||||
if (error1.code !== 'MODULE_NOT_FOUND') {
|
||||
throw error1;
|
||||
}
|
||||
try {
|
||||
module.exports = require("../../build/Debug/tree_sitter_clojure_binding");
|
||||
} catch (error2) {
|
||||
if (error2.code !== 'MODULE_NOT_FOUND') {
|
||||
throw error2;
|
||||
}
|
||||
throw error1
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
module.exports.nodeTypeInfo = require("../../src/node-types.json");
|
||||
} catch (_) {}
|
||||
@ -1,40 +0,0 @@
|
||||
fn main() {
|
||||
let src_dir = std::path::Path::new("src");
|
||||
|
||||
let mut c_config = cc::Build::new();
|
||||
c_config.include(&src_dir);
|
||||
c_config
|
||||
.flag_if_supported("-Wno-unused-parameter")
|
||||
.flag_if_supported("-Wno-unused-but-set-variable")
|
||||
.flag_if_supported("-Wno-trigraphs");
|
||||
let parser_path = src_dir.join("parser.c");
|
||||
c_config.file(&parser_path);
|
||||
|
||||
// If your language uses an external scanner written in C,
|
||||
// then include this block of code:
|
||||
|
||||
/*
|
||||
let scanner_path = src_dir.join("scanner.c");
|
||||
c_config.file(&scanner_path);
|
||||
println!("cargo:rerun-if-changed={}", scanner_path.to_str().unwrap());
|
||||
*/
|
||||
|
||||
println!("cargo:rerun-if-changed={}", parser_path.to_str().unwrap());
|
||||
c_config.compile("parser");
|
||||
|
||||
// If your language uses an external scanner written in C++,
|
||||
// then include this block of code:
|
||||
|
||||
/*
|
||||
let mut cpp_config = cc::Build::new();
|
||||
cpp_config.cpp(true);
|
||||
cpp_config.include(&src_dir);
|
||||
cpp_config
|
||||
.flag_if_supported("-Wno-unused-parameter")
|
||||
.flag_if_supported("-Wno-unused-but-set-variable");
|
||||
let scanner_path = src_dir.join("scanner.cc");
|
||||
cpp_config.file(&scanner_path);
|
||||
println!("cargo:rerun-if-changed={}", scanner_path.to_str().unwrap());
|
||||
cpp_config.compile("scanner");
|
||||
*/
|
||||
}
|
||||
@ -1,52 +0,0 @@
|
||||
//! This crate provides clojure language support for the [tree-sitter][] parsing library.
|
||||
//!
|
||||
//! Typically, you will use the [language][language func] function to add this language to a
|
||||
//! tree-sitter [Parser][], and then use the parser to parse some code:
|
||||
//!
|
||||
//! ```
|
||||
//! let code = "";
|
||||
//! let mut parser = tree_sitter::Parser::new();
|
||||
//! parser.set_language(tree_sitter_javascript::language()).expect("Error loading clojure grammar");
|
||||
//! let tree = parser.parse(code, None).unwrap();
|
||||
//! ```
|
||||
//!
|
||||
//! [Language]: https://docs.rs/tree-sitter/*/tree_sitter/struct.Language.html
|
||||
//! [language func]: fn.language.html
|
||||
//! [Parser]: https://docs.rs/tree-sitter/*/tree_sitter/struct.Parser.html
|
||||
//! [tree-sitter]: https://tree-sitter.github.io/
|
||||
|
||||
use tree_sitter::Language;
|
||||
|
||||
extern "C" {
|
||||
fn tree_sitter_clojure() -> Language;
|
||||
}
|
||||
|
||||
/// Get the tree-sitter [Language][] for this grammar.
|
||||
///
|
||||
/// [Language]: https://docs.rs/tree-sitter/*/tree_sitter/struct.Language.html
|
||||
pub fn language() -> Language {
|
||||
unsafe { tree_sitter_clojure() }
|
||||
}
|
||||
|
||||
/// The content of the [`node-types.json`][] file for this grammar.
|
||||
///
|
||||
/// [`node-types.json`]: https://tree-sitter.github.io/tree-sitter/using-parsers#static-node-types
|
||||
pub const NODE_TYPES: &'static str = include_str!("../../src/node-types.json");
|
||||
|
||||
// Uncomment these to include any queries that this grammar contains
|
||||
|
||||
// pub const HIGHLIGHTS_QUERY: &'static str = include_str!("../../queries/highlights.scm");
|
||||
// pub const INJECTIONS_QUERY: &'static str = include_str!("../../queries/injections.scm");
|
||||
// pub const LOCALS_QUERY: &'static str = include_str!("../../queries/locals.scm");
|
||||
// pub const TAGS_QUERY: &'static str = include_str!("../../queries/tags.scm");
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn test_can_load_grammar() {
|
||||
let mut parser = tree_sitter::Parser::new();
|
||||
parser
|
||||
.set_language(super::language())
|
||||
.expect("Error loading clojure language");
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,28 @@
|
||||
# Limits
|
||||
|
||||
The following items are known to not necessarily work:
|
||||
|
||||
* [Some template
|
||||
files](https://github.com/sogaiu/tree-sitter-clojure/issues/42#issuecomment-1426727973) -
|
||||
these are often not strictly speaking Clojure, though they look pretty close
|
||||
* Other code that is not standard Clojure
|
||||
[1](https://github.com/fjarri/clojure-scribble#basic-usage)
|
||||
[2](https://github.com/dgrnbrg/piplin/blob/4c39386413d62ec9c2d679fa4c742313d97f75ef/src/piplin/mips.clj#L12)
|
||||
because it uses functionality that modifies Clojure's reader behavior
|
||||
in certain ways [1](https://github.com/jwymanm/chiara#the-syntax)
|
||||
[2](https://github.com/dgrnbrg/piplin/blob/4c39386413d62ec9c2d679fa4c742313d97f75ef/src/piplin/types/bits.clj#L231-L251)
|
||||
* Some older Clojure code - for example, `^` used to mean "wrap the
|
||||
following thing in `(meta ...)`"
|
||||
[1](https://github.com/clojure/clojure/blob/1.0.x/src/jvm/clojure/lang/LispReader.java#L71)
|
||||
[2](https://github.com/clojure/clojure/blob/1.0.x/src/clj/clojure/zip.clj#L58)
|
||||
* [ClojureCLR's pipe syntax for
|
||||
symbols](https://github.com/sogaiu/tree-sitter-clojure/issues/35#issuecomment-1407320526)
|
||||
([comment at #42](https://github.com/sogaiu/tree-sitter-clojure/issues/42#issuecomment-1450290140))
|
||||
* [Files that contain one or more
|
||||
zero-bytes](https://github.com/sogaiu/tree-sitter-clojure/issues/42#issuecomment-1430546851)
|
||||
[1](https://github.com/santifa/clj-dbase/blob/a269ca62d529cf82cec7bffce2e38b71458c6087/src/clj_dbase/core.clj#L121)
|
||||
[2](https://github.com/ont-app/vocabulary/blob/5929b9b1a16b07dc60f1012070da684e8f073326/resources/uri-escapes.edn) -
|
||||
this might be a tree-sitter limitation
|
||||
|
||||
See [#42](https://github.com/sogaiu/tree-sitter-clojure/issues/42) for
|
||||
more details.
|
||||
@ -0,0 +1,17 @@
|
||||
# Resources
|
||||
|
||||
Below is a list of resources related to tree-sitter and/or Clojure.
|
||||
|
||||
Some may be a bit dated at this point.
|
||||
|
||||
* [Guide to your first Tree-sitter
|
||||
grammar](https://gist.github.com/Aerijo/df27228d70c633e088b0591b8857eeef)
|
||||
* [sublime-clojure](https://github.com/tonsky/sublime-clojure)
|
||||
* [syntax-highlighter](https://github.com/EvgeniyPeshkov/syntax-highlighter)
|
||||
* [tree-sitter](http://tree-sitter.github.io/tree-sitter/)
|
||||
* [tree-sitter-clojure.oakmac](https://github.com/oakmac/tree-sitter-clojure)
|
||||
* [tree-sitter-clojure.SergeevPavel](https://github.com/SergeevPavel/tree-sitter-clojure)
|
||||
* [tree-sitter-clojure.Tavistock](https://github.com/Tavistock/tree-sitter-clojure)
|
||||
* [vscode-tree-sitter](https://github.com/georgewfraser/vscode-tree-sitter)
|
||||
* [web-tree-sitter
|
||||
API](https://github.com/tree-sitter/tree-sitter/blob/master/lib/binding_web/tree-sitter-web.d.ts)
|
||||
@ -0,0 +1,101 @@
|
||||
# What the Repository Provides and Why
|
||||
|
||||
This document describes what files and directories the repository
|
||||
provides and associated reasoning. First it covers things which are
|
||||
likely to remain in place for some time (except perhaps the `src`
|
||||
directory). This is followed by a description of things that are more
|
||||
likely to change or be removed.
|
||||
|
||||
One might be interested in this content out of academic curiosity but
|
||||
more likely it might be because one is thinking of depending on the
|
||||
repository in some way.
|
||||
|
||||
## What and Why
|
||||
|
||||
The order of the following files and directories is alphabetical and
|
||||
not meant to reflect relative importance.
|
||||
|
||||
* `CHANGELOG.md` - this file contains a changelog.
|
||||
|
||||
* `COPYING.txt` - this file contains license information for the
|
||||
repository.
|
||||
|
||||
* `grammar.js` - this file contains a grammar description and is used
|
||||
in the process of generating parser source code that lives in `src`.
|
||||
It's likely that this (or something comparable) will continue to be
|
||||
provided assuming tree-sitter doesn't change the way it works.
|
||||
|
||||
* `package.json` - this file is needed by a
|
||||
[component](https://github.com/cursorless-dev/vscode-parse-tree/) of
|
||||
[Cursorless](https://www.cursorless.org/). It uses our grammar via
|
||||
yarn and `package.json` seems to be essential.
|
||||
|
||||
* `queries` - this directory and the simple file it contains are
|
||||
provided on request from
|
||||
[`difftastic`](https://github.com/Wilfred/difftastic) folks. The
|
||||
file it contains doesn't contain much and is not likely to be the
|
||||
sort of thing one expects to be used in an editor.
|
||||
|
||||
* `README.md` - this file contains the repository's README content.
|
||||
|
||||
* `src` - this directory contains source files that are generated [1]
|
||||
from `grammar.js`. The files are typically used to generate a
|
||||
dynamic library / shared object that can be used by the tree-sitter
|
||||
library to handle Clojure / ClojureScript source code. Although the
|
||||
content of this directory is generated, the files are provided
|
||||
because in practice, multiple parties have already become dependant
|
||||
on them. There have been opinions voiced that this should not
|
||||
remain so, but change in that direction has not been widespread. We
|
||||
would prefer not to be hosting this directory and its content, but
|
||||
are leaving it in place for the time being. See
|
||||
[here](https://github.com/sogaiu/ts-questions/blob/master/questions/should-parser-source-be-committed/README.md)
|
||||
for more on the topic if interested.
|
||||
|
||||
* `test/corpus` - this directory contains tree-sitter's corpus
|
||||
test-related files.
|
||||
|
||||
## Other Content
|
||||
|
||||
The rest of the content of the repository is currently documentation
|
||||
that lives in the `doc` directory.
|
||||
|
||||
## About Bindings
|
||||
|
||||
The repository does not host any bindings (e.g. for Rust, Node, or
|
||||
other programming language).
|
||||
|
||||
They should be straight-forward to generate as long as one has a
|
||||
suitable `tree-sitter` cli and the `grammar.js` file mentioned above.
|
||||
|
||||
Binding code used to be created by the `generate` subcommand, but this
|
||||
appears to have [changed from version 0.24.0 of the `tree-sitter`
|
||||
cli](https://github.com/tree-sitter/tree-sitter/releases/tag/v0.24.0):
|
||||
|
||||
> Move generation of grammar files to an init command ([#3694](https://github.com/tree-sitter/tree-sitter/pull/3694))
|
||||
|
||||
Note that "grammar files" here seems to refer to "bindings" files.
|
||||
|
||||
Further evidence in support of this change is [this
|
||||
documentation](https://tree-sitter.github.io/tree-sitter/cli/init.html#binding-files):
|
||||
|
||||
> When you run tree-sitter init, the CLI will also generate a number
|
||||
> of files in your repository that allow for your parser to be used
|
||||
> from different language.
|
||||
|
||||
Which languages bindings files are generated for is affected by [the
|
||||
`bindings` field in
|
||||
`tree-sitter.json`](https://tree-sitter.github.io/tree-sitter/cli/init.html#the-bindings-field).
|
||||
(It appears that omitting the field means "don't generate any
|
||||
bindings".)
|
||||
|
||||
Probably it's better to consult the official documentation and/or ask
|
||||
around about what the latest procedure is rather than rely on these
|
||||
brief notes though.
|
||||
|
||||
## Footnotes
|
||||
|
||||
[1] If the grammar uses an external scanner, `src` may contain
|
||||
non-generated files such as `scanner.c`, `scanner.cc`, etc. In the
|
||||
current case, no external scanner is used and the `src` directory
|
||||
content is entirely generated.
|
||||
|
||||
@ -1,278 +0,0 @@
|
||||
// NOTES
|
||||
//
|
||||
// - possibilities (may be as separate grammars?)
|
||||
// - no fields (but likely that means metadata lives "outside")
|
||||
// - retain whitespace and comments (for round-tripping)
|
||||
// - clojure clr's pipe-escaping:
|
||||
// https://github.com/clojure/clojure-clr/wiki/Specifying-types
|
||||
//
|
||||
// - traveral issues
|
||||
// - use of fields (e.g. value, prefix, tag, metadata)
|
||||
// - allows skipping certain nodes such as:
|
||||
// - metadata
|
||||
// - comment
|
||||
// - discard-related
|
||||
// - allows targeted navigation without having to know the
|
||||
// node type (e.g. field value vs node type map)
|
||||
// - limitations
|
||||
// - a bit slower?
|
||||
// - cannot use fields for things without names, e.g.
|
||||
// - seq(...) cannot be the 2nd arg to field()
|
||||
// - $._foo won't work unless it "resolves" to $.bar (non underscore)
|
||||
// - for a given node, examine child nodes in reverse, that is,
|
||||
// starting at the end and working backwards
|
||||
//
|
||||
// - probably won't do
|
||||
// - support def, if, and other "primitives"
|
||||
// - support for {{}} template constructs
|
||||
//
|
||||
// - testing
|
||||
// - clj, cljc, cljs
|
||||
// - what about edn?
|
||||
// - approaches
|
||||
// - "port" hand-written tests
|
||||
// - oakmac (done)
|
||||
// - Tavistock (done)
|
||||
// - tonsky
|
||||
// - generative testing for token testing (done via hypothesis and py-tree-sitter)
|
||||
// - look for parsing errors across large sample (e.g. clojars) (done)
|
||||
// - how to "package" testing facilities
|
||||
// - currently each approach has its own project directory
|
||||
//
|
||||
// - debugging
|
||||
// - npx tree-sitter parse filepath + look for ERROR in console output
|
||||
// - npx tree-sitter parse --debug-graph filepath + view log.html
|
||||
// - npx tree-sitter parse --debug filepath + view console output
|
||||
//
|
||||
// - loosening ideas:
|
||||
// - allow ##Other (not just ##Inf, -##Inf, ##NaN)
|
||||
// - allow # in keywords
|
||||
// - allow ::/
|
||||
// - don't handle "no repeating colons" in symbols and in non-leading
|
||||
// portions of keywords (currently unimplemented anyway)
|
||||
//
|
||||
// - can strings have unicode escapes in them?
|
||||
//
|
||||
// - tree-sitter
|
||||
// - parse subcommand
|
||||
// - parse from stdin
|
||||
// - recursively traverse multiple directories (globbing exists)
|
||||
// - parsing within zips/jars
|
||||
// - more flexible file type specification
|
||||
// - custom parsing / processing per "file"
|
||||
// - web-ui subcommand
|
||||
// - didn't work when grammar used externals
|
||||
// - file browsing + loading better than copy-paste
|
||||
// - indiciate error via color
|
||||
// - jump to error
|
||||
// - somehow searching for error doesn't seem to work sometimes
|
||||
// - ~/.tree-sitter
|
||||
// - bin
|
||||
// - contains shared libraries for each grammar
|
||||
// - parse command seems to install stuff here
|
||||
// - config.json
|
||||
// - parser-directories used to customize "scan" for grammars
|
||||
// - theme used for highlight subcommand
|
||||
|
||||
// symbolPat from LispReader.java (for keywords and symbols?)
|
||||
// "[:]?([\\D&&[^/]].*/)?(/|[\\D&&[^/]][^/]*)"
|
||||
//
|
||||
// https://clojure.org/reference/reader#_symbols
|
||||
// 1. Symbols begin with a non-numeric char -- XXX: see 2 for limits?
|
||||
// 2. Can contain alphanumeric chars and *, +, !, -, _, ', ?, <, > and =
|
||||
// 3. / can be used once in the middle of a symbol (sep the ns from the name)
|
||||
// 4. / by itself names the division function
|
||||
// 5. . special meaning can be used >= 1 times in the middle of a symbol
|
||||
// to designate a fully-qualified class name, e.g. java.util.BitSet,
|
||||
// or in namespace names.
|
||||
// 6. Symbols beginning or ending with . are reserved by Clojure
|
||||
// 7. Symbols beginning or ending with : are reserved by Clojure
|
||||
// 8. A symbol can contain one or more non-repeating ':'s
|
||||
//
|
||||
// missing
|
||||
// 9. $, &, % -- in body and end of symbol
|
||||
//
|
||||
// undocumented
|
||||
// -1a can be made a symbol, but reader will reject? repl rejects
|
||||
// => number parsing takes priority?
|
||||
// 'a can be made a symbol, but reader will reject? repl -> quote
|
||||
//
|
||||
// implied?
|
||||
// doesn't start with ,
|
||||
// doesn't start with '
|
||||
// doesn't start with #
|
||||
// doesn't start with `
|
||||
// doesn't start with @
|
||||
// doesn't start with ^
|
||||
// doesn't start with \
|
||||
// doesn't start with ;
|
||||
// doesn't start with ~
|
||||
// doesn't start with "
|
||||
// doesn't start with ( )
|
||||
// doesn't start with { }
|
||||
// doesn't start with [ ]
|
||||
//
|
||||
// extra:
|
||||
//
|
||||
// is my-ns// valid?
|
||||
//
|
||||
// "Consistency of symbols between different readers/edn"
|
||||
//
|
||||
// foo// should be valid.
|
||||
//
|
||||
// 2014-09-16 clojure-dev google group alex miller
|
||||
//
|
||||
// https://groups.google.com/d/msg/clojure-dev/b09WvRR90Zc/c3zzMFqDsRYJ
|
||||
//
|
||||
// "CLJ-1238 Allow EdnReader to read foo// (matches LispReader behavior)"
|
||||
//
|
||||
// changelog for clojure 1.6
|
||||
//
|
||||
// is # allowed as a constituent character in keywords?
|
||||
//
|
||||
// following points are reasoning based on edn docs
|
||||
//
|
||||
// "Bug in reader or repl? reading keyword :#abc"
|
||||
//
|
||||
// Symbols begin with a non-numeric character and can contain
|
||||
// alphanumeric characters and . * + ! - _ ? $ % & =. If -, + or
|
||||
// . are the first character, the second character must be
|
||||
// non-numeric. Additionally, : # are allowed as constituent
|
||||
// characters in symbols other than as the first character.
|
||||
//
|
||||
// 2013-05-02 clojure google group colin jones (fwd by dave sann)
|
||||
//
|
||||
// https://groups.google.com/d/msg/clojure/lK7juHxsPCc/TeYjxoW_3csJ
|
||||
//
|
||||
// Keywords are identifiers that typically designate
|
||||
// themselves. They are semantically akin to enumeration
|
||||
// values. Keywords follow the rules of symbols, except they can
|
||||
// (and must) begin with :, e.g. :fred or :my/fred. If the target
|
||||
// platform does not have a keyword type distinct from a symbol
|
||||
// type, the same type can be used without conflict, since the
|
||||
// mandatory leading : of keywords is disallowed for symbols.
|
||||
//
|
||||
// https://github.com/edn-format/edn#symbols
|
||||
//
|
||||
// https://clojure.org/reference/reader#_literals
|
||||
// 0. Keywords are like symbols, except:
|
||||
// 1. They can and must begin with a colon, e.g. :fred.
|
||||
// ~~2. They cannot contain '.' in the name part, or name classes.~~
|
||||
// 3. They can contain a namespace, :person/name, which may contain '.'s.
|
||||
// 4. A keyword that begins with two colons is auto-resolved in the current
|
||||
// namespace to a qualified keyword:
|
||||
// - If the keyword is unqualified, the namespace will be the current
|
||||
// namespace. In user, ::rect is read as :user/rect.
|
||||
// - If the keyword is qualified, the namespace will be resolved using
|
||||
// aliases in the current namespace. In a namespace where x is aliased
|
||||
// to example, ::x/foo resolves to :example/foo.
|
||||
//
|
||||
// extra:
|
||||
//
|
||||
// :/ is a legal keyword(?):
|
||||
//
|
||||
// alexmiller: @gfredericks :/ is "open for the language to start
|
||||
// interpreting" and not an invalid keyword so should be ok to generate.
|
||||
// and cljs should fix it's weirdness. (#clojure-dev 2019-06-07)
|
||||
//
|
||||
// https://clojurians-log.clojureverse.org/clojure-dev/2019-06-07
|
||||
//
|
||||
// It is undefined/left for future expansion.
|
||||
//
|
||||
// Clojurescript's reading seems weird but given that this is undefined
|
||||
// it's hard to say it's wrong. :)
|
||||
//
|
||||
// 2020-07-10 (or so) alexmiller
|
||||
//
|
||||
// https://ask.clojure.org/index.php/9427/clarify-the-position-of-as-a-keyword
|
||||
// https://clojure.atlassian.net/browse/TCHECK-155
|
||||
//
|
||||
// . CAN be in the name part:
|
||||
//
|
||||
// "[Bug?] Keyword constraints not enforced"
|
||||
//
|
||||
// I think you've both misread "they cannot name classes" to be - "They
|
||||
// cannot contain class names".
|
||||
//
|
||||
// The symbol String can name a class but the keyword :String can't,
|
||||
// that's all I meant there.
|
||||
//
|
||||
// As far as '.', that restriction has been relaxed. I'll try to touch
|
||||
// up the docs for the next release.
|
||||
//
|
||||
// 2008-11-25 clojure google group rich hickey
|
||||
//
|
||||
// https://groups.google.com/d/msg/clojure/CCuIp_bZ-ZM/THea7NF91Z4J
|
||||
//
|
||||
// Whether keywords can start with numbers:
|
||||
//
|
||||
// "puzzled by RuntimeException"
|
||||
//
|
||||
// we currently allow keywords starting with numbers and seem to have
|
||||
// decided this is ok. I would like to get Rich to approve a change to
|
||||
// the page and do so.
|
||||
//
|
||||
// 2014-04-25 clojure google group alex miller
|
||||
//
|
||||
// https://groups.google.com/forum/#!msg/clojure/XP1XAaDdKLY/kodfZTk8eeoJ
|
||||
//
|
||||
// From a discussion in #clojure, it emerged that while :foo/1 is
|
||||
// currently not allowed, ::1 is.
|
||||
//
|
||||
// 2014-12-10 nicola mometto
|
||||
//
|
||||
// https://clojure.atlassian.net/browse/CLJ-1286
|
||||
//
|
||||
// "Clarify and align valid symbol and keyword rules for Clojure (and edn)"
|
||||
//
|
||||
// https://clojure.atlassian.net/browse/CLJ-1527
|
||||
//
|
||||
// consistency of symbols between different readers/edn
|
||||
//
|
||||
// https://groups.google.com/forum/#!topic/clojure-dev/b09WvRR90Zc
|
||||
//
|
||||
// :1 is accepted because it once accidentally worked and they
|
||||
// don't like breaking existing code
|
||||
//
|
||||
// it was never meant to
|
||||
//
|
||||
// 2020-06-14 ish noisesmith on #clojure (slack)
|
||||
//
|
||||
// There are libraries out there that assume :1 works. They changed
|
||||
// Clojure at one point in an alpha to disallow such keywords and it broke
|
||||
// code so they decided to continue allowing them (even tho' they are
|
||||
// not "legal").
|
||||
//
|
||||
// 2020-06-14 ish seancorfield on #clojure (slack)
|
||||
//
|
||||
// Whether # is allowed in a keyword:
|
||||
//
|
||||
// "Clarification on # as valid symbol character"
|
||||
//
|
||||
// this works now, but is not guaranteed to always be valid
|
||||
//
|
||||
// 2016-11-08 clojure google group alex miller
|
||||
//
|
||||
// https://groups.google.com/forum/#!topic/clojure/CwZHu1Eszbk
|
||||
|
||||
// https://clojure.org/reference/reader#_literals
|
||||
// 1. Integers can be indefinitely long and will be read as Longs when
|
||||
// in range and clojure.lang.BigInts otherwise.
|
||||
// 2. Integers with an N suffix are always read as BigInts.
|
||||
// 3. When possible, they can be specified in any base with radix from
|
||||
// 2 to 36 (see Long.parseLong()); for example 2r101010, 8r52, 36r16,
|
||||
// and 42 are all the same Long.
|
||||
// 4. Floating point numbers are read as Doubles; with M suffix they are
|
||||
// read as BigDecimals.
|
||||
// 5. Ratios are supported, e.g. 22/7.
|
||||
|
||||
// intPat
|
||||
// "([-+]?)(?:(0)|([1-9][0-9]*)|0[xX]([0-9A-Fa-f]+)|0([0-7]+)|([1-9][0-9]?)[rR]([0-9A-Za-z]+)|0[0-9]+)(N)?"
|
||||
|
||||
// 0[0-9]+ is for better errors -- thanks seancorfield and andyfingerhut
|
||||
|
||||
// ratioPat
|
||||
// "([-+]?[0-9]+)/([0-9]+)"
|
||||
|
||||
// floatPat
|
||||
// "([-+]?[0-9]+(\\.[0-9]*)?([eE][-+]?[0-9]+)?)(M)?"
|
||||
@ -1,19 +0,0 @@
|
||||
{
|
||||
"name": "tree-sitter-clojure",
|
||||
"version": "0.0.11",
|
||||
"lockfileVersion": 1,
|
||||
"requires": true,
|
||||
"dependencies": {
|
||||
"nan": {
|
||||
"version": "2.14.2",
|
||||
"resolved": "https://registry.npmjs.org/nan/-/nan-2.14.2.tgz",
|
||||
"integrity": "sha512-M2ufzIiINKCuDfBSAUr1vWQ+vuVcA9kqx8JJUsbQi6yf1uGRyb7HfpdfUr5qLXf3B/t8dPvcjhKMmlfnP47EzQ=="
|
||||
},
|
||||
"tree-sitter-cli": {
|
||||
"version": "0.19.3",
|
||||
"resolved": "https://registry.npmjs.org/tree-sitter-cli/-/tree-sitter-cli-0.19.3.tgz",
|
||||
"integrity": "sha512-UlntGxLrlkQCKVrhm7guzfi+ovM4wDLVCCu3z5jmfDgFNoUoKa/23ddaQON5afD5jB9a02xv4N5MXJfCx+/mpw==",
|
||||
"dev": true
|
||||
}
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
Loading…
Reference in New Issue