From 3ccbb02999c9611d4fdcdd8406e88bb94f2e6221 Mon Sep 17 00:00:00 2001 From: sogaiu <983021772@users.noreply.github.com> Date: Tue, 28 Feb 2023 17:51:28 +0900 Subject: [PATCH] Update README and adapt docs --- README.md | 158 +++++--------------------------------------- doc/resources.md | 17 +++++ doc/what-and-why.md | 96 +++++++++++++++++++++++++++ 3 files changed, 128 insertions(+), 143 deletions(-) create mode 100644 doc/resources.md create mode 100644 doc/what-and-why.md diff --git a/README.md b/README.md index 8a9300ab2..fa6c0c032 100644 --- a/README.md +++ b/README.md @@ -1,155 +1,27 @@ # tree-sitter-clojure -## Notice +A tree-sitter grammar for Clojure and ClojureScript -Although no major changes are anticipated at this point, there are no -guarantees. To get a heads-up before such changes occur, please -consider subscribing to the [Potential Changes Announcements -issue](https://github.com/sogaiu/tree-sitter-clojure/issues/33) to be -notified beforehand. The hope is that by commnuicating early enough -about these sorts of things, unnecessary breakage can be avoided -and/or mitigated. - -## Status - -tree-sitter-clojure has been: - -* [Tested in various ways](doc/testing.md) -* [Used in some ways](doc/use.md) -* [Scoped for better behavior](doc/scope.md) -* [Brought about through cooperation](doc/credits.md) - -## Prerequisites - -Unfortunately, the short of it is that it may be a bit complicated depending on what you want to do. - -* If you don't use any of the wasm-related functionality (e.g. previewing parse results in your web browser or you want to build a `.wasm` file for use in a plugin or extension), you probably just need: - * an appropriate version of node (I've tested with various versions >= 12, 14) and - * other typical development-related bits (e.g. git, appropriate c compiler, etc.) - -* If you want wasm-related functionality, you get to have fun figuring out which version of [emsdk](https://emscripten.org/docs/getting_started/downloads.html#installation-instructions) currently works with tree-sitter. At the time of this writing, [this file](https://github.com/tree-sitter/tree-sitter/blob/master/cli/emscripten-version) indicates a version that might be appropriate. That may depend on precisely what the versions of other bits (e.g. tree-sitter-cli, web-tree-sitter, etc.) might be though, so if something doesn't work right away, you might consider trying [different versions that have been recorded](https://github.com/tree-sitter/tree-sitter/commits/master/emscripten-version). - -Note that there may be an upside to using emsdk though -- it may figure out and arrange for an appropriate version of node, making a separate installation of node unnecessary. I don't use such a setup on a day-to-day basis, but it did work for me at least once. - -## Fine Print - -* The instructions below assume emsdk has been installed, but `emcc` (tool that can be used to compile to wasm) is not necessarily on one's `PATH`. If an appropriate `emcc` is on one's `PATH` (e.g. emscripten installed via homebrew), the emsdk steps (e.g. `source ~/src/emsdk/emsdk_env.sh`) below may be ignored. - -* `node-gyp` (tool for compiling native addon modules for Node.js) may fail on machines upgraded to macos Catalina. [This document](https://github.com/nodejs/node-gyp/blob/master/macOS_Catalina.md) may help cope with such a situation. - -## Initial Setup - -Suppose typical development sources are stored under `~/src`. - -### Short Version - -``` -# clone repository -cd ~/src -git clone https://github.com/sogaiu/tree-sitter-clojure -cd tree-sitter-clojure - -# install tree-sitter-cli and dependencies, then build -npm ci -``` - -### Long Version - -``` -# clone repository -cd ~/src -git clone https://github.com/sogaiu/tree-sitter-clojure -cd tree-sitter-clojure - -# ensure tree-sitter-cli is avaliable as a dev dependency -npm install --save-dev --save-exact tree-sitter-cli - -# create `src` and populate with tree-sitter `.c` goodness -npx tree-sitter generate +## What the Repository Provides -# populate `node_modules` with dependencies -npm install +This repository provides some files used to create various artifacts +(e.g. dynamic libraries) used for handling Clojure and ClojureScript +source code via tree-sitter. -# create `build` and populate appropriately -npx node-gyp configure +Please see the [what and why document](doc/what-and-why.md) for +detailed information. -# create `build/Release` and build `tree_sitter_clojure_binding.node` -npx node-gyp rebuild -``` +## Potential Changes Announcements -## Grammar Development - -Hack on grammar. - -``` -# edit grammar.js using some editor - -# rebuild tree-sitter stuff -npx tree-sitter generate && \ -npx node-gyp rebuild -``` - -Parse individual files. - -``` -# create and populate sample code file for parsing named `sample.clj` - -# parse sample file -npx tree-sitter parse sample.clj - -# if output has errors, figure out what's wrong -``` - -Interactively test in the browser (requires emsdk). - -``` -# prepare emsdk (specifically emcc) for building .wasm -source ~/src/emsdk/emsdk_env.sh - -# build .wasm bits and invoke web-ui for interactive testing -npx tree-sitter build-wasm && \ -npx tree-sitter web-ui - -# in appropriate browser window, paste code in left pane - -# examine results in right pane -- can even click on nodes - -# if output has errors, figure out what's wrong -``` - -## Measure Performance - -``` -# single measurement -npx tree-sitter parse --time sample.clj - -# mutliple measurements with `multitime` -multitime -n10 -s1 npx tree-sitter parse --time --quiet sample.clj -``` - -## Build .wasm - -Assuming emsdk is installed appropriately under `~/src/emsdk`. - -``` -# prepare emsdk (specifically emcc) for use -source ~/src/emsdk/emsdk_env.sh - -# create `tree-sitter-clojure.wasm` -npx tree-sitter build-wasm -``` +Although no major changes are anticipated at this point, there are no +guarantees. To get a heads-up before such changes occur, please +consider subscribing to the [Potential Changes Announcements +issue](https://github.com/sogaiu/tree-sitter-clojure/issues/33) to be +notified beforehand. -## Resources +## Other Documents -* [Guide to your first Tree-sitter grammar](https://gist.github.com/Aerijo/df27228d70c633e088b0591b8857eeef) -* [sublime-clojure](https://github.com/tonsky/sublime-clojure) -* [syntax-highlighter](https://github.com/EvgeniyPeshkov/syntax-highlighter) -* [tree-sitter](http://tree-sitter.github.io/tree-sitter/) -* [tree-sitter-clojure.oakmac](https://github.com/oakmac/tree-sitter-clojure) -* [tree-sitter-clojure.SergeevPavel](https://github.com/SergeevPavel/tree-sitter-clojure) -* [tree-sitter-clojure.Tavistock](https://github.com/Tavistock/tree-sitter-clojure) -* [vscode-tree-sitter](https://github.com/georgewfraser/vscode-tree-sitter) -* [web-tree-sitter API](https://github.com/tree-sitter/tree-sitter/blob/master/lib/binding_web/tree-sitter-web.d.ts) +Please see the [`doc` directory](doc/). ## Acknowledgments diff --git a/doc/resources.md b/doc/resources.md new file mode 100644 index 000000000..f8388ba4b --- /dev/null +++ b/doc/resources.md @@ -0,0 +1,17 @@ +# Resources + +Below is a list of resources related to tree-sitter and/or Clojure. + +Some may be a bit dated at this point. + +* [Guide to your first Tree-sitter + grammar](https://gist.github.com/Aerijo/df27228d70c633e088b0591b8857eeef) +* [sublime-clojure](https://github.com/tonsky/sublime-clojure) +* [syntax-highlighter](https://github.com/EvgeniyPeshkov/syntax-highlighter) +* [tree-sitter](http://tree-sitter.github.io/tree-sitter/) +* [tree-sitter-clojure.oakmac](https://github.com/oakmac/tree-sitter-clojure) +* [tree-sitter-clojure.SergeevPavel](https://github.com/SergeevPavel/tree-sitter-clojure) +* [tree-sitter-clojure.Tavistock](https://github.com/Tavistock/tree-sitter-clojure) +* [vscode-tree-sitter](https://github.com/georgewfraser/vscode-tree-sitter) +* [web-tree-sitter + API](https://github.com/tree-sitter/tree-sitter/blob/master/lib/binding_web/tree-sitter-web.d.ts) diff --git a/doc/what-and-why.md b/doc/what-and-why.md new file mode 100644 index 000000000..40cc99f0a --- /dev/null +++ b/doc/what-and-why.md @@ -0,0 +1,96 @@ +# What the Repository Provides and Why + +This document describes what files and directories the repository +provides and associated reasoning. First it covers the bits that are +likely to be around in the near future. This is followed by a +description of things that may change or be removed. + +One might be interested in this content out of academic curiosity but +more likely it might be because one is thinking of depending on the +repository in some way. + +## What and Why + +The following is a list of files and directories that are likely to +remain "provided" by the repository unless significant changes occur +elsewhere (e.g. upstream tree-sitter changes what / how they do things +or other grammar repositories decide to change certain practices). +The order is alphabetical and not meant to reflect relative +importance. + +* `CHANGELOG.md` - this file contains a changelog. + +* `COPYING.txt` - this file contains license information for the + repository. + +* `grammar.js` - this file contains a grammar description and is used + in the process of generating parser source code that lives in `src`. + It's likely that this (or something comparable) will continue to be + provided assuming tree-sitter doesn't change the way it works. + +* `package.json` - this file is needed by a + [component](https://github.com/cursorless-dev/vscode-parse-tree/) of + [Cursorless](https://www.cursorless.org/). It uses our grammar via + yarn and `package.json` seems to be essential [1]. + +* `queries` - this directory and the simple file it contains are + provided on request from + [`difftastic`](https://github.com/Wilfred/difftastic) folks. The + file it contains doesn't contain much and is not likely to be the + sort of thing one expects to be used in an editor. + +* `README.md` - this file contains the repository's README content. + +* `src` - this directory contains source files that are generated [2] + from `grammar.js`. The files are typically used to generate a + dynamic library / shared object that can be used by the tree-sitter + library to handle Clojure / ClojureScript source code. Although the + content of this directory is generated, the files are provided + because in practice, multiple parties have already become dependant + on them. There have been opinions voiced that this should not + remain so, but change in that direction has not been widespread + AFAICT. See + [here](https://github.com/sogaiu/ts-questions/blob/master/questions/should-parser-source-be-committed/README.md) + for more on the topic if interested. + +* `test` - this directory contains test-related information. It + exists partly for tree-sitter's corpus test functionality to work, + but there is also other content that is used for other testing. + +## Other Content + +The rest of the content of the repository is subject to change / +removal so depending on such remaining in place is likely not a good +idea. + +ATM, some of that includes: + +* documentation +* Node.js bindings +* Rust bindings + +The bindings are not tested in any way and only really exist as a +side-effect of running the `tree-sitter` cli `generate` subcommand. +They may be removed at some point, but they should be straight-forward +to generate as long as one has a suitable `tree-sitter` cli and the +`grammar.js` file mentioned above. + +## Footnotes + +[1] The file `package.json` may also be required if it's important to +use some of the capabilities of the `tree-sitter` cli such as the +`tags` and `highlight` subcommands (which we don't typically use). + +It's not necessary for all subcommands though (e.g. neither the +`generate` nor `test` subcommands seem to require it). Its presence +also doesn't signify necessary use of `npm`. + +Possibly contrary to what might be indicated elsewhere, `npm` is +not necessary for certain core parts of tree-sitter grammar +development. However, at the moment, an appropriate version of `node` +_is_ required for the `generate` subcommand to work. + +[2] If the grammar uses an external scanner, `src` may contain +non-generated files such as `scanner.c`, `scanner.cc`, etc. In the +current case, no external scanner is used and the `src` directory +content is entirely generated.