sascha
/
difftastic
mirror of https://github.com/Wilfred/difftastic/
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Update README and adapt docs

pull/856/head
sogaiu 2023-02-28 17:51:28 +07:00
parent c3db36be39
commit 3ccbb02999
3 changed files with 128 additions and 143 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

158
README.md
Unescape Escape View File

@ -1,155 +1,27 @@
# tree-sitter-clojure
## Notice
A tree-sitter grammar for Clojure and ClojureScript
Although no major changes are anticipated at this point, there are no
guarantees. To get a heads-up before such changes occur, please
consider subscribing to the [Potential Changes Announcements
issue](https://github.com/sogaiu/tree-sitter-clojure/issues/33) to be
notified beforehand. The hope is that by commnuicating early enough
about these sorts of things, unnecessary breakage can be avoided
and/or mitigated.
## Status
tree-sitter-clojure has been:
* [Tested in various ways](doc/testing.md)
* [Used in some ways](doc/use.md)
* [Scoped for better behavior](doc/scope.md)
* [Brought about through cooperation](doc/credits.md)
## Prerequisites
Unfortunately, the short of it is that it may be a bit complicated depending on what you want to do.
* If you don't use any of the wasm-related functionality (e.g. previewing parse results in your web browser or you want to build a `.wasm` file for use in a plugin or extension), you probably just need:
* an appropriate version of node (I've tested with various versions >= 12, 14) and
* other typical development-related bits (e.g. git, appropriate c compiler, etc.)
* If you want wasm-related functionality, you get to have fun figuring out which version of [emsdk](https://emscripten.org/docs/getting_started/downloads.html#installation-instructions) currently works with tree-sitter. At the time of this writing, [this file](https://github.com/tree-sitter/tree-sitter/blob/master/cli/emscripten-version) indicates a version that might be appropriate. That may depend on precisely what the versions of other bits (e.g. tree-sitter-cli, web-tree-sitter, etc.) might be though, so if something doesn't work right away, you might consider trying [different versions that have been recorded](https://github.com/tree-sitter/tree-sitter/commits/master/emscripten-version).
Note that there may be an upside to using emsdk though -- it may figure out and arrange for an appropriate version of node, making a separate installation of node unnecessary. I don't use such a setup on a day-to-day basis, but it did work for me at least once.
## Fine Print
* The instructions below assume emsdk has been installed, but `emcc` (tool that can be used to compile to wasm) is not necessarily on one's `PATH`. If an appropriate `emcc` is on one's `PATH` (e.g. emscripten installed via homebrew), the emsdk steps (e.g. `source ~/src/emsdk/emsdk_env.sh`) below may be ignored.
* `node-gyp` (tool for compiling native addon modules for Node.js) may fail on machines upgraded to macos Catalina. [This document](https://github.com/nodejs/node-gyp/blob/master/macOS_Catalina.md) may help cope with such a situation.
## Initial Setup
Suppose typical development sources are stored under `~/src`.
### Short Version
```
# clone repository
cd ~/src
git clone https://github.com/sogaiu/tree-sitter-clojure
cd tree-sitter-clojure
# install tree-sitter-cli and dependencies, then build
npm ci
```
### Long Version
```
# clone repository
cd ~/src
git clone https://github.com/sogaiu/tree-sitter-clojure
cd tree-sitter-clojure
# ensure tree-sitter-cli is avaliable as a dev dependency
npm install --save-dev --save-exact tree-sitter-cli
# create `src` and populate with tree-sitter `.c` goodness
npx tree-sitter generate
## What the Repository Provides
# populate `node_modules` with dependencies
npm install
This repository provides some files used to create various artifacts
(e.g. dynamic libraries) used for handling Clojure and ClojureScript
source code via tree-sitter.
# create `build` and populate appropriately
npx node-gyp configure
Please see the [what and why document](doc/what-and-why.md) for
detailed information.
# create `build/Release` and build `tree_sitter_clojure_binding.node`
npx node-gyp rebuild
```
## Potential Changes Announcements
## Grammar Development
Hack on grammar.
```
# edit grammar.js using some editor
# rebuild tree-sitter stuff
npx tree-sitter generate && \
npx node-gyp rebuild
```
Parse individual files.
```
# create and populate sample code file for parsing named `sample.clj`
# parse sample file
npx tree-sitter parse sample.clj
# if output has errors, figure out what's wrong
```
Interactively test in the browser (requires emsdk).
```
# prepare emsdk (specifically emcc) for building .wasm
source ~/src/emsdk/emsdk_env.sh
# build .wasm bits and invoke web-ui for interactive testing
npx tree-sitter build-wasm && \
npx tree-sitter web-ui
# in appropriate browser window, paste code in left pane
# examine results in right pane -- can even click on nodes
# if output has errors, figure out what's wrong
```
## Measure Performance
```
# single measurement
npx tree-sitter parse --time sample.clj
# mutliple measurements with `multitime`
multitime -n10 -s1 npx tree-sitter parse --time --quiet sample.clj
```
## Build .wasm
Assuming emsdk is installed appropriately under `~/src/emsdk`.
```
# prepare emsdk (specifically emcc) for use
source ~/src/emsdk/emsdk_env.sh
# create `tree-sitter-clojure.wasm`
npx tree-sitter build-wasm
```
Although no major changes are anticipated at this point, there are no
guarantees. To get a heads-up before such changes occur, please
consider subscribing to the [Potential Changes Announcements
issue](https://github.com/sogaiu/tree-sitter-clojure/issues/33) to be
notified beforehand.
## Resources
## Other Documents
* [Guide to your first Tree-sitter grammar](https://gist.github.com/Aerijo/df27228d70c633e088b0591b8857eeef)
* [sublime-clojure](https://github.com/tonsky/sublime-clojure)
* [syntax-highlighter](https://github.com/EvgeniyPeshkov/syntax-highlighter)
* [tree-sitter](http://tree-sitter.github.io/tree-sitter/)
* [tree-sitter-clojure.oakmac](https://github.com/oakmac/tree-sitter-clojure)
* [tree-sitter-clojure.SergeevPavel](https://github.com/SergeevPavel/tree-sitter-clojure)
* [tree-sitter-clojure.Tavistock](https://github.com/Tavistock/tree-sitter-clojure)
* [vscode-tree-sitter](https://github.com/georgewfraser/vscode-tree-sitter)
* [web-tree-sitter API](https://github.com/tree-sitter/tree-sitter/blob/master/lib/binding_web/tree-sitter-web.d.ts)
Please see the [`doc` directory](doc/).
## Acknowledgments

17
doc/resources.md
Unescape Escape View File

@ -0,0 +1,17 @@
# Resources
Below is a list of resources related to tree-sitter and/or Clojure.
Some may be a bit dated at this point.
* [Guide to your first Tree-sitter
grammar](https://gist.github.com/Aerijo/df27228d70c633e088b0591b8857eeef)
* [sublime-clojure](https://github.com/tonsky/sublime-clojure)
* [syntax-highlighter](https://github.com/EvgeniyPeshkov/syntax-highlighter)
* [tree-sitter](http://tree-sitter.github.io/tree-sitter/)
* [tree-sitter-clojure.oakmac](https://github.com/oakmac/tree-sitter-clojure)
* [tree-sitter-clojure.SergeevPavel](https://github.com/SergeevPavel/tree-sitter-clojure)
* [tree-sitter-clojure.Tavistock](https://github.com/Tavistock/tree-sitter-clojure)
* [vscode-tree-sitter](https://github.com/georgewfraser/vscode-tree-sitter)
* [web-tree-sitter
API](https://github.com/tree-sitter/tree-sitter/blob/master/lib/binding_web/tree-sitter-web.d.ts)

96
doc/what-and-why.md
Unescape Escape View File

@ -0,0 +1,96 @@
# What the Repository Provides and Why
This document describes what files and directories the repository
provides and associated reasoning. First it covers the bits that are
likely to be around in the near future. This is followed by a
description of things that may change or be removed.
One might be interested in this content out of academic curiosity but
more likely it might be because one is thinking of depending on the
repository in some way.
## What and Why
The following is a list of files and directories that are likely to
remain "provided" by the repository unless significant changes occur
elsewhere (e.g. upstream tree-sitter changes what / how they do things
or other grammar repositories decide to change certain practices).
The order is alphabetical and not meant to reflect relative
importance.
* `CHANGELOG.md` - this file contains a changelog.
* `COPYING.txt` - this file contains license information for the
repository.
* `grammar.js` - this file contains a grammar description and is used
in the process of generating parser source code that lives in `src`.
It's likely that this (or something comparable) will continue to be
provided assuming tree-sitter doesn't change the way it works.
* `package.json` - this file is needed by a
[component](https://github.com/cursorless-dev/vscode-parse-tree/) of
[Cursorless](https://www.cursorless.org/). It uses our grammar via
yarn and `package.json` seems to be essential [1].
* `queries` - this directory and the simple file it contains are
provided on request from
[`difftastic`](https://github.com/Wilfred/difftastic) folks. The
file it contains doesn't contain much and is not likely to be the
sort of thing one expects to be used in an editor.
* `README.md` - this file contains the repository's README content.
* `src` - this directory contains source files that are generated [2]
from `grammar.js`. The files are typically used to generate a
dynamic library / shared object that can be used by the tree-sitter
library to handle Clojure / ClojureScript source code. Although the
content of this directory is generated, the files are provided
because in practice, multiple parties have already become dependant
on them. There have been opinions voiced that this should not
remain so, but change in that direction has not been widespread
AFAICT. See
[here](https://github.com/sogaiu/ts-questions/blob/master/questions/should-parser-source-be-committed/README.md)
for more on the topic if interested.
* `test` - this directory contains test-related information. It
exists partly for tree-sitter's corpus test functionality to work,
but there is also other content that is used for other testing.
## Other Content
The rest of the content of the repository is subject to change /
removal so depending on such remaining in place is likely not a good
idea.
ATM, some of that includes:
* documentation
* Node.js bindings
* Rust bindings
The bindings are not tested in any way and only really exist as a
side-effect of running the `tree-sitter` cli `generate` subcommand.
They may be removed at some point, but they should be straight-forward
to generate as long as one has a suitable `tree-sitter` cli and the
`grammar.js` file mentioned above.
## Footnotes
[1] The file `package.json` may also be required if it's important to
use some of the capabilities of the `tree-sitter` cli such as the
`tags` and `highlight` subcommands (which we don't typically use).
It's not necessary for all subcommands though (e.g. neither the
`generate` nor `test` subcommands seem to require it). Its presence
also doesn't signify necessary use of `npm`.
Possibly contrary to what might be indicated elsewhere, `npm` is
not necessary for certain core parts of tree-sitter grammar
development. However, at the moment, an appropriate version of `node`
_is_ required for the `generate` subcommand to work.
[2] If the grammar uses an external scanner, `src` may contain
non-generated files such as `scanner.c`, `scanner.cc`, etc. In the
current case, no external scanner is used and the `src` directory
content is entirely generated.