Commit Graph

227 Commits (c6be0bc16a95319f210820f0e488e5c9e87d5f6b)

Author SHA1 Message Date
Wilfred Hughes 0af76db498 Update regression tests for humansize file formatting 2023-08-13 09:10:16 +07:00
Wilfred Hughes 757c297412 Adjust header style
Show the hunk count and detected language in a dimmed style. This
information is less important than the diff content itself, so this
change makes the important information more prominent.

First part of #544
2023-07-31 08:35:27 +07:00
Wilfred Hughes 797af40ae8 Improve Java highlighting 2023-07-27 08:33:38 +07:00
Wilfred Hughes 7caaaf7fcf Handle nested sliders correctly when preferring the outer delimiter
Previously we didn't check the state of children, which was an
oversight from the original implementation. As a result, we fixed
nested sliders in fewer situations.

Fixes #535
2023-07-14 08:49:55 +07:00
Wilfred Hughes 5606c04261 Treat qualified modules and variables as atoms in Haskell 2023-07-12 12:34:39 +07:00
Wilfred Hughes a814e01d22 Improve word diffing heuristic and add another sample file 2023-07-12 12:12:32 +07:00
Wilfred Hughes c2b7042b80 Do subword highlighting in more cases
This is useful when two strings substantially differ, but have the
same e.g. end.
2023-07-10 21:26:24 +07:00
Wilfred Hughes 5824322244 Require some common words to do subword highlighting
This is important when comparing short string literals. This change
has improved several cases in sample_files/ but I've added a new
example that made the previous unwanted behaviour much more obvious.
2023-07-10 09:03:21 +07:00
Wilfred Hughes 27f59c0b3a Don't treat - as a word constituent
This produces slightly better results with some string replacements.
2023-07-08 17:16:14 +07:00
Zhenge Chen ffd49d523a Detect replaced strings
If a string is replaced with another, apply subword highlighting
similar to how we handle replaced comments.

Co-authored-by: Wilfred Hughes <me@wilfred.me.uk>
2023-07-08 17:16:06 +07:00
Wilfred Hughes 87d27c5598 Only split numbers inside comments
Inside text files, it seems to be better to be conservative and
consider abc123def as one word rather than three.

This is noticeable when looking at changes to the compare.expected
file, which contains hashes. 123c456 and 345c789 don't really have a
`c` in common, so subword highlighting is ugly.
2023-07-07 08:40:06 +07:00
Wilfred Hughes c07e640b24 Remove contiguous penalty
The contiguous penalty was an attempt to fix the slider problem:

// Old
A B
C D

// New
A B
A B
C D

// Unwanted diff
A +B+
+A+ B
C D

However, it doesn't make sense for Dijkstra, which is stateless. The
best route from vertex X is independent of how we got to vertex X.

This worked by dumb luck: in some circumstances we terminate early
rather than fully executing Dijkstra's algorithm. This cost tweak
improved results on a few test files. However, the post-processing
slider logic is a proper, general solution. This was added much later.

There's no reason to keep the contiguous penalty now. It's confusing,
and makes adding new edge costs with consistent 'X costs more than Y'
behaviours more difficult.

Performance is essentially neutral: a small decrease in
typing_before.ml, a small increase in slow_before.rs.
2023-07-06 08:37:02 +07:00
Wilfred Hughes 43c24047b4 Don't track contiguous status on novel delimiter edges
This is harder to reason about, and
2e6666041f did not include a motivating
test case.

Removing contiguous status is a minor perf improvement (2% reduction
in instructions), makes the code simpler, and does not significantly
affect diffing results.

Of the two sample files that have changed, the erlang_before.erl file
has improved and nest_before.rs is neutral.
2023-07-04 23:53:16 +07:00
Wilfred Hughes c405b58327 Fix cost for ReplacedComment
This needs to be 2x novel nodes, or we prefer it far too often.
2023-07-02 23:12:31 +07:00
Wilfred Hughes 3730580ca3 Improve word splitting heuristics
This is particularly noticeable when diffing comments with timestamps
2000-12-31T23:59:59 where we don't want 31T23 to be a single word.
2023-06-29 08:33:30 +07:00
Wilfred Hughes 8b842387a1 Don't clean trailing newline before diffing
Difftastic should take the user's input as-is, or it risks performing
an incorrect diff in both textual and syntactic diffing.

Fixes #499
2023-03-30 08:46:11 +07:00
Wilfred Hughes 0ec2f3a319 Add test case that reproduces #499 2023-03-24 23:19:13 +07:00
Wilfred Hughes 3263612150 Update expected output for latest QML grammar 2023-03-17 00:55:59 +07:00
Wilfred Hughes 7fbe0d6c2f Improve UTF-16 detection heuristics and add test 2023-03-16 00:31:58 +07:00
Wilfred Hughes 3e4df7d7dd Add CLI test for the example in #433 2023-03-16 00:13:41 +07:00
Wilfred Hughes a0f7ed5e78 Set explicit locales for ordering globs in integration test 2023-03-15 15:47:27 +07:00
Karl Ding 4a861c376e Manually order expected sample comparison file
It appears that the GitHub Actions runner is returning the glob path
results in a different ordering than the ordering obtained when locally
running the compare_all.sh script. This difference in the ordering
causes CI to fail due to differences to the generated expectation file.

This also seems to have been an issue in previous PRs---the solution
here is likely to sort the output before processing or figure out what
shell options cause the difference in glob ordering, and explicitly set
those in the shell script to eliminate the difference (or prevent the
script from inheriting anything but shell defaults).

For now, try reordering the output by hand to match the ordering the
GitHub Action runner likely expects.
2023-03-14 21:46:40 +07:00
Karl Ding 6b2947b4e9 Add Ada "Hello World" sample file 2023-03-14 21:46:40 +07:00
Wilfred Hughes 2d1a2c906e Count errors on the root node too
Fixes #377
2023-03-03 00:25:41 +07:00
Wilfred Hughes 045d6a2c58 Treat Newick and Racket as lisps 2023-03-03 00:23:11 +07:00
Wilfred Hughes 03985066f5 Treat Makefile text as atoms
Improves another case identified in #476
2023-03-02 23:52:01 +07:00
Wilfred Hughes 9556cd978e Merge branch 'delehef/master' 2023-02-21 08:46:07 +07:00
Franklin Delehelle 21ded51e90 Add newick example files 2023-02-21 08:45:49 +07:00
6cdh 5e659e2d98 add racket highlight 2023-02-21 08:30:00 +07:00
6cdh 3c418168a4 fix bug in here_string 2023-02-12 16:45:38 +07:00
6cdh 2fade3e0bf fix here string 2023-02-12 13:57:45 +07:00
6cdh fe756905bf added Racket support 2023-02-12 13:39:58 +07:00
Wilfred Hughes 96fc044e6d Improve syntax highlighting
`@include` and `@exception` are both used for highlighting keywords in
several languages.
2023-02-10 08:50:43 +07:00
Wilfred Hughes 0f1d323bf3 Disable lua-match? highlighting predicates
This fixes inaccurate type highlighting in Scala.

See #310
2023-02-10 08:45:41 +07:00
Wilfred Hughes 5db907f393
Merge pull request #481 from hugo-vrijswijk/master
Update tree-sitter-scala
2023-02-08 17:52:46 +07:00
Wilfred Hughes df6f4618b6 Update expected output for 62da0d56cc 2023-02-08 17:51:28 +07:00
Wilfred Hughes 18b46812c0 Update C++ expected output for 63cf71641a 2023-02-08 17:49:05 +07:00
Hugo van Rijswijk afad33c71e update compare.expected 2023-02-08 11:16:19 +07:00
Wilfred Hughes 7d5afd78dc Respect --error-limit when parsing
Next step for #472
2023-02-04 22:29:18 +07:00
Wilfred Hughes 5ed4bac8a5 Add support for R
Fixes #470
2023-01-26 08:50:00 +07:00
Wilfred Hughes 9ce60140ce Update screenshot and ensure sample files match 2023-01-22 20:19:36 +07:00
Wilfred Hughes 0e3c57c64a Skip unique items before computing Myer's diff on text
This substantially improves performance on text files where there are
few lines in common.

For example, 10,000 line files with no lines in common is more than 10x
faster (8.5 seconds to 0.49 seconds on my machine), and
sample_files/huge_cpp_before.cpp is nearly 2% faster.

Fixes the case mentioned by @quackenbush in #236.

This is inspired by the heuristics discussions at
https://github.com/mitsuhiko/similar/issues/15
2023-01-15 11:38:02 +07:00
Wilfred Hughes efec759504 Only set language_used after a full syntactic diff
This fixes cases where the language is detected but the file hits the
byte limit.

Fixes #462.
2023-01-14 12:52:08 +07:00
Wilfred Hughes 63a3bf0c91 Ensure we use the correct config for sublanguage parsing
Otherwise get the wrong node names for atoms.
2023-01-08 22:24:43 +07:00
Wilfred Hughes 8ed4fbccfa Treat colour values (e.g. `#FFF`) as atoms in CSS 2023-01-08 22:22:46 +07:00
Wilfred Hughes 34967f588d Treat predefined_type as an atom in TypeScript
Currently it contains a nested string node, even though it's a fixed
set of known types. This was preventing us from applying good syntax
highlighting.

This was particularly noticeable with `string`, which wasn't
previously highlighted as a type.
2023-01-07 22:43:50 +07:00
Wilfred Hughes cd87796552 Treat doctype nodes as atoms in HTML
The tree-sitter parser doesn't include the text after DOCTYPE in the
inner tag.
2023-01-03 08:40:39 +07:00
Steinar H. Gunderson 9133918dd4 Support parsing of sub-languages.
This allows given nodes (configurable per-language, using tree-sitter's
query syntax) to be re-parsed as other languages. The canonical example
is CSS or JavaScript inside HTML, which normally would be a single token
but now can get the full range of syntax highlighting and tree diffing.

The config sets this up for only two languages: HTML (contains CSS or
JavaScript in <script> or <style> tags; we don't support style="" or
onclick="" etc. at this point), and Makefiles (contains Bash in
$(shell ...) commands). The latter is fairly obscure; the big win is
in the former.

It would be nice to also have this support for PHP; however, the HTML
parser seems to be a bit confused when asked to parse the partial HTML
blocks we get if we just mark the "text" blocks as HTML, so for this
to work well, probably the PHP blocks should be parsed as sub-languages
of HTML instead of vice versa.

Also, as a minor quibble, there should be support for bash in Perl's
backticks (similar to in Makefiles), but the tree-sitter Perl parser
does not support backticks at all (it goes into error recovery).

There may have been languages that I've missed, e.g. some languages
might have nodes that contain e.g. SQL.

Fixes #382. Potentially relevant to #376.
2023-01-03 08:31:48 +07:00
Wilfred Hughes 0fc1842595 Improve word highlighting heuristics in comments
Previously we highlighted changed whitespace, which led to ugly
results if the number of words changed (there was a different number
of whitespace characters so some were highlighted).

Also treat _ and - as word constituents, as it produces nicer results
when people write example CLI invocations in comments.
2023-01-02 16:56:31 +07:00
Wilfred Hughes e8e5ca8e47 Replace tabs during display, so parsing sees the original source
Fixes #350
2023-01-01 22:44:47 +07:00