Replace tabs with spaces before parsing

Difftastic uses offsets to store the positions of AST nodes, so we
cannot replace tabs after parsing.

A string literal with a tab character "	" being replaced by a four
space string literal "    " is now ignored by difftastic. This is
wrong: difftastic should only ignore whitespace between atoms.

This is still an improvement for source code that uses tab characters,
which is common in Go. The above case should be rare until we have a
full solution.

Fixes #24
ida_star
Wilfred Hughes 2021-07-27 23:35:49 +07:00
parent a7f688ee82
commit 6f2140e901
2 changed files with 6 additions and 2 deletions

@ -24,6 +24,9 @@ lines.
Improved horizontal spacing between before and after code shown.
Fixed an issue where source code containing tab characters was not
correctly aligned.
### Command Line Interface
Removed unused `--inline` and `--context` arguments.

@ -94,8 +94,9 @@ fn main() {
return;
}
let lhs_src = String::from_utf8_lossy(&lhs_bytes).to_string();
let rhs_src = String::from_utf8_lossy(&rhs_bytes).to_string();
// TODO: don't replace tab characters inside string literals.
let lhs_src = String::from_utf8_lossy(&lhs_bytes).to_string().replace("\t", " ");
let rhs_src = String::from_utf8_lossy(&rhs_bytes).to_string().replace("\t", " ");
let terminal_width = match matches.value_of("COLUMNS") {
Some(width) => width.parse::<usize>().unwrap(),