Comparing Text: A Diff Workflow That Catches Real Changes

Pick the granularity before you read

Every diff tool answers the question "what is the smallest set of insertions and deletions that turns A into B", but the unit it works in changes the answer completely. Line granularity is the default because source control works that way, and it is right for code, configuration and logs, where a line is a meaningful unit. It is actively misleading for a long single-line value: change one character in a minified bundle or a Base64 payload and a line diff reports that the entire file was replaced.

Character granularity is the opposite trade. It finds the one digit that changed in an identifier, the swapped letters in a hostname, the invisible character someone pasted from a document. On anything longer than a few lines it produces a speckled mess that is harder to read than the originals. Word granularity sits between the two and is the right default for prose, translations and documentation, where a sentence rewritten with different punctuation should not look like a deleted paragraph.

The useful habit is to compare twice when something looks wrong. Read the line diff to understand the shape of the change, then re-run the comparison at character level on the specific lines that matter. A tool that lets you switch granularity without re-pasting is worth more than one with a better algorithm, because the second look is where the actual bug is usually found.

Normalise the noise before you judge the change

Most disappointing diffs are dominated by changes nobody made deliberately. A file edited on Windows and committed from macOS differs on every line because CRLF became LF. A formatter run with different settings reindents a whole file. An editor strips trailing whitespace on save. A byte-order mark appears at the top of a file that a text editor rewrote. In each case the real change is one or two lines, buried in hundreds.

Deal with this by suppressing the known-irrelevant differences rather than by reading past them. Turn on whitespace-insensitive comparison, or normalise line endings, before you read anything. If your tool shows a whitespace-only change indicator, trust it and move on. In source control, the same job is done by ignoring whitespace in the review, and by keeping a formatter configuration in the repository so the noise never gets generated in the first place.

Be careful about one asymmetry, though: whitespace is not always noise. It is significant inside string literals, in Markdown where two trailing spaces are a line break, in YAML and Python where indentation is syntax, in Makefiles where a tab is required, and in test fixtures that assert on exact output. Suppress whitespace to find the change, then look at the raw bytes of the specific line before concluding that the change was harmless.

• Line endings: CRLF versus LF, often shown as a trailing ^M or as every line changed.
• A missing final newline, which git reports as a backslash-prefixed note at the end of the hunk.
• A byte-order mark on the first line, invisible in most editors.
• Tabs converted to spaces, or an indentation width change from a different formatter configuration.
• Non-breaking spaces and typographic quotes pasted in from a document or a chat client.

Reading a unified diff

The unified format is what git, patch, code review tools and most diff utilities emit, and the part people skim is the part that carries the information. A hunk begins with a header of the form @@ -12,7 +12,8 @@. The first pair is the starting line and line count in the original file, the second pair is the starting line and line count in the new file. A count that grows by one means the hunk added a net line; counts that match mean lines were only modified or moved within the hunk.

Inside the hunk, a leading space is context, a minus sign is a line only in the original, and a plus sign is a line only in the new version. There is no "changed" marker: a modified line always appears as a deletion immediately followed by an insertion, which is why a one-character fix looks like two lines of churn. Some tools append the enclosing function or section name after the closing @@, which is the fastest way to orient yourself in a long file.

Two conventions are worth recognising because they look like errors and are not. A note reading "No newline at end of file" means exactly that and is usually worth fixing rather than ignoring. And the context size is configurable — three lines by default — so a hunk that seems to lack the surrounding code you wanted is a flag away from showing it.

--- a/app.yaml          old file
+++ b/app.yaml          new file
@@ -12,7 +12,8 @@ server:
    |  |  |  |  |    |
    |  |  |  |  |    +-- enclosing section, for orientation
    |  |  |  |  +------- new file: 8 lines in this hunk
    |  |  |  +---------- new file: hunk starts at line 12
    |  |  +------------- old file: 7 lines in this hunk
    |  +---------------- old file: hunk starts at line 12

   host: 0.0.0.0        leading space = context, in both files
   port: 8080
   timeout: 30
-  workers: 4           minus = present only in the old file
+  workers: 8           plus  = present only in the new file
+  keepalive: 75

 logging:
   level: info

7 old lines, 8 new: one line modified (shown as - then +) and one added.

Moved blocks, and the thing a diff cannot say

A line-based diff has no concept of movement. If a forty-line function is relocated from the top of a file to the bottom, the algorithm sees forty deletions and forty insertions, and a reviewer sees eighty lines of change where nothing changed at all. Worse, the reverse is equally invisible: if that function was moved and one line inside it was also edited, the edit sits in the middle of eighty lines that everyone has already decided to skim.

The defence is procedural more than technical. When a diff is large and the deleted block resembles the inserted block, extract both into a separate comparison and diff them against each other directly. That second diff is usually empty, which lets you dismiss the move in one step, or it contains exactly the change that was hiding — which is the one you needed to see.

Some tools help. Git has move detection for review, and several review platforms grey out relocated blocks. Where the tooling does not help, the discipline that does is to separate moves from edits in the change itself: one commit that only moves code, a second that only changes it. Reviewers can then verify the first mechanically and read the second properly.

Reordering is not change

A large family of false differences comes from comparing ordered text that represents an unordered collection. Two JSON documents with the same fields in a different key order are semantically identical and textually very different. A list of dependencies, a set of environment variables, a database dump, a list of permissions — in all of these, order is an artefact of how the file was produced rather than a property of the data.

Normalise before comparing. Format both JSON documents with the same formatter and, where the consumer does not depend on key order, sort the keys. Sort a list before diffing it if the list is a set. If you are comparing two sets of values and you want to know what is present in one and missing in the other, a line diff is a poor instrument for the question — a set comparison tells you the three additions and two removals directly, without showing you the four hundred lines that merely moved.

The opposite mistake exists too. Sorting hides an ordering bug when order is significant: route tables matched in sequence, middleware chains, CSS rules where later declarations win, migration files. Before you normalise, be sure the order genuinely carries no meaning; if it does, the reordering is the change you are looking for.

Reviewing generated output

Machine-generated files break the assumptions that make review work. A lockfile changes thousands of lines when one dependency moves; a snapshot test rewrites an entire fixture; a code generator reformats everything it touches; output from a model arrives as a large block that looks plausible and has to be checked rather than read. In every case the diff is too large to read line by line, and reviewing it as if it were hand-written is how unreviewed changes get in.

Change the question you are asking. For a lockfile, do not read the diff: read the summary of which direct dependencies changed version, and check those. For a regenerated file, verify that the generator version and its input changed as expected, and that the output is reproducible — regenerate it yourself and confirm the diff against the committed version is empty. For a large refactor performed by a tool, diff the behaviour rather than the text: run the tests, compare the build output, compare the rendered result.

Generated output also has a specific review risk: it is uniformly plausible. Hand-written code has a rhythm, and a reviewer notices the line that does not fit. Generated text has no such rhythm, and a wrong constant sits perfectly comfortably among a hundred right ones. The countermeasures are to narrow the diff to the parts a human must actually check, to compare against the source of truth rather than against intuition, and to keep generated files out of the same commit as hand edits so the two can be read differently.

• Separate generated and hand-written changes into different commits so each can be reviewed in the right way.
• Regenerate the artefact locally and diff it against the committed version; an empty diff answers the reproducibility question in one step.
• For a lockfile, review the direct dependency changes and let the tooling account for the transitive ones.
• For large mechanical rewrites, verify behaviour with tests and build output rather than reading the text.

When a diff is the wrong tool

Diffing compares representations. When two representations differ but mean the same thing, a textual diff answers a question you did not ask. Two XML documents with different attribute order, two SQL queries formatted differently, two JSON payloads with different number formatting, two CSVs with different quoting conventions — all of these need canonicalisation, or a comparison that understands the format, before a diff means anything.

The general fix is to normalise both sides into a canonical form and then diff that. Format both with the same formatter, sort what can be sorted, round or stringify what has floating-point representation differences, and only then compare. This is also the most reliable way to compare API responses across environments: pretty-print both, sort object keys, and the remaining diff is the real one.

And when the comparison is about set membership rather than sequence — which identifiers exist in one export and not the other, which feature flags are enabled in staging but not production — reach for a list comparison instead. It answers in terms of added, removed and common entries, which is the shape of the answer you actually want, and it does not care what order either file was written in.

Continue with related checks and tools