The earlier check for FEFF was not sensible: A BOM in the middle of an UTF-16 file could easily arise when two UTF-16LE or twe UTF-16BE files are concatenated. Nothing goes wrong in that case, the BOM should just be displayed a zero-width-space. Therefore, diff should not bail out.

let looks_like_text() give different values for UTF-16 BE/LE. Not used yet.

For UTF-16, the line-length limit is set to 2/3th of the line length limit for other text, because UTF-16 -> UTF-8 conversion can increase the line length (in bytes) by max 50%. This guarantees that a UTF-16 diff can be made by converting the two UTF-16 files to UTF-8 and then do a normal diff.

re-use looks_like_blob

Trying to commit a file with an UTF-16 BOM will now give a warning, just as a file containing crlf

The --args command line option now accept UTF-16 files as well

src/stash.c: In function ‘stash_cmd’:
src/stash.c:377:16: warning: ‘stashid’ may be used uninitialized in this function
src/stash.c:472:7: note: ‘stashid’ was declared here

Implement looks_like_text(), similar to looks_like_binary() but it can distinguish more types of text/binary files.

This has the effect that on Windows wiki pages can now be in unicode as well.

This has the effect that on Windows the --args file accepts a unicode file, starting with a UTF-16 BOM as well

checkin.c: Make limits the same as in looks_like_binary(), preparation to move the function to diff.c

In the case of only appending, the user was not notified not properly redirected, and the ticket chagnes were not committed.

Still, there should be some kind of warning to the user, if it ends up in such situation, I think. ... (check-in: f02946aa user: viriketo tags: trunk)

struct utimbuf -> struct _utimbuf (compiler warning with mingw-w64)

a few "const" additions (lower memory footprint, allows C-compiler to optimize better)

struct utimbuf -> struct _utimbuf (compiler warning with mingw-w64)

a few "const" additions

This is not a complete backout of [0c37874941c89] as the underlying infrastructure used to compute the age of files is retained. ... (check-in: 5c0843a8 user: drh tags: trunk)