I found out the hard way that SubscribeStar doesn't have an auto-draft writing feature and you can lose your entire post just by hitting [ESC] on the keyboard. This is the third time I've written this post, so it'll be much shorter than the previous two.

I've just released jdupes-1.17.0 and I think you're going to love it. This release fixes every long-standing problem with dedupe (used on Linux with BTRFS and more recently XFS) and adds an exciting new substring matching feature for finer-grained control over what gets scanned.

The dedupe code had several problems. It allocated a potentially huge memory structure that the kernel couldn't process due to internal size limits anyway, and it exhibited some strange misbehavior in several somewhat common cases. I decided to do a complete rewrite of the dedupe code from scratch. The new code seems to work in every test I've tried. I don't use dedupe, so if you notice a problem, please report it on the GitHub issue tracker for jdupes.

Remember how I added extension matching in jdupes-1.15.0? Now I've also added substring matching that works in a similar way. You can only specify one substring per -X nostr/onlystr to match (no comma-separated lists like the noext/onlyext tags) but the no/only meaning is the same. The substring match is done against the full relative file path, so tricks like directory matching are also possible. For detailed help, type jdupes -X help and learn about the new substring match extfilter tags. Some examples:

Exclude a directory: jdupes -X nostr:some_dir/

Only include paths containing the word "index": jdupes -X onlystr:index

Only scan JPEGs with IMG_ in the name: jdupes -X onlyext:jpg -X onlystr:IMG_

Only scan .tar.xz files: jdupes -X onlystr:".tar.xz"

Note that these substring matches happen across the whole file path, so if you use jdupes -X onlystr:IMG_ as in the above example (hoping to only scan files beginning with "IMG_")and there happens to be a directory called SOME_IMG_STUFF or even just IMG_STUFF in a file's path, all of the subdirectories and files will match, despite the files not necessarily starting with the desired substring. You can strengthen this check by requiring a leading slash: jdupes -X onlystr:"/IMG_" and/or you can exclude problematic directories using a nostr match that will catch them: jdupes -X onlystr:"/IMG_" -X nostr:"IMG_STUFF"

Head over to https://github.com/jbruchon/jdupes/releases and snag yourself a copy of the latest goodness! Let me know if you have any problems ([email protected]). I'm always happy to help

Also, if you're reading this and you don't already support my work, please consider tossing me a few bucks. The more support I receive here,. the more I can work on this stuff. I'd rather continue to work on fixes and features for my software than be beholden to a "real job," and this is how I'm trying to do it. Thank you to everyone who uses my software and helps me, be it through positive feedback, bug reports, code contributions, or good old-fashioned cold hard cash.