Downloads:
3,032
Downloads of v 1.104:
264
Last Update:
13 Feb 2023
Package Maintainer(s):
Software Author(s):
- Hermann Schinagl
Tags:
cli ntfs data-integrity checksum dedupe freeware- Software Specific:
- Software Site
- Software License
- Package Specific:
- Package Source
- Package outdated?
- Package broken?
- Contact Maintainers
- Contact Site Admins
- Software Vendor?
- Report Abuse
- Download
Dupemerge
- 1
- 2
- 3
1.104 | Updated: 13 Feb 2023
- Software Specific:
- Software Site
- Software License
- Package Specific:
- Package Source
- Package outdated?
- Package broken?
- Contact Maintainers
- Contact Site Admins
- Software Vendor?
- Report Abuse
- Download
Downloads:
3,032
Downloads of v 1.104:
264
Software Author(s):
- Hermann Schinagl
Dupemerge 1.104
Legal Disclaimer: Neither this package nor Chocolatey Software, Inc. are affiliated with or endorsed by Hermann Schinagl. The inclusion of Hermann Schinagl trademark(s), if any, upon this webpage is solely to identify Hermann Schinagl goods or services and not for commercial purposes.
- 1
- 2
- 3
All Checks are Passing
3 Passing Tests
Deployment Method: Individual Install, Upgrade, & Uninstall
To install Dupemerge, run the following command from the command line or from PowerShell:
To upgrade Dupemerge, run the following command from the command line or from PowerShell:
To uninstall Dupemerge, run the following command from the command line or from PowerShell:
Deployment Method:
This applies to both open source and commercial editions of Chocolatey.
1. Enter Your Internal Repository Url
(this should look similar to https://community.chocolatey.org/api/v2/)
2. Setup Your Environment
1. Ensure you are set for organizational deployment
Please see the organizational deployment guide
2. Get the package into your environment
Option 1: Cached Package (Unreliable, Requires Internet - Same As Community)-
Open Source or Commercial:
- Proxy Repository - Create a proxy nuget repository on Nexus, Artifactory Pro, or a proxy Chocolatey repository on ProGet. Point your upstream to https://community.chocolatey.org/api/v2/. Packages cache on first access automatically. Make sure your choco clients are using your proxy repository as a source and NOT the default community repository. See source command for more information.
- You can also just download the package and push it to a repository Download
-
Open Source
-
Download the package:
Download - Follow manual internalization instructions
-
-
Package Internalizer (C4B)
-
Run: (additional options)
choco download dupemerge --internalize --source=https://community.chocolatey.org/api/v2/
-
For package and dependencies run:
choco push --source="'INTERNAL REPO URL'"
- Automate package internalization
-
Run: (additional options)
3. Copy Your Script
choco upgrade dupemerge -y --source="'INTERNAL REPO URL'" [other options]
See options you can pass to upgrade.
See best practices for scripting.
Add this to a PowerShell script or use a Batch script with tools and in places where you are calling directly to Chocolatey. If you are integrating, keep in mind enhanced exit codes.
If you do use a PowerShell script, use the following to ensure bad exit codes are shown as failures:
choco upgrade dupemerge -y --source="'INTERNAL REPO URL'"
$exitCode = $LASTEXITCODE
Write-Verbose "Exit code was $exitCode"
$validExitCodes = @(0, 1605, 1614, 1641, 3010)
if ($validExitCodes -contains $exitCode) {
Exit 0
}
Exit $exitCode
- name: Install dupemerge
win_chocolatey:
name: dupemerge
version: '1.104'
source: INTERNAL REPO URL
state: present
See docs at https://docs.ansible.com/ansible/latest/modules/win_chocolatey_module.html.
chocolatey_package 'dupemerge' do
action :install
source 'INTERNAL REPO URL'
version '1.104'
end
See docs at https://docs.chef.io/resource_chocolatey_package.html.
cChocoPackageInstaller dupemerge
{
Name = "dupemerge"
Version = "1.104"
Source = "INTERNAL REPO URL"
}
Requires cChoco DSC Resource. See docs at https://github.com/chocolatey/cChoco.
package { 'dupemerge':
ensure => '1.104',
provider => 'chocolatey',
source => 'INTERNAL REPO URL',
}
Requires Puppet Chocolatey Provider module. See docs at https://forge.puppet.com/puppetlabs/chocolatey.
4. If applicable - Chocolatey configuration/installation
See infrastructure management matrix for Chocolatey configuration elements and examples.
Private CDN cached downloads available for licensed customers. Never experience 404 breakages again! Learn more...
This package was approved by moderator Windos on 19 Feb 2023.
Overview
Dupemerge searches for identical files on a logical drive and creates hardlinks among those files, thus saving lots of hard disk space. Most hard disks contain quite a lot of completely identical files, which consume a lot of disk space. This waste of space can be drastically reduced by using the NTFS file system hardlink functionality to link the identical files ("dupes") together.
### Technical
Dupemerge creates a cryptological hashsum for each file found below the given paths and compares those hashes to each other to find the dupes. There is no file date comparison involved in detecting dupes, only the size and content of the files.
To speed up comparison, only files with the same size get compared to each other. Furthermore the hashsums for equal sized files get calculated incrementally, which means that during the first pass only the first 4 kilobyte are hashed and compared, and during the next rounds more and more data are hashed and compared.
Due to long run time on large disks, a file which has already been hashsummed might change before all dupes to that file are found. To prevent false hardlink creation due to intermediate changes, dupemerge saves the file write time of a file when it hashsums the file and checks back if this time changed when it tries to hardlink dupes.
#### Multiple Runs
If dupemerge is run once, hardlinks among identical files are created. To save time during a second run on the same locations, dupemerge checks if a file is already a hardlink, and tries to find the other hardlinks by comparing the unique NTFS file-id. This saves a lot of time, because checksums for large files need not be created twice.
#### Transaction based Hardlinking
Before DupeMerge hardlinks file together it renames the file to a temporary name, then creates the hardlink, and afterwards deletes the temporary file. All that is done to be able to roll-back the operation if e.g the hardlinking failes.
#### TimeStamp Handling
A tupel of hardlinks for one file has always the one timestamp. This is by design of NTFS. But things are a bit confusing sometimes, because after hardlinking the same timestamp is only shown, after the hardlink was once e.g. opened/accessed. So it may happen, that immediatley after dupemerge one observes different timestamps within a tupel of hardlinks, but after such a hardlink has been opened for e.g. read, the timestamp changes to the timestamp of the whole tupel. That's also by design of NTFS.
Dupemerge has a dupe-find algorithm which is tuned to perform especially well on large server disks, where it has been tested in depth to guarantee data integrity.
Log in or click on link to see number of positives.
- dupemerge.1.104.nupkg (dd6255172263) - ## / 54
- dupemerge64.zip (363d756ec91d) - ## / 65
- dupemerge.zip (a5af1a0cf95f) - ## / 65
In cases where actual malware is found, the packages are subject to removal. Software sometimes has false positives. Moderators do not necessarily validate the safety of the underlying software, only that a package retrieves software from the official distribution point and/or validate embedded software against official distribution point (where distribution rights allow redistribution).
Chocolatey Pro provides runtime protection from possible malware.
Add to Builder | Version | Downloads | Last Updated | Status |
---|---|---|---|---|
Dupemerge 1.104 | 264 | Monday, February 13, 2023 | Approved | |
Dupemerge 1.103 | 694 | Saturday, November 30, 2019 | Approved | |
Dupemerge 1.102 | 365 | Monday, January 28, 2019 | Approved | |
Dupemerge 1.080.0.20170625 | 505 | Thursday, September 21, 2017 | Approved | |
Dupemerge 1.080 | 572 | Sunday, June 18, 2017 | Approved | |
Dupemerge 1.07 | 554 | Sunday, April 19, 2015 | Approved |
© 1999 - 2020 Hermann Schinagl
-
- vcredist2017 (≥ 14.16.27033)
Ground Rules:
- This discussion is only about Dupemerge and the Dupemerge package. If you have feedback for Chocolatey, please contact the Google Group.
- This discussion will carry over multiple versions. If you have a comment about a particular version, please note that in your comments.
- The maintainers of this Chocolatey Package will be notified about new comments that are posted to this Disqus thread, however, it is NOT a guarantee that you will get a response. If you do not hear back from the maintainers after posting a message below, please follow up by using the link on the left side of this page or follow this link to contact maintainers. If you still hear nothing back, please follow the package triage process.
- Tell us what you love about the package or Dupemerge, or tell us what needs improvement.
- Share your experiences with the package, or extra configuration or gotchas that you've found.
- If you use a url, the comment will be flagged for moderation until you've been whitelisted. Disqus moderated comments are approved on a weekly schedule if not sooner. It could take between 1-5 days for your comment to show up.