Chocolatey Community Coffee Break

Join the Chocolatey Team on our regular monthly stream where we discuss all things Community, what we do, how you can get involved and answer your Chocolatey questions.

Learn More

Chocolatey Product Spotlight

Join the Chocolatey Team on our regular monthly stream where we put a spotlight on the most recent Chocolatey product releases. You'll have a chance to have your questions answered in a live Ask Me Anything format.

Learn More

Announcing Chocolatey Central Management 0.10.0

Livestream from
Thursday, 06 October 2022

We recently released our largest update to Chocolatey Central Management so far. Join Gary and Steph to find out more about Chocolatey Central Management and the new features and fixes we've added to this release.

Watch On-Demand
Chocolatey and Intune Overview

Webinar Replay from
Wednesday, 30 March 2022

At Chocolatey Software we strive for simple, and teaching others. Let us teach you just how simple it could be to keep your 3rd party applications updated across your devices, all with Intune!

Watch On-Demand
Chocolatey For Business. In Azure. In One Click.

Livestream from
Thursday, 9 June 2022

Join James and Josh to show you how you can get the Chocolatey For Business recommended infrastructure and workflow, created, in Azure, in around 20 minutes.

Watch On-Demand
The Future of Chocolatey CLI

Livestream from
Thursday, 04 August 2022

Join Paul and Gary to hear more about the plans for the Chocolatey CLI in the not so distant future. We'll talk about some cool new features, long term asks from Customers and Community and how you can get involved!

Watch On-Demand
Hacktoberfest Tuesdays 2022

Livestreams from
October 2022

For Hacktoberfest, Chocolatey ran a livestream every Tuesday! Re-watch Cory, James, Gary, and Rain as they share knowledge on how to contribute to open-source projects such as Chocolatey CLI.

Watch On-Demand
Chocolatey Product Spotlight: Chocolatey 1.2.0 and Chocolatey Licensed Extension 5.0.0

Livestream from
Thursday, 03 November 2022

Join Paul and Gary for this months Chocolatey product livestream where we look at the latest release of Chocolatey 1.2.0, Chocolatey Licensed Extension 5.0.0 and shine a spotlight on the new hook scripts functionality. This opens up so many possibilities for Chocolatey CLI users!

Watch On-Demand
Chocolatey Coding Livestream

Livestream from
Tuesday, 29 November 2022

Join Josh as he adds the ability to manage Chocolatey GUI config and features with the Chocolatey Ansible Collection.

Watch On-Demand
Introduction into Chocolatey with Veeam

Webinar from
Tuesday, 13 December 2022

Join Gary, Paul, and Maurice as they introduce and demonstrate how to use Chocolatey! Questions will be answered live in an Ask Me Anything format.

Watch On-Demand

Downloads:

11,598

Downloads of v 3.3.2:

290

Last Update:

07 Mar 2022

Package Maintainer(s):

Software Author(s):

  • Apache Software Foundation

Tags:

apache admin hadoop hdfs yarn cluster mapreduce namenode datanode

Hadoop

This is not the latest version of Hadoop available.

  • 1
  • 2
  • 3

3.3.2 | Updated: 07 Mar 2022

Downloads:

11,598

Downloads of v 3.3.2:

290

Maintainer(s):

Software Author(s):

  • Apache Software Foundation

Hadoop 3.3.2

This is not the latest version of Hadoop available.

  • 1
  • 2
  • 3

Some Checks Have Failed or Are Not Yet Complete

Not All Tests Have Passed


Validation Testing Passed


Verification Testing Passed

Details

Scan Testing Resulted in ScannerExempted

Details
Learn More

Deployment Method: Individual Install, Upgrade, & Uninstall

To install Hadoop, run the following command from the command line or from PowerShell:

>

To upgrade Hadoop, run the following command from the command line or from PowerShell:

>

To uninstall Hadoop, run the following command from the command line or from PowerShell:

>

Deployment Method:

NOTE

This applies to both open source and commercial editions of Chocolatey.

1. Enter Your Internal Repository Url

(this should look similar to https://community.chocolatey.org/api/v2/)


2. Setup Your Environment

1. Ensure you are set for organizational deployment

Please see the organizational deployment guide

2. Get the package into your environment

  • Open Source or Commercial:
    • Proxy Repository - Create a proxy nuget repository on Nexus, Artifactory Pro, or a proxy Chocolatey repository on ProGet. Point your upstream to https://community.chocolatey.org/api/v2/. Packages cache on first access automatically. Make sure your choco clients are using your proxy repository as a source and NOT the default community repository. See source command for more information.
    • You can also just download the package and push it to a repository Download

3. Copy Your Script

choco upgrade hadoop -y --source="'INTERNAL REPO URL'" --version="'3.3.2'" [other options]

See options you can pass to upgrade.

See best practices for scripting.

Add this to a PowerShell script or use a Batch script with tools and in places where you are calling directly to Chocolatey. If you are integrating, keep in mind enhanced exit codes.

If you do use a PowerShell script, use the following to ensure bad exit codes are shown as failures:


choco upgrade hadoop -y --source="'INTERNAL REPO URL'" --version="'3.3.2'" 
$exitCode = $LASTEXITCODE

Write-Verbose "Exit code was $exitCode"
$validExitCodes = @(0, 1605, 1614, 1641, 3010)
if ($validExitCodes -contains $exitCode) {
  Exit 0
}

Exit $exitCode

- name: Install hadoop
  win_chocolatey:
    name: hadoop
    version: '3.3.2'
    source: INTERNAL REPO URL
    state: present

See docs at https://docs.ansible.com/ansible/latest/modules/win_chocolatey_module.html.


chocolatey_package 'hadoop' do
  action    :install
  source   'INTERNAL REPO URL'
  version  '3.3.2'
end

See docs at https://docs.chef.io/resource_chocolatey_package.html.


cChocoPackageInstaller hadoop
{
    Name     = "hadoop"
    Version  = "3.3.2"
    Source   = "INTERNAL REPO URL"
}

Requires cChoco DSC Resource. See docs at https://github.com/chocolatey/cChoco.


package { 'hadoop':
  ensure   => '3.3.2',
  provider => 'chocolatey',
  source   => 'INTERNAL REPO URL',
}

Requires Puppet Chocolatey Provider module. See docs at https://forge.puppet.com/puppetlabs/chocolatey.


4. If applicable - Chocolatey configuration/installation

See infrastructure management matrix for Chocolatey configuration elements and examples.

WARNING

This package is exempt from moderation. While it is likely safe for you, there is more risk involved.

Description

The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures.

The project includes these modules:

  • Hadoop Common: The common utilities that support the other Hadoop modules.
  • Hadoop Distributed File System (HDFS™): A distributed file system that provides high-throughput access to application data.
  • Hadoop YARN: A framework for job scheduling and cluster resource management.
  • Hadoop MapReduce: A YARN-based system for parallel processing of large data sets.

Notes

The package sets the HADOOP_HOME environment variable and adds HADOOP_HOME\bin to the machine path.

The package by default will unzip to C:\Hadoop. This location can be changed, see below.

Package Parameters

The following package parameters can be set:

  • /unzipLocation - Unzip to a different path than default.

These parameters can be passed to the installer with the use of -params.
For example: choco install hadoop -params '"/unzipLocation:D:\Hadoop"'.


tools\chocolateyuninstall.ps1
$packageName = 'hadoop'
$packageVersion = $env:chocolateyPackageVersion
$hadoop_home = $env:HADOOP_HOME

# remove from Path
$path = [Environment]::GetEnvironmentVariable("Path", 'Machine')
$newPath = ($path.Split(';') | Where-Object { $_ -notmatch '.*hadoop.*' }) -join ';'
[Environment]::SetEnvironmentVariable("Path", $newPath, 'Machine')


Install-ChocolateyEnvironmentVariable `
    -VariableName "HADOOP_HOME" `
    -VariableValue $null `
    -VariableType 'Machine'

Install-ChocolateyEnvironmentVariable `
    -VariableName "JAVA_HOME" `
    -VariableValue $null `
    -VariableType 'User'


Remove-Item $hadoop_home -Recurse -Force
tools\chocolateyinstall.ps1
$ErrorActionPreference = 'Stop';

$packageName    = 'hadoop'
$packageVersion = $env:chocolateyPackageVersion
$toolsDir       = "$(Split-Path -parent $MyInvocation.MyCommand.Definition)"
$hadoop_home    = "C:\Hadoop"
$mirrors        = 'https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.3.2/hadoop-3.3.2.tar.gz'
$checksum32     = 'b341587495b12eec0b244b517f21df88eb46ef634dc7dc3e5969455b80ce2ce5'

# d/l from closest mirror
$get_mirror_page = Invoke-WebRequest -Uri $mirrors -UseBasicParsing
$url32 = $get_mirror_page.links | ? href -match '\.tar\.gz$' | select -First 1 -expand href


$pp = Get-PackageParameters
if ($pp.unzipLocation) { 
    $hadoop_home = $pp.unzipLocation
    Write-Host "Param: Unzipping (installing) to $hadoop_home"
}


$packageArgs = @{
  packageName   = $packageName
  unzipLocation = $toolsDir
  url           = $url32
  checksum      = $checksum32
  checksumType  = 'sha256'
  validExitCodes= @(0)
}

Install-ChocolateyZipPackage @packageArgs

# unzip tar
Get-ChocolateyUnzip -FileFullPath "$toolsDir\*.tar" -Destination $hadoop_home

Install-ChocolateyEnvironmentVariable `
    -VariableName "HADOOP_HOME" `
    -VariableValue $hadoop_home `
    -VariableType 'Machine'

Install-ChocolateyPath `
    -PathToInstall "%HADOOP_HOME%\$packageName-$packageVersion\bin" `
    -PathType 'Machine'

# Hadoop needs 8.3 path to find Java    
$sp = New-Object -ComObject Scripting.FileSystemObject
$f = $sp.GetFolder($env:JAVA_HOME)

Install-ChocolateyEnvironmentVariable `
    -VariableName "JAVA_HOME" `
    -VariableValue $f.ShortPath `
    -VariableType 'User'

# don't need tar anymore
Remove-Item $toolsDir\*.tar -ErrorAction SilentlyContinue -Force

Log in or click on link to see number of positives.

In cases where actual malware is found, the packages are subject to removal. Software sometimes has false positives. Moderators do not necessarily validate the safety of the underlying software, only that a package retrieves software from the official distribution point and/or validate embedded software against official distribution point (where distribution rights allow redistribution).

Chocolatey Pro provides runtime protection from possible malware.

Add to Builder Version Downloads Last Updated Status
Hadoop 3.3.5 595 Thursday, March 23, 2023 Approved
Hadoop 3.3.4 551 Monday, August 8, 2022 Exempted
Hadoop 3.3.3 221 Sunday, May 22, 2022 Exempted
Hadoop 3.3.2 290 Monday, March 7, 2022 Exempted
Hadoop 3.3.1 504 Wednesday, June 16, 2021 Exempted
Hadoop 3.3.0 935 Friday, July 17, 2020 Approved
Hadoop 3.2.1 970 Wednesday, September 25, 2019 Approved
Hadoop 3.2.0 601 Wednesday, January 23, 2019 Approved
Hadoop 3.1.1 941 Friday, August 10, 2018 Approved
Hadoop 3.1.0 664 Monday, April 9, 2018 Approved
Hadoop 3.0.1 318 Tuesday, March 27, 2018 Approved
Hadoop 3.0.0 515 Friday, December 15, 2017 Approved
Hadoop 2.9.0 1221 Tuesday, November 21, 2017 Approved
Hadoop 2.8.2 400 Monday, November 6, 2017 Approved
Hadoop 2.8.1 531 Monday, July 24, 2017 Approved
Hadoop 2.8.0 513 Saturday, April 1, 2017 Approved
Hadoop 2.7.3 770 Monday, February 27, 2017 Approved
Discussion for the Hadoop Package

Ground Rules:

  • This discussion is only about Hadoop and the Hadoop package. If you have feedback for Chocolatey, please contact the Google Group.
  • This discussion will carry over multiple versions. If you have a comment about a particular version, please note that in your comments.
  • The maintainers of this Chocolatey Package will be notified about new comments that are posted to this Disqus thread, however, it is NOT a guarantee that you will get a response. If you do not hear back from the maintainers after posting a message below, please follow up by using the link on the left side of this page or follow this link to contact maintainers. If you still hear nothing back, please follow the package triage process.
  • Tell us what you love about the package or Hadoop, or tell us what needs improvement.
  • Share your experiences with the package, or extra configuration or gotchas that you've found.
  • If you use a url, the comment will be flagged for moderation until you've been whitelisted. Disqus moderated comments are approved on a weekly schedule if not sooner. It could take between 1-5 days for your comment to show up.
comments powered by Disqus