Edit

Share via

New-AzHDInsightClusterConfig

Module:
Az.HDInsight Module

Creates a non-persisted cluster configuration object that describes an Azure HDInsight cluster configuration.

Syntax

Default (Default) 

New-AzHDInsightClusterConfig
    [-StorageAccountResourceId <String>]
    [-StorageAccountKey <String>]
    [-StorageAccountType <StorageType>]
    [-OozieMetastore <AzureHDInsightMetastore>]
    [-HiveMetastore <AzureHDInsightMetastore>]
    [-HeadNodeSize <String>]
    [-WorkerNodeSize <String>]
    [-EdgeNodeSize <String>]
    [-ZookeeperNodeSize <String>]
    [-ClusterType <String>]
    [-ClusterTier <String>]
    [-ObjectId <Guid>]
    [-ApplicationId <Guid>]
    [-CertificateFileContents <Byte[]>]
    [-CertificateFilePath <String>]
    [-CertificatePassword <String>]
    [-AadTenantId <Guid>]
    [-MinSupportedTlsVersion <String>]
    [-AssignedIdentity <String>]
    [-EncryptionAlgorithm <String>]
    [-EncryptionKeyName <String>]
    [-EncryptionKeyVersion <String>]
    [-EncryptionVaultUri <String>]
    [-EncryptionInTransit <Boolean>]
    [-EncryptionAtHost <Boolean>]
    [-DefaultProfile <IAzureContextContainer>]
    [<CommonParameters>]

Description

The New-AzHDInsightClusterConfig cmdlet creates a non-persisted object that describes an Azure HDInsight cluster configuration.

Examples

Example 1: Create a cluster configuration object

# Primary storage account info
$storageAccountResourceGroupName = "Group"
$storageAccountResourceId = "yourstorageaccountresourceid"
$storageAccountName = "yourstorageaccountname"
$storageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $storageAccountResourceGroupName -Name $storageAccountName)[0].value


$storageContainer = "container002"

# Cluster configuration info
$location = "East US 2"
$clusterResourceGroupName = "Group"
$clusterName = "your-hadoop-002"
$clusterCreds = Get-Credential

# If the cluster's resource group doesn't exist yet, run:
#   New-AzResourceGroup -Name $clusterResourceGroupName -Location $location

# Create the cluster
New-AzHDInsightClusterConfig `
            | Add-AzHDInsightStorage `
                -StorageAccountName "$secondStorageAccountName.blob.core.contoso.net" `
                -StorageAccountKey $key2 `
            | New-AzHDInsightCluster `
                -ClusterType Hadoop `
                -OSType Windows `
                -ClusterSizeInNodes 4 `
                -ResourceGroupName $clusterResourceGroupName `
                -ClusterName $clusterName `
                -HttpCredential $clusterCreds `
                -Location $location `
                -StorageAccountResourceId $storageAccountResourceId `
                -StorageAccountKey $storageAccountKey `
                -StorageContainer $storageContainer

This command creates a cluster configuration object.

Parameters

-AadTenantId

Specifies the Microsoft Entra tenant ID that will be used when accessing Azure Data Lake Store.

Parameter properties

Type:Guid
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-ApplicationId

Gets or sets the Service Principal Application Id for accessing Azure Data Lake.

Parameter properties

Type:Guid
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-AssignedIdentity

Gets or sets the assigned identity.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-CertificateFileContents

Specifies file contents of the certificate that will be used when accessing Azure Data Lake Store.

Parameter properties

Type:

Byte[]

Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-CertificateFilePath

Specifies the file path to the certificate that will be used to authenticate as the Service Principal. The cluster will use this when accessing Azure Data Lake Store.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-CertificatePassword

Specifies the password for the certificate that will be used to authenticate as the Service Principal. The cluster will use this when accessing Azure Data Lake Store.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-ClusterTier

Specifies the HDInsight cluster tier. The acceptable values for this parameter are:

  • Standard
  • Premium The default value is Standard. The Premium tier can only be used with Linux clusters, and it enables the use of some new features.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-ClusterType

Specifies the type of cluster to create. The acceptable values for this parameter are:

  • Hadoop
  • HBase
  • Storm
  • Spark
  • INTERACTIVEHIVE
  • Kafka
  • RServer

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-DefaultProfile

The credentials, account, tenant, and subscription used for communication with azure

Parameter properties

Type:IAzureContextContainer
Default value:None
Supports wildcards:False
DontShow:False
Aliases:AzContext, AzureRmContext, AzureCredential

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-EdgeNodeSize

Specifies the size of the virtual machine for the edge node. Use Get-AzVMSize for acceptable VM sizes, and see HDInsight's pricing page. This parameter is valid only for RServer clusters.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-EncryptionAlgorithm

Gets or sets the encryption algorithm.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-EncryptionAtHost

Gets or sets the flag which indicates whether enable encryption at host or not.

Parameter properties

Type:

Nullable<T>[Boolean]

Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-EncryptionInTransit

Gets or sets the flag which indicates whether enable encryption in transit or not.

Parameter properties

Type:

Nullable<T>[Boolean]

Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-EncryptionKeyName

Gets or sets the encryption key name.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-EncryptionKeyVersion

Gets or sets the encryption key version.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-EncryptionVaultUri

Gets or sets the encryption vault uri.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-HeadNodeSize

Specifies the size of the virtual machine for the Head node. Use Get-AzVMSize for acceptable VM sizes, and see HDInsight's pricing page.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-HiveMetastore

Specifies the metastore to store Hive metadata. You can alternatively use the Add-AzHDInsightMetastore cmdlet.

Parameter properties

Type:AzureHDInsightMetastore
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-MinSupportedTlsVersion

Gets or sets the minimal supported TLS version.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-ObjectId

Specifies the Microsoft Entra object ID (a GUID) of the Microsoft Entra service principal that represents the cluster. The cluster will use this when accessing Azure Data Lake Store.

Parameter properties

Type:Guid
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-OozieMetastore

Specifies the metastore to store Oozie metadata. You can alternatively use the Add-AzHDInsightMetastore cmdlet.

Parameter properties

Type:AzureHDInsightMetastore
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-StorageAccountKey

Gets or sets the storage account access key.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-StorageAccountResourceId

Gets or sets the storage account resource id.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-StorageAccountType

Gets or sets the type of the default storage account.

Parameter properties

Type:StorageType
Default value:None
Accepted values:AzureStorage, AzureDataLakeStore, AzureDataLakeStorageGen2
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-WorkerNodeSize

Specifies the size of the virtual machine for the Worker node. Use Get-AzVMSize for acceptable VM sizes, and see HDInsight's pricing page.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

-ZookeeperNodeSize

Specifies the size of the virtual machine for the Zookeeper node. Use Get-AzVMSize for acceptable VM sizes, and see HDInsight's pricing page. This parameter is valid only for HBase or Storm clusters.

Parameter properties

Type:String
Default value:None
Supports wildcards:False
DontShow:False

Parameter sets

(All)
Position:Named
Mandatory:False
Value from pipeline:False
Value from pipeline by property name:False
Value from remaining arguments:False

CommonParameters

This cmdlet supports the common parameters: -Debug, -ErrorAction, -ErrorVariable, -InformationAction, -InformationVariable, -OutBuffer, -OutVariable, -PipelineVariable, -ProgressAction, -Verbose, -WarningAction, and -WarningVariable. For more information, see about_CommonParameters.

Inputs

None

Outputs

AzureHDInsightConfig