Construct Hub

• Implements: IInspectable

A CloudFormation AWS::Glue::Crawler.

The AWS::Glue::Crawler resource specifies an AWS Glue crawler. For more information, see Cataloging Tables with a Crawler and Crawler Structure in the AWS Glue Developer Guide .

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html

Example

// The code below shows an example of how to instantiate this type.
// The values are placeholders you should change.
import * as glue from '@aws-cdk/aws-glue';

declare const tags: any;
const cfnCrawler = new glue.CfnCrawler(this, 'MyCfnCrawler', {
  role: 'role',
  targets: {
    catalogTargets: [{
      databaseName: 'databaseName',
      tables: ['tables'],
    }],
    dynamoDbTargets: [{
      path: 'path',
    }],
    jdbcTargets: [{
      connectionName: 'connectionName',
      exclusions: ['exclusions'],
      path: 'path',
    }],
    mongoDbTargets: [{
      connectionName: 'connectionName',
      path: 'path',
    }],
    s3Targets: [{
      connectionName: 'connectionName',
      dlqEventQueueArn: 'dlqEventQueueArn',
      eventQueueArn: 'eventQueueArn',
      exclusions: ['exclusions'],
      path: 'path',
      sampleSize: 123,
    }],
  },

  // the properties below are optional
  classifiers: ['classifiers'],
  configuration: 'configuration',
  crawlerSecurityConfiguration: 'crawlerSecurityConfiguration',
  databaseName: 'databaseName',
  description: 'description',
  name: 'name',
  recrawlPolicy: {
    recrawlBehavior: 'recrawlBehavior',
  },
  schedule: {
    scheduleExpression: 'scheduleExpression',
  },
  schemaChangePolicy: {
    deleteBehavior: 'deleteBehavior',
    updateBehavior: 'updateBehavior',
  },
  tablePrefix: 'tablePrefix',
  tags: tags,
});

import { CfnCrawler } from '@aws-cdk/aws-glue'

new CfnCrawler(scope: Construct, id: string, props: CfnCrawlerProps)

• Type: Construct

scope in which this resource is defined.

• Type: string

scoped id of the resource.

• Type: CfnCrawlerProps

resource properties.

public toString(): string

Returns a string representation of this construct.

public overrideLogicalId(newLogicalId: string): void

Overrides the auto-generated logical ID with a specific ID.

• Type: string

The new logical ID to use for this stack element.

public addDeletionOverride(path: string): void

Syntactic sugar for addOverride(path, undefined).

• Type: string

The path of the value to delete.

public addDependsOn(target: CfnResource): void

Indicates that this resource depends on another resource and cannot be provisioned unless the other resource has been successfully provisioned.

This can be used for resources across stacks (or nested stack) boundaries and the dependency will automatically be transferred to the relevant scope.

• Type: CfnResource

public addMetadata(key: string, value: any): void

Add a value to the CloudFormation Resource Metadata.

[https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/metadata-section-structure.html

Note that this is a different set of metadata from CDK node metadata; this metadata ends up in the stack template under the resource, whereas CDK node metadata ends up in the Cloud Assembly.](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/metadata-section-structure.html

Note that this is a different set of metadata from CDK node metadata; this metadata ends up in the stack template under the resource, whereas CDK node metadata ends up in the Cloud Assembly.)

• Type: string

• Type: any

public addOverride(path: string, value: any): void

Adds an override to the synthesized CloudFormation resource.

To add a property override, either use addPropertyOverride or prefix path with "Properties." (i.e. Properties.TopicName).

If the override is nested, separate each nested level using a dot (.) in the path parameter. If there is an array as part of the nesting, specify the index in the path.

To include a literal . in the property name, prefix with a \. In most programming languages you will need to write this as "\\." because the \ itself will need to be escaped.

For example,

cfnResource.addOverride('Properties.GlobalSecondaryIndexes.0.Projection.NonKeyAttributes', ['myattribute']);
cfnResource.addOverride('Properties.GlobalSecondaryIndexes.1.ProjectionType', 'INCLUDE');

would add the overrides

"Properties": {
   "GlobalSecondaryIndexes": [
     {
       "Projection": {
         "NonKeyAttributes": [ "myattribute" ]
         ...
       }
       ...
     },
     {
       "ProjectionType": "INCLUDE"
       ...
     },
   ]
   ...
}

The value argument to addOverride will not be processed or translated in any way. Pass raw JSON values in here with the correct capitalization for CloudFormation. If you pass CDK classes or structs, they will be rendered with lowercased key names, and CloudFormation will reject the template.

• Type: string

The path of the property, you can use dot notation to override values in complex types.

Any intermdediate keys will be created as needed.

• Type: any

The value.

Could be primitive or complex.

public addPropertyDeletionOverride(propertyPath: string): void

Adds an override that deletes the value of a property from the resource definition.

• Type: string

The path to the property.

public addPropertyOverride(propertyPath: string, value: any): void

Adds an override to a resource property.

Syntactic sugar for addOverride("Properties.<...>", value).

• Type: string

The path of the property.

• Type: any

The value.

public applyRemovalPolicy(policy?: RemovalPolicy, options?: RemovalPolicyOptions): void

Sets the deletion policy of the resource based on the removal policy specified.

The Removal Policy controls what happens to this resource when it stops being managed by CloudFormation, either because you've removed it from the CDK application or because you've made a change that requires the resource to be replaced.

The resource can be deleted (RemovalPolicy.DESTROY), or left in your AWS account for data recovery and cleanup later (RemovalPolicy.RETAIN).

• Type: RemovalPolicy

• Type: RemovalPolicyOptions

public getAtt(attributeName: string): Reference

Returns a token for an runtime attribute of this resource.

Ideally, use generated attribute accessors (e.g. resource.arn), but this can be used for future compatibility in case there is no generated attribute.

• Type: string

The name of the attribute.

public getMetadata(key: string): any

Retrieve a value value from the CloudFormation Resource Metadata.

[https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/metadata-section-structure.html

Note that this is a different set of metadata from CDK node metadata; this metadata ends up in the stack template under the resource, whereas CDK node metadata ends up in the Cloud Assembly.](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/metadata-section-structure.html

Note that this is a different set of metadata from CDK node metadata; this metadata ends up in the stack template under the resource, whereas CDK node metadata ends up in the Cloud Assembly.)

• Type: string

public inspect(inspector: TreeInspector): void

Examines the CloudFormation resource and discloses attributes.

• Type: TreeInspector

tree inspector to collect and process attributes.

import { CfnCrawler } from '@aws-cdk/aws-glue'

CfnCrawler.isConstruct(x: any)

Return whether the given object is a Construct.

• Type: any

import { CfnCrawler } from '@aws-cdk/aws-glue'

CfnCrawler.isCfnElement(x: any)

Returns true if a construct is a stack element (i.e. part of the synthesized cloudformation template).

Uses duck-typing instead of instanceof to allow stack elements from different versions of this library to be included in the same stack.

• Type: any

import { CfnCrawler } from '@aws-cdk/aws-glue'

CfnCrawler.isCfnResource(construct: IConstruct)

Check whether the given construct is a CfnResource.

• Type: IConstruct

public readonly node: ConstructNode;

• Type: ConstructNode

The construct tree node associated with this construct.

public readonly creationStack: string[];

• Type: string[]

public readonly logicalId: string;

• Type: string

The logical ID for this CloudFormation stack element.

The logical ID of the element is calculated from the path of the resource node in the construct tree.

To override this value, use overrideLogicalId(newLogicalId).

public readonly stack: Stack;

• Type: Stack

The stack in which this element is defined.

CfnElements must be defined within a stack scope (directly or indirectly).

public readonly ref: string;

• Type: string

Return a string that will be resolved to a CloudFormation { Ref } for this element.

If, by any chance, the intrinsic reference of a resource is not a string, you could coerce it to an IResolvable through Lazy.any({ produce: resource.ref }).

public readonly cfnOptions: ICfnResourceOptions;

• Type: ICfnResourceOptions

Options for this resource, such as condition, update policy etc.

public readonly cfnResourceType: string;

• Type: string

AWS resource type.

public readonly tags: TagManager;

• Type: TagManager

The tags to use with this crawler.

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-tags

public readonly role: string;

• Type: string

The Amazon Resource Name (ARN) of an IAM role that's used to access customer resources, such as Amazon Simple Storage Service (Amazon S3) data.

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-role

public readonly targets: IResolvable | TargetsProperty;

• Type: IResolvable | TargetsProperty

A collection of targets to crawl.

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-targets

public readonly classifiers: string[];

• Type: string[]

A list of UTF-8 strings that specify the names of custom classifiers that are associated with the crawler.

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-classifiers

public readonly configuration: string;

• Type: string

Crawler configuration information.

This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Configuring a Crawler .

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-configuration

public readonly crawlerSecurityConfiguration: string;

• Type: string

The name of the SecurityConfiguration structure to be used by this crawler.

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-crawlersecurityconfiguration

public readonly databaseName: string;

• Type: string

The name of the database in which the crawler's output is stored.

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-databasename

public readonly description: string;

• Type: string

A description of the crawler.

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-description

public readonly name: string;

• Type: string

The name of the crawler.

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-name

public readonly recrawlPolicy: IResolvable | RecrawlPolicyProperty;

• Type: IResolvable | RecrawlPolicyProperty

A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-recrawlpolicy

public readonly schedule: IResolvable | ScheduleProperty;

• Type: IResolvable | ScheduleProperty

For scheduled crawlers, the schedule when the crawler runs.

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-schedule

public readonly schemaChangePolicy: IResolvable | SchemaChangePolicyProperty;

• Type: IResolvable | SchemaChangePolicyProperty

The policy that specifies update and delete behaviors for the crawler.

The policy tells the crawler what to do in the event that it detects a change in a table that already exists in the customer's database at the time of the crawl. The SchemaChangePolicy does not affect whether or how new tables and partitions are added. New tables and partitions are always created regardless of the SchemaChangePolicy on a crawler.

The SchemaChangePolicy consists of two components, UpdateBehavior and DeleteBehavior .

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-schemachangepolicy

public readonly tablePrefix: string;

• Type: string

The prefix added to the names of tables that are created.

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-tableprefix

public readonly CFN_RESOURCE_TYPE_NAME: string;

• Type: string

The CloudFormation resource type name for this resource class.