AWS solutions documentation change
Summary
Complete restructuring of connectors documentation: Removed detailed technical sections about triggers, field mappings, and security configuration. Added conceptual framework focusing on connector purposes (transformation, enrichment, publishing), governance model, execution flow, and composable step types. Introduced AWS Deadline Cloud connector and reorganized supported connector types.
Security assessment
The changes are a conceptual rewrite focusing on connector functionality and governance without addressing specific vulnerabilities. The retained security note about testing configurations is general best practice. The governance model (multi-level approvals) is an existing feature, not a new security capability. No CVE, vulnerability fix, or incident response is referenced.
Diff
diff --git a/solutions/latest/spatial-data-management-on-aws/connectors-overview.md b/solutions/latest/spatial-data-management-on-aws/connectors-overview.md index 72416e511..7628cd9bc 100644 --- a//solutions/latest/spatial-data-management-on-aws/connectors-overview.md +++ b//solutions/latest/spatial-data-management-on-aws/connectors-overview.md @@ -5 +5 @@ -How connectors workConnector directionsSupported connector typesTriggersField mappingsPayload fieldsVariable substitutionSecurity configurationConnected resourcesSetting up a connectorConnector configuration referenceFailure policyError handlingSecurity modelConnectors, templates, and projects +How connectors workSupported connector typesConnectors, templates, and projects @@ -9 +9 @@ How connectors workConnector directionsSupported connector typesTriggersField ma -Connectors are the extensibility layer of Spatial Data Management on AWS (SDMA). They let you define event-driven workflows that integrate SDMA with external systems and AWS services — without writing custom code. A connector is a JSON configuration that describes what happens when a resource event occurs: which steps to run, how to transform the data, and where to send it. +Spatial assets become more valuable when they work with the applications your teams already use — geospatial platforms, rendering services, quality analysis tools, and publishing catalogs. Connectors are how SDMA leverages those external applications and services to transform, enrich, and interoperate with your content — and how those systems reach back in, governed by SDMA. @@ -11 +11 @@ Connectors are the extensibility layer of Spatial Data Management on AWS (SDMA). -Connectors support a range of integration patterns, including publishing metadata to external destinations, invoking custom processing logic, and orchestrating multi-step workflows across different services. As SDMA evolves, new step types and capabilities can be added to connectors without changes to the core platform. +A connector can submit a compute job that converts a CAD file into a reviewable 3D format and writes the result back as a derived file. It can call an inference service that classifies imagery and writes quality scores into the asset record. It can look up metadata from a database or CSV and apply it across hundreds of files in one operation. It can publish structured metadata to a geospatial catalog, a data lake, or a project management system every time an asset changes. It can do several of these things in one governed contract. @@ -13,26 +13 @@ Connectors support a range of integration patterns, including publishing metadat -###### Important - -Connectors execute the configuration you provide. SDMA validates the structure and security credentials of a connector, but does not evaluate the business logic of the configuration. For example, if a connector is configured to overwrite data or send metadata to an unintended destination, SDMA executes those actions as configured. Test connector configurations thoroughly in a non-production environment before enabling them. - -## How connectors work - -Connectors are event-driven. When a resource event occurs (for example, an asset is created, updated, or deleted), SDMA evaluates the connector’s triggers and executes the matching steps. Each connector defines: - - * **Direction** – Whether the connector publishes data to an external system or synchronizes bidirectionally. - - * **Connector type** – The destination type (for example, REST API, Amazon S3, AWS Lambda, Amazon EventBridge). - - * **Field mappings** – How resource metadata fields map to the external system’s expected format. - - * **Triggers** – Which resource events activate the connector and what steps to execute. - - * **Security configuration** – How the connector authenticates with the external system. - - - - -The following diagram shows the connector execution flow: - - 1. A resource event occurs (for example, an asset is created). - - 2. SDMA evaluates all connectors associated with the asset’s template. +A domain expert or administrator defines this once — what transforms to run, how to assess file quality, what metadata to enrich from which sources, where to publish — and attaches those connectors to a template. From that point on, every asset created under that template gets the benefit automatically. Files are converted, metadata is enriched, quality is assessed, and external systems are updated as content arrives. The people uploading and working with assets do not need to know how any of this works. They see governed, enriched, publication-ready assets from the moment of ingestion. @@ -40 +15 @@ The following diagram shows the connector execution flow: - 3. For each connector, SDMA matches the event against the connector’s triggers. +Connectors work through configuration, not code. You declare what should happen, when, with what data, under what credentials, and SDMA handles execution, security, failure policy, and audit. Every invocation is tracked. Every result flows back through the asset record with provenance. Connectors are governed through an explicit authorization chain: a connector is created and approved at the library level, a template author requests and approves which connectors are permitted for that class of assets, and a project uses that template. A connector does nothing until it is explicitly approved at each level. This means the people who define integrations, the people who define asset standards, and the people who create projects each control their own part of the chain. @@ -42 +17 @@ The following diagram shows the connector execution flow: - 4. If a trigger matches, SDMA executes the trigger’s steps in order. +With connectors, you can: @@ -44 +19 @@ The following diagram shows the connector execution flow: - 5. Each step uses field mappings and variable substitution to build the output payload. + * **Transform and derive content.** Submit compute jobs to AWS Deadline Cloud that convert file formats, generate thumbnails, extract spatial properties, or run custom processing pipelines that leverage your existing licensed software. Results come back as derived files and metadata on the asset — governed, versioned, and traceable to the job that produced them. @@ -46 +21 @@ The following diagram shows the connector execution flow: - 6. The connector sends the payload to the external system using the configured security credentials. + * **Enrich metadata from external sources.** Look up values from S3 files, DynamoDB tables, or REST APIs and write them as metadata attributes on assets or files. Match external records to SDMA resources by filename, ID, or any field. Enrich in bulk on upload or on demand. @@ -47,0 +23 @@ The following diagram shows the connector execution flow: + * **Publish to external systems.** Push metadata to REST APIs, write to S3, invoke Lambda functions, or send events to EventBridge when assets are created, updated, or deleted. Keep external systems in sync with your asset record. @@ -48,0 +25 @@ The following diagram shows the connector execution flow: + * **Integrate with external applications.** Maintain a governed relationship with an external application that spans both directions — create external records on project creation, enrich metadata from the external catalog, and publish derived content back. One connector, one set of credentials, one failure policy. @@ -49,0 +27 @@ The following diagram shows the connector execution flow: + * **Expose external resources for guided workflows.** List available resources from a connected application — projects, collections, models — so users can select and link them during template authoring without leaving the Spatial Data Portal. @@ -51 +29 @@ The following diagram shows the connector execution flow: -## Connector directions + * **Chain connectors through the asset record.** When one connector produces a derived file or updates metadata, that change is a lifecycle event that can trigger other connectors. A format conversion connector can feed a quality analysis connector, which can feed a publishing connector — each governed independently, composing through the asset record without referencing each other. @@ -53 +30,0 @@ The following diagram shows the connector execution flow: -SDMA supports the following connector directions. @@ -55,13 +31,0 @@ SDMA supports the following connector directions. -Direction | Description ----|--- -`publish` | Sends resource metadata to an external system when events occur. Supports REST API, Amazon S3, AWS Lambda, Amazon EventBridge, and STAC API destinations. -`derive` | Enriches resource metadata by looking up data from an external source (Amazon S3, REST API, or Amazon DynamoDB) and applying it to assets or files. See [Metadata derivation connector](./connector-derivation.html). -`synchronization` | Bidirectional integration that can combine multiple step types (for example, REST API calls followed by Lambda invocations) in a single trigger. - -## Supported connector types - -SDMA provides several connector types, each designed for a specific integration pattern. Each type is also a reusable step type — see Composable step types. - -### REST API - -Step type: `rest` @@ -69 +32,0 @@ Step type: `rest` -Sends HTTP requests to external REST endpoints. Supports GET, POST, PUT, PATCH, and DELETE methods. Use this connector to integrate with any system that exposes a REST API. @@ -71,37 +34 @@ Sends HTTP requests to external REST endpoints. Supports GET, POST, PUT, PATCH, -For configuration details, see [REST API connector](./connector-rest-api.html). - -### Amazon S3 - -Step types: `s3PutObject`, `s3DeleteObject` - -Writes JSON metadata to an S3 bucket or deletes S3 objects based on resource events. Use this connector to publish structured metadata to a data lake or to feed downstream processing pipelines. - -For configuration details, see [Amazon S3 connector](./connector-s3.html). - -### AWS Lambda - -Step type: `lambdaInvoke` - -Invokes a Lambda function with resource metadata as the payload. Use this connector for custom processing logic, transformations, or integrations that require code. - -For configuration details, see [AWS Lambda connector](./connector-lambda.html). - -### Amazon EventBridge - -Step type: `eventBridgePutEvents` - -Publishes events to a custom EventBridge event bus. Use this connector to fan out resource events to multiple downstream consumers through EventBridge rules. - -For configuration details, see [Amazon EventBridge connector](./connector-eventbridge.html). - -### STAC API - -Step type: `rest` - -Publishes resource metadata as STAC Items to a STAC-compliant geoportal API. This connector maps SDMA assets to the STAC Item specification, including geometry, bounding box, temporal properties, and per-file asset entries. - -For configuration details, see [STAC API connector](./connector-stac.html). - -### Metadata derivation - -Operations: `s3Lookup`, `restLookup`, `ddbLookup` +###### Important @@ -109 +36 @@ Operations: `s3Lookup`, `restLookup`, `ddbLookup` -Enriches resource metadata by looking up data from an external source and applying it to assets or files. Supports Amazon S3 (CSV/JSON), REST API, and Amazon DynamoDB sources. +Connectors execute the configuration you provide. SDMA validates the structure and security credentials of a connector, but does not evaluate the business logic of the configuration. For example, if a connector is configured to produce malformed data or send metadata to an unintended destination, SDMA executes those actions as configured. Test connector configurations thoroughly in a non-production environment before enabling them. @@ -111 +38 @@ Enriches resource metadata by looking up data from an external source and applyi -For configuration details, see [Metadata derivation connector](./connector-derivation.html). +## How connectors work @@ -113 +40 @@ For configuration details, see [Metadata derivation connector](./connector-deriv -### Composable step types +SDMA provides a small set of integration primitives — REST, S3, Deadline Cloud, Lambda, EventBridge — that connect your assets with the outside world. Each primitive can produce new content and bring it into SDMA (derive), or send content outward to external systems (publish). For a simple, single action — push metadata to an API, look up values from a CSV, submit a compute job — use a single-step publisher or deriver. When you need more, compose those same primitives into multi-step workflows within a single trigger. The primitives are the same either way. @@ -115 +42 @@ For configuration details, see [Metadata derivation connector](./connector-deriv -Each connector type is also a reusable step type. Step types are composable — a single trigger can combine steps of different types to build multi-step workflows. For example, a trigger can write metadata to Amazon S3, then invoke a Lambda function to post-process it, and finally publish an event to EventBridge: +A connector is a governed composition of five surfaces: @@ -116,0 +44 @@ Each connector type is also a reusable step type. Step types are composable — + * **Resource surface** – What external resources can be discovered, validated, and displayed. Not all connectors need this — it is for connectors that expose selectable values or linked external identities. @@ -118,20 +46 @@ Each connector type is also a reusable step type. Step types are composable — - "triggers": [{ - "resources": ["asset"], - "events": ["create"], - "steps": [ - { - "stepType": "s3PutObject", - "s3Config": { "objectKey": "raw/${asset.assetId}.json" } - }, - { - "stepType": "lambdaInvoke", - "lambdaConfig": { "functionArn": "arn:aws:lambda:us-west-2:123456789012:function:post-process" } - }, - { - "stepType": "eventBridgePutEvents", - "eventBridgeConfig": { "eventBusArn": "arn:aws:events:us-west-2:123456789012:event-bus/my-bus", "source": "sdma", "detailType": "AssetPublished" } - } - ] - }] - -This composability means you do not need a separate connector for each destination. A single connector can orchestrate a complete workflow by combining different step types in a single trigger. + * **Mapping surface** – How values move between SDMA and the external system. Field mappings transform resource metadata into the structure the external system expects, and can route external response values back into SDMA metadata. @@ -139 +48 @@ This composability means you do not need a separate connector for each destinati -## Triggers + * **Execution surface** – What triggers exist, what events they match, and what steps run. SDMA has two execution models: the lookup-derivation model for bulk metadata enrichment, and the trigger+steps model for step-composed workflows and publishing. @@ -141 +50 @@ This composability means you do not need a separate connector for each destinati -Triggers define which resource events activate the connector. Each trigger specifies: + * **Security surface** – How the connector authenticates, what secrets it uses, and what IAM roles it assumes. @@ -143 +52 @@ Triggers define which resource events activate the connector. Each trigger speci - * `resources` – The resource types to match. + * **Operational surface** – How failures are handled, what policy disables runaway behavior, and what invocation state is visible for troubleshooting. @@ -145 +53,0 @@ Triggers define which resource events activate the connector. Each trigger speci - * `events` – The event types to match. @@ -147 +54,0 @@ Triggers define which resource events activate the connector. Each trigger speci - * `stepType` – The type of step to execute (for publish connectors). @@ -149 +55,0 @@ Triggers define which resource events activate the connector. Each trigger speci - * `steps` – One or more step configurations to execute in order. @@ -150,0 +57 @@ Triggers define which resource events activate the connector. Each trigger speci +### Execution flow @@ -151,0 +59 @@ Triggers define which resource events activate the connector. Each trigger speci +When a resource event occurs (for example, an asset is created or a file upload completes): @@ -152,0 +61 @@ Triggers define which resource events activate the connector. Each trigger speci + 1. SDMA evaluates all connectors associated with the asset’s template. @@ -154 +63 @@ Triggers define which resource events activate the connector. Each trigger speci -### Supported resources + 2. For each connector, SDMA matches the event against the connector’s triggers. SDMA skips any connector with no matching trigger for the current event — attachment does not imply execution. @@ -156 +65 @@ Triggers define which resource events activate the connector. Each trigger speci -The following resource types can activate a trigger. + 3. Every trigger whose resource type, event, and filters match the current event executes independently, each producing its own invocation record. @@ -158,5 +67 @@ The following resource types can activate a trigger. -Resource | Description ----|--- -`project` | Triggered when a project is created, updated, or deleted. -`asset` | Triggered when an asset is created, updated, or deleted. -`file` | Triggered when a file within an asset is processed. + 4. For triggers with a `dependsOn` list, SDMA checks whether the upstream connectors have completed within the same lifecycle context before dispatching. @@ -164 +69 @@ Resource | Description -### Supported events + 5. Each trigger’s steps execute in order, sharing intermediate state through `$temp.*` variables. @@ -166 +71 @@ Resource | Description -The following events can activate a trigger. + 6. SDMA routes results back to the asset record through output routing — as metadata attributes, derived files, or both. @@ -168,8 +73 @@ The following events can activate a trigger. -Event | Description ----|--- -`create` | A resource is created. -`update` | A resource is updated.