Enrich your datasets with enterprise context: Migrating from legacy Matters to semantic datasets in Amazon Fast

When you’ve been managing Amazon Fast legacy Matters alongside your datasets, you realize the problem: two property that should keep completely synchronized, every with its personal permissions, lineage, and versioning. Column synonyms drift. Calculated fields diverge. A rename within the dataset breaks the Legacy Subject silently. Now you can use Amazon Fast to embed that enterprise context instantly into the dataset itself by Dataset Enrichment within the new knowledge prep expertise. Column descriptions, synonyms, calculated fields, customized directions, and enterprise guidelines all stay alongside the info. Dataset Enrichment bakes enterprise context instantly into the dataset. Every part (permissions, semantics, AI context) travels with the info and is mechanically inherited by something constructed on prime of it. One asset, one supply of fact, one place to manipulate.

On this put up, we stroll by what Dataset Enrichment is, the way it differs from legacy Matters, and supply three migration eventualities with step-by-step steering so you’ll be able to transfer your online business context into the dataset layer with confidence.

Subject is now the multi-dataset semantic and reasoning layer, the assemble the place a number of datasets are composed, relationships are outlined, enterprise metrics are authored, and enterprise terminology is mapped. Reasonably than introducing a net-new assemble, we’re re-purposing Subject to satisfy this position extra utterly. Shifting dataset-intrinsic semantics all the way down to the place they belong, and elevating Subject to personal the cross-dataset relationships, metrics, and enterprise terminology that it was at all times meant to hold. This isn’t a beauty change. It establishes a clear, forward-looking structure that helps each deterministic BI workflows and versatile AI-driven analytics from a shared semantic basis. It additionally units up the framework for catalog integration.

What’s Matters (legacy)

Legacy Matters supplied the preliminary method to including enterprise context to datasets in Amazon Fast Sight. It saved column synonyms, calculated fields, named entities, filters, and customized directions in a separate object that sat on prime of the dataset, linked however independently managed. Going ahead, we classify present Matters as legacy. The brand new model of Matters is being elevated to a multi-dataset semantic layer. A single-entry level for cross-dataset Q&A that lets enterprise customers and AI workflows question throughout a number of enriched datasets in a single dialog. Dataset Enrichment is the muse that makes this attainable: every dataset should carry its personal semantic context earlier than Matters can unify them at the next degree.

Key variations: Matters (legacy) vs. Dataset Enrichment (new knowledge prep)

What stays the identical throughout migration

Not every part modifications. These facets of your Fast Sight surroundings stay unchanged throughout migration:

• Guidelines datasets don’t require migration. Rule-based logic continues to work as earlier than.
• SPICE storage and Direct Question mode are unaffected. Your knowledge entry patterns stay the identical no matter whether or not you utilize Matters or Dataset Enrichment.
• Dashboards and analyses aren’t rebuilt. They use the enriched dataset.
• The user-facing Q&A interplay mannequin doesn’t change. Customers nonetheless ask questions in pure language by Amazon Fast chat. The distinction is invisible to them.

Why transfer enterprise context into the dataset?

When enterprise metadata lives in a separate legacy Subject object, you handle two property that should keep synchronized. Permissions, lineage, and discoverability all break up throughout boundaries. Dataset Enrichment collapses this:

• Single supply of fact. Enterprise context travels with the info. There’s no separate asset to float out of sync.
• Automated inheritance. Any dashboard, evaluation, legacy Subject, or AI-powered chat characteristic constructed on the enriched dataset inherits the semantic context with out extra configuration.
• Easier governance. One asset to handle permissions, audit, and model management.
• AI-readiness. The dataset turns into self-describing for pure language querying. Amazon Fast’s chat expertise can resolve ambiguous enterprise language instantly from dataset metadata with out requiring a separate Subject.

What’s modified within the dataset definition as a part of Dataset Enrichment

The brand new knowledge prep expertise introduces semantic_model_configuration on datasets. It has two layers:

Column-level metadata (inside TableMap):

Dataset-level metadata (inside SemanticMetadata):

Mapping from Legacy Subject fields to Dataset Enrichment:

Answer overview

The migration is a four-step course of. You establish your goal dataset, find the supply legacy Subject, and run a Python script that extracts the legacy Subject’s metadata and writes it into the dataset’s SemanticModelConfiguration by the Fast Sight API. The script handles column descriptions, synonyms, calculated fields, named entities, filters, and customized directions in a single cross. No handbook UI work required.

Stipulations

Earlier than you start, confirm that you’ve the next in place:

• AWS Command Line Interface (AWS CLI) v2 (model 2.34.50 or later) put in and configured with legitimate credentials. Run aws sts get-caller-identity to verify your session is energetic. See the AWS CLI set up information for setup directions. Confirm with aws --version.
• Python 3.6 or later is required to run the mapping script in Step 3. Confirm with python3 --version.
• Amazon Fast Enterprise version with Q enabled. The migration depends on Fast Sight APIs which can be solely out there in Enterprise accounts with the Q add-on energetic.
• A goal dataset utilizing the brand new knowledge prep expertise. The dataset should use DataPrepConfiguration (not LogicalTableMap) and have column IDs on all InputColumns. See Making a dataset in new knowledge prep for extra particulars.
• An present legacy Subject (supply) with columns, descriptions, synonyms, and optionally calculated fields, entities, and filters that you simply wish to migrate.
• AWS Identification and Entry Administration (IAM) permissions for the caller id: quicksight:DescribeDataSet, quicksight:UpdateDataSet, and quicksight:DescribeTopic. See Fast Sight IAM actions for the complete coverage reference.

Migration eventualities

The correct method is dependent upon your present setup. We cowl three eventualities, from easiest to most concerned.

State of affairs 1: Legacy datasets with no legacy Matters

Your dataset was constructed utilizing the traditional Fast Sight expertise and has no semantic layer of any type. Customers see uncooked column names (TXN_DT, CUST_ID, AMT_USD) in dashboards, filters, and tooltips. When somebody sorts a pure language query in Amazon Fast chat, the system has zero context to work with: no descriptions, no synonyms, no enterprise guidelines. A query like “what’s our income this quarter?” returns nothing as a result of nothing maps the phrase “income” to any column within the schema.

Legacy datasets ( utilizing LogicalTableMap relatively than DataPrepConfiguration) don’t help Dataset Enrichment. There’s no in-place improve path at present. You possibly can’t add SemanticModelConfiguration to a legacy dataset, and there’s no migration API that converts one to the opposite. You possibly can select to question the dataset instantly with uncooked column names utilizing Dataset Q&A characteristic.

State of affairs 2: Legacy Matters with legacy datasets

A Subject sits on prime of a legacy dataset, offering the semantic layer your customers rely upon, column synonyms, calculated fields, named entities, named filters, and customized directions. Customers question by the Subject and get outcomes. However beneath, the dataset itself has no enrichment. It nonetheless makes use of LogicalTableMap and exposes uncooked column names with no descriptions or enterprise context of its personal.

Like State of affairs 1, the underlying legacy dataset doesn’t help SemanticModelConfiguration. There’s no approach to push your legacy Subject metadata into the dataset instantly. The trail ahead is creating a brand new dataset utilizing the brand new knowledge prep expertise, migrating your legacy Subject’s metadata into it as Dataset Enrichment, validating the outcomes, and reducing over. Alternatively you need to use the Dataset Q&A characteristic with the legacy dataset.

State of affairs 3: Legacy Matters with new knowledge prep datasets

Present state: You already use the brand new knowledge prep expertise to your dataset. It has DataPrepConfiguration with SourceTableMap, TransformStepMap, and DestinationTableMap. However a legacy Subject continues to be layered on prime offering the semantic context your customers depend on: column synonyms, calculated fields, named entities, filters, and customized directions. The dataset construction helps enrichment natively. It hasn’t been utilized but. That is the one state of affairs the place a direct, in-place migration is feasible. Your dataset already speaks the best API language, so that you don’t must recreate it.

As a result of the dataset makes use of DataPrepConfiguration, you’ll be able to cross SemanticModelConfiguration on to the update-data-set API. This implies you can too:

• Migrate column descriptions and synonyms out of your legacy Subject into ColumnProperties.Description and ColumnProperties.AdditionalNotes on the dataset. Each column that had a ColumnDescription or ColumnSynonyms array within the legacy Subject will get an equal entry within the dataset’s semantic layer.
• Migrate calculated fields from legacy Subject-level expressions into CreateColumnsStep entries in DataPrepConfiguration. These change into first-class computed columns within the dataset, seen in dashboards and out there for pure language queries and not using a legacy Subject.
• Migrate named entities, filters, and customized directions into CustomInstructions.InlineCustomInstruction.InstructionText. Entity definitions, enterprise guidelines, and components documentation are preserved as structured textual content that the chat system reads at question time.
• Maintain the legacy Subject energetic throughout migration. Each the legacy Subject and the dataset enrichment can coexist briefly. You validate the enriched dataset’s Q&A conduct towards the legacy Subject’s and solely take away the legacy Subject after you’re prepared.
• Run the legacy Subject migration with a script. As a result of each describe-topic and update-data-set are API calls, the transformation could be automated end-to-end. Extract legacy Subject metadata, rework it, apply it to the dataset, and confirm. No handbook UI work required.

The next step automates the enrichment of an Amazon Fast Sight dataset with semantic metadata extracted from a Fast Sight legacy Subject. It transfers enterprise context, column descriptions, synonyms, calculated fields, named entities, filters, and customized directions from a legacy Subject instantly into the dataset’s SemanticModelConfiguration.

Step overview

You want 4 items of knowledge earlier than operating the script. Get the dataset ID and Subject ID (Legacy) from the Fast console URLs (proven in Steps 1 and a pair of), and make sure your AWS account ID and AWS Area.

Required parameters

• ACCOUNT_ID = <<0358134xxxxx>>
• REGION = <>
• DATASET_ID = <<30c1e26a-e02a-4f2a-ac48-xxa1xxxxx916>>
• TOPIC_ID = <>

Step 1: Get goal dataset ID

Open the Fast Sight console. Within the left navigation panel, choose Information, then swap to the Datasets tab. Choose the dataset that you simply wish to enrich. The dataset ID is the UUID on the finish of your browser’s URL.

Instance:

Step 2: Get the supply legacy Subject ID

Open the Fast Sight console. Within the left navigation panel, choose Information, then swap to the Matters tab. Choose the supply legacy Subject that you simply wish to migrate from. The Subject ID is the string on the finish of your browser’s URL.

Instance:

Step 3: Python code to complement a dataset with info from a legacy Subject

The next script connects to the Fast Sight APIs, retrieves your present legacy Subject metadata and goal dataset schema, then intelligently maps each piece of enterprise context to its new residence within the dataset’s SemanticModelConfiguration. Column descriptions and synonyms change into ColumnProperties, calculated fields are injected as CreateColumnsStep entries in your knowledge prep pipeline, and named entities, filters, and customized directions are consolidated right into a structured CustomInstructions textual content block. After the mapping is full, it applies the enriched payload on to your dataset by a SigV4-signed API name. Your dataset turns into totally self-describing in a single execution, with no handbook UI work and no danger of transcription errors.

Save the next script as enrich_dataset.py.

"""
enrich_dataset_local.py --- Enrich a Fast Sight dataset with Legacy Subject metadata.
Utilization: python enrich_dataset_local.py    
NOTE: Makes use of uncooked SigV4 API name to bypass botocore validation (requires botocore >= 1.43.11).
"""
import json
import sys
import boto3
import botocore.session
from botocore.auth import SigV4Auth
from botocore.awsrequest import AWSRequest
import urllib3
http = urllib3.PoolManager()

def raw_update_data_set(aws_account_id, dataset_id, payload, area):
    """Name UpdateDataSet by way of uncooked SigV4 request, bypassing botocore validation."""
    session = botocore.session.get_session()
    credentials = session.get_credentials().get_frozen_credentials()
    url = f"https://quicksight.{area}.amazonaws.com/accounts/{aws_account_id}/data-sets/{dataset_id}"
    physique = json.dumps(payload)
    headers = {"Content material-Sort": "utility/json"}
    request = AWSRequest(methodology="PUT", url=url, knowledge=physique, headers=headers)
    SigV4Auth(credentials, "quicksight", area).add_auth(request)
    response = http.request("PUT", request.url, headers=dict(request.headers), physique=request.physique)
    response_body = json.hundreds(response.knowledge.decode("utf-8"))
    if response.standing >= 400:
        increase Exception(f"UpdateDataSet failed ({response.standing}): {json.dumps(response_body, indent=2)}")
    return response_body

def enrich_dataset(topic_id, dataset_id, aws_account_id, aws_region):
    qs = boto3.shopper("quicksight", region_name=aws_region)
    print(f"n{'='*60}")
    print(f"  Dataset Enrichment")
    print(f"{'='*60}")
    print(f"  Subject ID:   {topic_id}")
    print(f"  Dataset ID: {dataset_id}")
    print(f"  Account:    {aws_account_id}")
    print(f"  Area:     {aws_region}")
    print(f"{'='*60}n")
    # --- Fetch inputs from Fast Sight APIs ---
    print("[1/5] Fetching dataset schema...")
    ds_response = qs.describe_data_set(AwsAccountId=aws_account_id, DataSetId=dataset_id)
    ds = ds_response["DataSet"]
    print(f"      Dataset: {ds['Name']}")
    print(f"      ARN: {ds['Arn']}")
    print("[2/5] Fetching matter metadata...")
    topic_response = qs.describe_topic(AwsAccountId=aws_account_id, TopicId=topic_id)
    topic_raw = topic_response
    print(f"      Subject: {topic_raw['Topic'].get('Title', topic_id)}")
    print(f"      Datasets in matter: {len(topic_raw['Topic']['DataSets'])}")
    # --- Match matter dataset by ARN ---
    target_arn = ds["Arn"]
    topic_datasets = topic_raw["Topic"]["DataSets"]
    matter = None
    for td in topic_datasets:
        if td.get("DatasetArn") == target_arn:
            matter = td
            break
    if matter is None:
        print(f"      WARNING: No matter dataset matched ARN {target_arn}. Falling again to index 0.")
        matter = topic_datasets[0]
    else:
        print(f"      Matched matter dataset by ARN ✓")
    custom_instructions_str = topic_raw.get("CustomInstructions", {}).get("CustomInstructionsString", "")
    # --- Extract dataset construction ---
    physical_table_map = ds["PhysicalTableMap"]
    import_mode = ds.get("ImportMode", "SPICE")
    uses_new_experience = "DataPrepConfiguration" in ds
    print(f"      Expertise: {'NEW (DataPrepConfiguration)' if uses_new_experience else 'OLD (LogicalTableMap)'}")
    if uses_new_experience:
        data_prep = ds["DataPrepConfiguration"]
    else:
        logical_table_map = ds["LogicalTableMap"]
    ptable_key = checklist(physical_table_map.keys())[0]
    # Auto-detect supply sort
    source_entry = physical_table_map[ptable_key]
    if "RelationalTable" in source_entry:
        input_columns = source_entry["RelationalTable"]["InputColumns"]
        source_type = "RelationalTable"
    elif "S3Source" in source_entry:
        input_columns = source_entry["S3Source"]["InputColumns"]
        source_type = "S3Source"
    elif "CustomSql" in source_entry:
        input_columns = source_entry["CustomSql"]["Columns"]
        source_type = "CustomSql"
    else:
        increase ValueError(f"Unknown bodily desk sort: {checklist(source_entry.keys())}")
    print(f"      Supply sort: {source_type}")
    print(f"      Enter columns: {len(input_columns)}")
    col_id_map = {col["Name"]: col.get("Id", col["Name"]) for col in input_columns}
    # --- 1. Map column descriptions and synonyms ---
    print("n[3/5] Constructing enrichment...")
    column_metadata = []
    for tc in matter["Columns"]:
        col_id = col_id_map.get(tc["ColumnName"])
        if not col_id:
            proceed
        column_metadata.append({
            "ColumnNames": [tc["ColumnName"]],
            "ColumnProperties": [{
                "Description": {"Text": tc.get("ColumnDescription", "")},
                "AdditionalNotes": {"Text": ("The synonym for the column is " + ", ".join(tc.get("ColumnSynonyms", []))) if tc.get("ColumnSynonyms") else ""}
            }]
        })
    # --- 2. Add calculated fields ---
    if uses_new_experience:
        dest_key = checklist(data_prep["DestinationTableMap"].keys())[0]
        last_step = data_prep["DestinationTableMap"][dest_key]["Source"]["TransformOperationId"]
        for step_id, step in data_prep["TransformStepMap"].gadgets():
            if "CastColumnTypesStep" in step:
                for op in step["CastColumnTypesStep"].get("CastColumnTypeOperations", []):
                    op.pop("SubType", None)
        for i, calc in enumerate(matter.get("CalculatedFields", [])):
            col_name = calc["CalculatedFieldName"].higher().change(" ", "_")
            col_id = f"col-{col_name.decrease().change('_', '-')}"
            step_id = f"topic-calc-step-{i}"
            data_prep["TransformStepMap"][step_id] = {
                "CreateColumnsStep": {
                    "Alias": f"Add {calc['CalculatedFieldName']}",
                    "Columns": [{"ColumnName": col_name, "ColumnId": col_id, "Expression": calc["Expression"]}],
                    "Supply": {"TransformOperationId": last_step}
                }
            }
            last_step = step_id
            column_metadata.append({
                "ColumnNames": [col_name],
                "ColumnProperties": [{
                    "Description": {"Text": calc.get("CalculatedFieldDescription", "")},
                    "AdditionalNotes": {"Text": ("The synonym for the column is " + ", ".join(calc.get("CalculatedFieldSynonyms", []))) if calc.get("CalculatedFieldSynonyms") else ""}
                }]
            })
        data_prep["DestinationTableMap"][dest_key]["Source"]["TransformOperationId"] = last_step
    else:
        lt_key = checklist(logical_table_map.keys())[0]
        lt = logical_table_map[lt_key]
        if "DataTransforms" in lt:
            for rework in lt["DataTransforms"]:
                if "CastColumnTypeOperation" in rework:
                    rework["CastColumnTypeOperation"].pop("SubType", None)
        for i, calc in enumerate(matter.get("CalculatedFields", [])):
            col_name = calc["CalculatedFieldName"].higher().change(" ", "_")
            col_id = f"col-{col_name.decrease().change('_', '-')}"
            create_op = {"CreateColumnsOperation": {"Columns": [{"ColumnName": col_name, "ColumnId": col_id, "Expression": calc["Expression"]}]}}
            insert_idx = len(lt.get("DataTransforms", []))
            for idx, t in enumerate(lt.get("DataTransforms", [])):
                if "ProjectOperation" in t:
                    insert_idx = idx
                    break
            lt.setdefault("DataTransforms", []).insert(insert_idx, create_op)
            for t in lt.get("DataTransforms", []):
                if "ProjectOperation" in t:
                    t["ProjectOperation"]["ProjectedColumns"].append(col_name)
                    break
            column_metadata.append({
                "ColumnNames": [col_name],
                "ColumnProperties": [{
                    "Description": {"Text": calc.get("CalculatedFieldDescription", "")},
                    "AdditionalNotes": {"Text": ("The synonym for the column is " + ", ".join(calc.get("CalculatedFieldSynonyms", []))) if calc.get("CalculatedFieldSynonyms") else ""}
                }]
            })
    # --- 3. Construct CustomInstructions textual content ---
    traces = []
    if custom_instructions_str:
        traces.append(custom_instructions_str)
    for entity in matter.get("NamedEntities", []):
        synonyms = ", ".be a part of(entity.get("EntitySynonyms", []))
        fields = ", ".be a part of(d.get("FieldName", "") for d in entity.get("Definition", []))
        entry = f"Entity {entity['EntityName']}: {entity.get('EntityDescription', '')}"
        if fields:
            entry += f" (fields: {fields})"
        if synonyms:
            entry += f" (synonyms: {synonyms})"
        traces.append(entry)
    for f in matter.get("Filters", []):
        fname = f.get("FilterName", "Unnamed")
        discipline = f.get("OperandFieldName", "")
        ftype = f.get("FilterType", "")
        if ftype == "CATEGORY_FILTER":
            cf = f.get("CategoryFilter", {})
            func = cf.get("CategoryFilterFunction", "")
            values = cf.get("Fixed", {}).get("CollectiveConstant", {}).get("ValueList", [])
            inverse = cf.get("Inverse", False)
            op = "NOT IN" if inverse else "IN"
            traces.append(f"Enterprise Rule - {fname}: Filter the place {discipline} {func} {op} [{', '.join(values)}].")
        elif ftype == "NUMERIC_RANGE_FILTER":
            nrf = f.get("NumericRangeFilter", {})
            agg = nrf.get("Aggregation", "")
            inclusive = nrf.get("Inclusive", False)
            minimal = nrf.get("Fixed", {}).get("RangeConstant", {}).get("Minimal", "")
            most = nrf.get("Fixed", {}).get("RangeConstant", {}).get("Most", "")
            if minimal:
                op2 = ">=" if inclusive else ">"
                traces.append(f"Enterprise Rule - {fname}: Filter the place {agg}({discipline}) {op2} {minimal}.")
            elif most:
                op2 = "<=" if inclusive else "<"
                traces.append(f"Enterprise Rule - {fname}: Filter the place {agg}({discipline}) {op2} {most}.")
    instruction_text = "n".be a part of(traces)
    # --- 4. Assemble SemanticModelConfiguration ---
    topic_desc = topic_raw["Topic"].get("Description", "")
    if uses_new_experience:
        dest_key = checklist(data_prep["DestinationTableMap"].keys())[0]
    else:
        dest_key = lt_key
    semantic_model_config = {
        "TableMap": {
            "semantic-table": {
                "Alias": "Semantic Desk",
                "DestinationTableId": dest_key,
                "SemanticMetadata": {"ColumnMetadata": column_metadata}
            }
        },
    }
    if topic_desc or instruction_text:
        semantic_entry = {}
        if topic_desc:
            semantic_entry["Description"] = {"Textual content": topic_desc}
        if instruction_text:
            semantic_entry["CustomInstructions"] = [{"InlineCustomInstruction": {"InstructionText": instruction_text}}]
        semantic_model_config["SemanticMetadata"] = [semantic_entry]
    # --- Print enrichment abstract ---
    print(f"      Columns enriched: {len(column_metadata)}")
    print(f"      Calculated fields added: {len(matter.get('CalculatedFields', []))}")
    print(f"      Named entities: {len(matter.get('NamedEntities', []))}")
    print(f"      Filters as guidelines: {len(matter.get('Filters', []))}")
    print(f"      Customized directions: {len(instruction_text)} chars")
    # --- 5. Construct and apply replace payload ---
    update_payload = {
        "Title": ds["Name"],
        "PhysicalTableMap": physical_table_map,
        "ImportMode": import_mode,
        "SemanticModelConfiguration": semantic_model_config
    }
    if uses_new_experience:
        update_payload["DataPrepConfiguration"] = data_prep
    else:
        update_payload["LogicalTableMap"] = logical_table_map
    print("n[4/5] Making use of replace by way of SigV4 API name...")
    update_response = raw_update_data_set(aws_account_id, dataset_id, update_payload, aws_region)
    print(f"n[5/5] SUCCESS!")
    print(f"      Dataset '{ds['Name']}' enriched with Subject metadata.")
    print(f"      Standing: {update_response.get('Standing', 'OK')}")
    print(f"      Ingestion: {update_response.get('IngestionId', 'N/A')}")

if __name__ == "__main__":
    if len(sys.argv) < 5:
        print(__doc__)
        sys.exit(1)
    topic_id = sys.argv[1]
    dataset_id = sys.argv[2]
    aws_account_id = sys.argv[3]
    aws_region = sys.argv[4]
    attempt:
        enrich_dataset(topic_id, dataset_id, aws_account_id, aws_region)
    besides Exception as e:
        print(f"n ERROR: {e}")
        sys.exit(1)

3.2 Run the script

Execute the script, passing your 4 parameters (Subject ID, Dataset ID, Account ID, and Area):

3.3 Anticipated output

After the script completes, affirm that the enrichment was utilized accurately by inspecting the dataset by the Fast Sight API or within the Fast Sight console:

3.4 Confirm the output file

# Examine the three sections exist:
python -c "import json; d=json.load(open('enriched-payload.json')); print(checklist(d.keys()))"
# Anticipated: ['physical_table_map', 'data_prep_configuration', 'semantic_model_configuration']

• physical_table_map: unchanged from the supply dataset.
• data_prep_configuration: with an added CreateColumnsStep for calculated fields.
• semantic_model_configuration: with column descriptions, synonyms, and customized directions (formulation, entities, and enterprise guidelines).

Validation finest practices

Throughout all eventualities, we advocate the next validation method:

• Construct a validation query set. Earlier than migration, doc 10–20 pure language questions that work accurately towards your present legacy Subject. These change into your regression check suite.
• Run side-by-side comparisons. Ask the identical questions towards the legacy Subject (earlier than removing) and towards the enriched dataset. Outcomes ought to match.
• Take a look at edge instances. Strive ambiguous questions, synonym variations, and multi-step queries. The enriched dataset ought to deal with these as successfully as (or higher than) the legacy Subject did.
• Validate with precise customers. Have 2–3 enterprise customers check the enriched dataset chat expertise earlier than you retire the legacy Subject. Their pure phrasing could reveal gaps in your synonym or entity protection.
• Doc behavioral variations. Some behaviors will differ between legacy Subject-based querying and dataset enrichment querying:
  • Legacy Subject filters are pre-defined and selectable; enterprise guidelines in customized directions require the consumer to reference them by identify or intent.
  • Legacy Subject calculated fields help aggregation-level expressions; dataset calculated fields function row-by-row (aggregations occur at question time).
  • Named entities in legacy Matters have structured metadata; in dataset enrichment they’re text-based directions.

Key concerns

• The update-data-set API requires a full substitute of all configuration on each name. Incremental updates to particular person columns aren’t supported.
• Named entities and filters don’t have devoted API fields in SemanticModelConfiguration. They have to be expressed as textual content inside CustomInstructions.
• Not all legacy Subject calculation expressions translate on to dataset-level expressions. Aggregation-based calculations might have restructuring.
• At the moment, you’ll be able to’t improve legacy datasets in-place. You need to create a brand new dataset with the brand new knowledge prep expertise to complement it.

Clear up

To keep away from incurring ongoing costs, delete the AWS sources (IAM roles, Fast Sight knowledge sources, insurance policies) that you simply created as a part of experimentation. For directions, see the Amazon Fast documentation.

Conclusion

Dataset Enrichment in Amazon Fast strikes enterprise intelligence metadata from a separate legacy Subject layer into the dataset itself. One asset carries its personal enterprise that means: column descriptions, synonyms, calculation logic, entity definitions, and enterprise guidelines, all ruled in a single place.

To get began, establish your highest-value dataset, apply enrichment utilizing the patterns on this put up, and check it in Amazon Fast chat. For the complete SemanticModelConfiguration API reference, see the Amazon Fast documentation. For questions and group dialogue, go to the Amazon Fast Neighborhood..

In regards to the authors