Data Operations

Data operations handle API calls, data transformations, and business logic within your client-facing scripts. They enable communication with external services and manipulation of workflow data.

Operation Types

interface PreNextStepOperation {
  id: string
  type: "retrieveData" | "updateData" | "validate" | "transform" | "trackEvent"
  name: string
  description: string
  config: OperationConfig
  conditional?: ConditionalLogic
}

1. Retrieve Data Operations

Fetch data from external APIs and store it in fetched records.

Basic API Call

{
  "id": "get_customer_profile",
  "type": "retrieveData",
  "name": "Retrieve Customer Profile",
  "description": "Fetch customer information and order history",
  "config": {
    "apiEndpoint": "/ShopifyNode/get_customer",
    "apiMethod": "GET",
    "apiParams": {
      "email": "instanceData.customer_email"
    },
    "headers": {
      "Authorization": "Bearer {credentials.shopify_access_token}",
      "Content-Type": "application/json"
    },
    "responseMapping": [
      {
        "sourceField": "customer",
        "targetFetchedRecord": "customerProfile"
      },
      {
        "sourceField": "customer.orders",
        "targetFetchedRecord": "orderHistory"
      }
    ]
  }
}

Advanced API Configuration

{
  "id": "get_shipping_rates",
  "type": "retrieveData",
  "name": "Calculate Shipping Rates",
  "description": "Get available shipping options with rates",
  "config": {
    "apiEndpoint": "/EasyPostNode/get_rates",
    "apiMethod": "POST",
    "apiParams": {
      "from_address": "instanceData.return_address",
      "to_address": "instanceData.customer_address",
      "parcel": {
        "weight": "instanceData.total_weight",
        "length": "instanceData.total_length",
        "width": "instanceData.total_width", 
        "height": "instanceData.total_height"
      }
    },
    "timeout": 30000,
    "retries": 3,
    "retryDelay": 1000,
    "responseMapping": [
      {
        "sourceField": "shipment.rates",
        "targetFetchedRecord": "shippingRates",
        "transform": {
          "type": "array_map",
          "expression": "item => ({ id: item.id, service: item.service, rate: item.rate, delivery_days: item.delivery_days })"
        }
      }
    ],
    "errorHandling": {
      "onError": "continue",
      "defaultValue": [],
      "logError": true
    }
  }
}

GraphQL API Calls

{
  "id": "shopify_product_query",
  "type": "retrieveData",
  "name": "Query Product Variants",
  "description": "Get product variants using GraphQL",
  "config": {
    "apiEndpoint": "/ShopifyNode/graphql",
    "apiMethod": "POST",
    "apiFormat": "graphql",
    "query": `
      query getProductVariants($productId: ID!) {
        product(id: $productId) {
          id
          title
          variants(first: 100) {
            edges {
              node {
                id
                title
                price
                availableForSale
                selectedOptions {
                  name
                  value
                }
              }
            }
          }
        }
      }
    `,
    "variables": {
      "productId": "instanceData.selected_product_id"
    },
    "responseMapping": [
      {
        "sourceField": "data.product.variants.edges",
        "targetFetchedRecord": "productVariants",
        "transform": {
          "type": "graphql_edges_to_array",
          "nodeField": "node"
        }
      }
    ]
  }
}

2. Update Data Operations

Send data to external APIs to create, update, or delete records.

Create Record

{
  "id": "create_return_request",
  "type": "updateData",
  "name": "Create Return Request",
  "description": "Submit return request to fulfillment system",
  "config": {
    "apiEndpoint": "/DepositoNode/create_return",
    "apiMethod": "POST",
    "apiParams": {
      "order_id": "instanceData.order_id",
      "return_items": "instanceData.selected_line_items",
      "return_reason": "instanceData.return_reason",
      "customer_info": {
        "email": "instanceData.customer_email",
        "name": "instanceData.customer_name",
        "phone": "instanceData.phone_number"
      },
      "return_address": "instanceData.return_address",
      "notes": "instanceData.notes"
    },
    "responseMapping": [
      {
        "sourceField": "return_request.id",
        "targetInstanceData": "return_request_id"
      },
      {
        "sourceField": "return_request.tracking_number",
        "targetInstanceData": "tracking_number"
      }
    ],
    "successConditions": [
      {
        "field": "status",
        "operator": "equals",
        "value": "success"
      }
    ]
  }
}

Update Existing Record

{
  "id": "update_customer_profile",
  "type": "updateData",
  "name": "Update Customer Preferences",
  "description": "Update customer notification preferences",
  "config": {
    "apiEndpoint": "/ShopifyNode/update_customer",
    "apiMethod": "PUT",
    "apiParams": {
      "customer_id": "fetchedRecords.customerProfile.id",
      "metafields": [
        {
          "namespace": "preferences",
          "key": "return_notifications",
          "value": "instanceData.notification_preferences",
          "type": "json_string"
        }
      ]
    },
    "conditional": {
      "runIf": {
        "field": "notification_preferences_changed",
        "operator": "equals",
        "value": true
      }
    }
  }
}

3. Validation Operations

Perform data validation using business rules and external API calls.

Custom Validation

{
  "id": "validate_return_eligibility",
  "type": "validate",
  "name": "Check Return Eligibility",
  "description": "Validate that items are eligible for return",
  "config": {
    "validationRules": [
      {
        "type": "custom",
        "message": "Some items are past the return window",
        "customValidator": "validateReturnWindow",
        "params": {
          "items": "instanceData.selected_line_items",
          "orderDate": "instanceData.order_created_at",
          "returnWindow": 30
        }
      },
      {
        "type": "api_validation",
        "message": "Items cannot be returned due to store policy",
        "apiEndpoint": "/ValidationNode/check_return_policy",
        "apiParams": {
          "order_id": "instanceData.order_id",
          "items": "instanceData.selected_line_items"
        },
        "successCondition": {
          "field": "all_eligible",
          "operator": "equals",
          "value": true
        }
      }
    ],
    "onValidationFailure": {
      "action": "block",
      "showErrors": true,
      "allowOverride": false
    }
  }
}

Async Validation

{
  "id": "verify_inventory",
  "type": "validate",
  "name": "Verify Product Inventory",
  "description": "Check product availability for exchanges",
  "config": {
    "apiEndpoint": "/InventoryNode/check_availability",
    "apiMethod": "POST",
    "apiParams": {
      "products": "instanceData.exchange_items",
      "location": "instanceData.fulfillment_location"
    },
    "validationRules": [
      {
        "type": "inventory_check",
        "message": "Some exchange items are out of stock",
        "field": "availability",
        "condition": {
          "allItemsAvailable": true
        }
      }
    ],
    "loadingMessage": "Checking inventory...",
    "timeoutMessage": "Inventory check timed out. Please try again."
  }
}

4. Transform Operations

Manipulate and calculate data within the workflow.

Data Calculations

{
  "id": "calculate_refund_amount",
  "type": "transform",
  "name": "Calculate Refund Amount",
  "description": "Calculate total refund with fees and adjustments",
  "config": {
    "calculations": [
      {
        "field": "subtotal",
        "expression": "instanceData.selected_line_items.reduce((sum, item) => sum + (item.price * item.quantity), 0)"
      },
      {
        "field": "tax_amount",
        "expression": "subtotal * (instanceData.tax_rate || 0.08)"
      },
      {
        "field": "restocking_fee",
        "expression": "fetchedRecords.customerProfile.tier === 'premium' ? 0 : Math.min(subtotal * 0.05, 25)"
      },
      {
        "field": "processing_fee",
        "expression": "instanceData.refund_method === 'original_payment_method' ? 0 : 2.99"
      },
      {
        "field": "total_refund",
        "expression": "subtotal + tax_amount - restocking_fee - processing_fee"
      }
    ],
    "outputMapping": [
      {
        "sourceField": "subtotal",
        "targetInstanceData": "refund_subtotal"
      },
      {
        "sourceField": "total_refund",
        "targetInstanceData": "final_refund_amount"
      }
    ]
  }
}

Data Transformation

{
  "id": "format_address_data",
  "type": "transform",
  "name": "Format Address Data",
  "description": "Transform address into shipping API format",
  "config": {
    "transformations": [
      {
        "field": "shipping_address",
        "expression": {
          "name": "instanceData.first_name + ' ' + instanceData.last_name",
          "street1": "instanceData.address.street",
          "street2": "instanceData.address.apartment",
          "city": "instanceData.address.city",
          "state": "instanceData.address.state",
          "zip": "instanceData.address.postal_code",
          "country": "instanceData.address.country || 'US'",
          "phone": "instanceData.phone_number"
        }
      },
      {
        "field": "formatted_items",
        "expression": "instanceData.selected_line_items.map(item => ({ sku: item.sku, quantity: item.quantity, reason: item.return_reason }))"
      }
    ]
  }
}

Conditional Calculations

{
  "id": "apply_customer_tier_benefits",
  "type": "transform",
  "name": "Apply Customer Tier Benefits", 
  "description": "Apply benefits based on customer tier",
  "config": {
    "conditionalCalculations": [
      {
        "condition": {
          "field": "fetchedRecords.customerProfile.tier",
          "operator": "equals",
          "value": "premium"
        },
        "calculations": [
          {
            "field": "shipping_discount",
            "expression": "instanceData.shipping_cost"
          },
          {
            "field": "expedited_processing",
            "expression": "true"
          }
        ]
      },
      {
        "condition": {
          "field": "fetchedRecords.customerProfile.total_orders",
          "operator": "greater_than",
          "value": 10
        },
        "calculations": [
          {
            "field": "loyalty_discount",
            "expression": "instanceData.subtotal * 0.05"
          }
        ]
      }
    ],
    "defaultCalculations": [
      {
        "field": "shipping_discount",
        "expression": "0"
      },
      {
        "field": "loyalty_discount",
        "expression": "0"
      }
    ]
  }
}

5. Track Event Operations

Log analytics events and track user behavior.

Basic Event Tracking

{
  "id": "track_return_started",
  "type": "trackEvent",
  "name": "Track Return Initiation",
  "description": "Log when customer starts return process",
  "config": {
    "eventName": "return_process_started",
    "properties": {
      "order_id": "instanceData.order_id",
      "customer_email": "instanceData.customer_email",
      "item_count": "instanceData.selected_line_items.length",
      "order_value": "instanceData.order_total",
      "customer_tier": "fetchedRecords.customerProfile.tier"
    },
    "trackingProviders": [
      {
        "provider": "segment",
        "userId": "instanceData.customer_id"
      },
      {
        "provider": "google_analytics",
        "customDimensions": {
          "customer_tier": "fetchedRecords.customerProfile.tier",
          "return_reason": "instanceData.return_reason"
        }
      }
    ]
  }
}

Conversion Tracking

{
  "id": "track_return_completion",
  "type": "trackEvent",
  "name": "Track Return Completion",
  "description": "Log successful return completion",
  "config": {
    "eventName": "return_completed",
    "eventCategory": "ecommerce",
    "eventAction": "return_submitted",
    "properties": {
      "return_id": "instanceData.return_request_id",
      "return_value": "instanceData.final_refund_amount",
      "processing_time": "Date.now() - instanceData.process_start_time",
      "return_method": "instanceData.refund_method"
    },
    "conversionGoals": [
      {
        "provider": "google_ads",
        "conversionId": "AW-123456789",
        "conversionLabel": "return_completion"
      }
    ]
  }
}

Operation Chaining

Sequential Operations

{
  "preNextStepOperations": [
    {
      "id": "validate_customer",
      "type": "validate",
      "config": { "..." }
    },
    {
      "id": "fetch_order_details",
      "type": "retrieveData",
      "dependsOn": ["validate_customer"],
      "config": { "..." }
    },
    {
      "id": "calculate_eligibility",
      "type": "transform",
      "dependsOn": ["fetch_order_details"],
      "config": { "..." }
    }
  ]
}

Parallel Operations

{
  "preNextStepOperations": [
    {
      "id": "fetch_customer_profile",
      "type": "retrieveData",
      "config": { "..." },
      "parallel": true
    },
    {
      "id": "fetch_order_history",
      "type": "retrieveData", 
      "config": { "..." },
      "parallel": true
    },
    {
      "id": "process_parallel_results",
      "type": "transform",
      "dependsOn": ["fetch_customer_profile", "fetch_order_history"],
      "config": { "..." }
    }
  ]
}

Error Handling

Operation-Level Error Handling

{
  "config": {
    "errorHandling": {
      "onError": "retry",
      "maxRetries": 3,
      "retryDelay": [1000, 2000, 4000],
      "backoffStrategy": "exponential",
      "fallbackOperation": {
        "type": "updateData",
        "config": {
          "apiEndpoint": "/FallbackService/log_error"
        }
      },
      "continueOnError": false,
      "errorMessage": "We're experiencing technical difficulties. Please try again."
    }
  }
}

Global Error Handling

{
  "client_workflow_customization": {
    "errorHandling": {
      "globalErrorHandler": true,
      "errorDisplayComponent": "custom_error_display",
      "retryButtonText": "Try Again",
      "contactSupportText": "Contact Support",
      "logErrors": true,
      "errorReporting": {
        "service": "sentry",
        "apiKey": "{credentials.sentry_dsn}"
      }
    }
  }
}

Performance Optimization

Caching

{
  "config": {
    "caching": {
      "enabled": true,
      "ttl": 300,
      "key": "customer_profile_{instanceData.customer_email}",
      "strategy": "cache_first",
      "invalidateOn": ["customer_updated", "order_modified"]
    }
  }
}

Lazy Loading

{
  "config": {
    "lazyLoad": true,
    "loadTrigger": {
      "type": "user_interaction",
      "element": "exchange_tab"
    },
    "loadingComponent": "skeleton_loader",
    "priority": "low"
  }
}

Best Practices

1. API Design

• Use consistent endpoint naming and HTTP methods
• Implement proper authentication and authorization
• Handle rate limiting gracefully
• Use appropriate HTTP status codes

2. Data Management

• Validate data at operation boundaries
• Use appropriate data types and formats
• Implement proper error handling and logging
• Cache expensive operations when appropriate

3. Performance

• Minimize API calls by batching requests
• Use parallel operations where possible
• Implement proper timeout and retry logic
• Consider lazy loading for non-critical data

4. Security

• Never expose sensitive credentials in client-facing scripts
• Validate and sanitize all input data
• Use proper authentication for all API calls
• Log security-related events for monitoring

Next Steps

Continue exploring the CanScript documentation to build more advanced workflows and integrations.