How to Load Contact Transactions and Distribute Survey

Here we show the recommended way to load contacts and transactions, together called Contact Transactions, into a mailing list and do a distribution. By recommended we mean that large amounts of data should be divided into batches and then processed serially instead of sending many batches at the same time.

In the example program here we simulate importing a data file by generating random values.

APIs Used

Use these APIs in this order:

API	Notes
Create Mailing List	You could, of course, manually create a mailing list, if you use the same one each day.
Create Transaction Batch	Here we create an empty batch by not giving it any transaction IDs when we create it. We add transactions to it in the next steps.
Import Transactions into Batch	This contains contact and transaction information together in an array of JSON objects.
Check the Import Status	Since this should be done serially, keep checking the status of the import until the previous batch is done before you submit a new one.
Get the Import Summary	Get the results of the import, along with any errors that occurred.
Create Distribution	The distribution is sent to everyone in the transaction batch.
Generate Distribution Links	You could optionally create distribution links instead of sending emails. You might do that if you use a different system than Qualtrics to send emails.

Contact Transactions JSON Data Structure

When you create contacts and transactions you might think the best way to do that is to do those as separate steps. But we have a more efficient procedure which is to use the Create Transaction Contacts Import API described below. It uses one large JSON array for both the contacts and the transactions and processes them all at once.

The structure of the JSON file used for import contains these elements:

Element	Description
contacts	A JSON array [] containing contacts and transactions.
transactionData	The fields here are free-form.

The fields outside of the curly braces are contact fields. You should provide at a minimum:

lastName
externalReference
email
firstName

A sample contact and transaction looks like this:

{
		'transactionData': {
			'product': 'Credit Card',
			'amount': 5477.77,
			'bankBranch': 'London',
			'saleDate': '2019-01-18 00:00:00'
		},
		'lastName': 'yntfyp',
		'externalReference': 'iwvcnlz',
		'email': 'qekr.yntfyp@qualtrics.com',
		'firstName': 'qekr'
	}

**transactionMeta: ** This metadata shows both the transaction batch ID and the fields that make up the transactionData

transactionMeta:
{
		'batchId': 'BT_bdRnzDOypaPboTX',
		'fields': ['product', 'amount', 'saleDate', 'bankBranch']
	}

Here is what the entire JSON payload would look like:

{
  'contacts': [{
    'transactionData': {
      'product': 'Credit Card',
      'amount': 5477.77,
      'bankBranch': 'London',
      'saleDate': '2019-01-18 00:00:00'
    },
    'lastName': 'yntfyp',
    'externalReference': 'iwvcnlz',
    'email': 'qekr.yntfyp@qualtrics.com',
    'firstName': 'qekr'
  }, {
    'transactionData': {
      'product': 'Home Equity',
      'amount': 19567.72,
      'bankBranch': 'Los Angeles',
      'saleDate': '2019-01-31 00:00:00'
    },
    'lastName': 'vlyz',
    'externalReference': 'xoqyidu',
    'email': 'daj.vlyz@qualtrics.com',
    'firstName': 'daj'
  },
         ...
         
  }, {
    'transactionData': {
      'product': 'Car Loan',
      'amount': 16713.19,
      'bankBranch': 'Los Angeles',
      'saleDate': '2019-12-14 00:00:00'
    },
    'lastName': 'jftwish',
    'externalReference': 'ylhpgsh',
    'email': 'bsyc.jftwish@qualtrics.com',
    'firstName': 'bsyc'
  }],
  'transactionMeta': {
    'batchId': 'BT_bdRnzDOypaPboTX',
    'fields': ['product', 'amount', 'saleDate', 'bankBranch']
  }
}

Limits

JSON body cannot exceed 5 MB

No more than 100 transactions per batch.

De-dupe Settings

The directory should have deduplication enabled. This is set for email.

Troubleshooting and Reasons for Failure

Some of the reasons that an import would fail include:

transactionData or embeddedData are not JSON objects.

the email does not contain the at sign @

Deduplication

When a record with the email already exists then the transactions are added onto the existing contact.

Sample Code

Code and Qualtrics Support

Our support team does not offer assistance or consultation on custom coding, but you can always try asking our community of dedicated users. If you’d like to know more about our custom coding services you can commission, please contact your Qualtrics Account Executive.

The Python code below does all the steps outlined above in the order outlined above. It emulates reading an import text file, such as would be created by your interface process, by generating random values. The sample data here is from a bank, but, of course, it can be any data since transaction data is free-form.

First, you need to create a survey and a message (email) to distribute manually. So create those and then save those values in the file config.yml:

surveyId: "SV_ebdf20QralHTzQF"
libraryId: "UR_8cRedheujEbcxgN"
messageId: "MS_1Mt2Nj5kczdnjMh"
directoryId: "POOL_9H1MGk0YcdGVXKt"
batchSize: 100

To run this code use Python 3.x and send it the number of contacts to create as a command-line argument. You will need to use pip to import simplejson and requests.

Most of the code is self-explanatory. Some items to note are:

rsp.json()["result"]["id"] gives us the ID of the object we just created.
The apiToken and dataCenter should be set as environment variables.
We check rsp.json()["result"]["percentComplete"]and rsp.json()["result"]["status"]to know when the batch is finished processing.
The distribution is sent to the batch id "transactionBatchId": transactionBatchId as opposed to individual contacts or transactions.
We generate random number and strings for most fields. Other ones we pick from a list by generating a random integer and using that as the index.
We divide the number of contacts by the batch size to determine how many times to run the API. The remainder of this division is the number of records added in the last batch.

import string
import random
import csv
import requests
import simplejson as json
from time import gmtime, strftime
from datetime import date
from datetime import timedelta
import datetime
import sys
import os
import time
import yaml

def genDate():
    newdate = date.today() + timedelta(days=random.randint(-90, 90))
    return newdate.strftime('%Y-%m-%d %H:%M:%S')

def strGenerator(size):
    return ''.join(random.choice(string.ascii_lowercase) for _ in range(size))

def createMailingList(directoryId,apiToken, dataCenter):

    headers = {
        "content-type": "application/json",
        "x-api-token": apiToken,
       }

    url = "https://{0}.qualtrics.com/API/v3/directories/{1}/mailinglists".format(dataCenter,directoryId)
    
    data = {}
    data["name"] = "mass " + datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')

    rsp = requests.post(url, json=data, headers=headers)
    return rsp.json()["result"]["id"], data["name"]

def createTransactionBatch(directoryId,apiToken,dataCenter):

    headers = {
       "content-type": "application/json",
       "x-api-token": apiToken,
    }

    data = {
       "creationDate" : datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
       }

    url = "https://{0}.qualtrics.com/API/v3/directories/{1}/transactionbatches".format(dataCenter,directoryId)

    rsp = requests.post(url, json=data, headers=headers)
    return rsp.json()["result"]["id"]

def createContactTransactions(numContacts,batchSize,batchId,directoryId,mailingListId,apiToken,dataCenter):

    url = "https://{0}.qualtrics.com/API/v3/directories/{1}/mailinglists/{2}/transactioncontacts".format(dataCenter,directoryId,mailingListId)

    headers = {
    "content-type": "application/json",
    "x-api-token": apiToken,
    }

    products=["Home Equity", "Credit Card", "Car Loan"]
    bankBranch=["Los Angeles", "Paris", "London"]
    accountManager=["George Washington", "Thomas Jefferson", "Thomas Cromwell"]

    modulo = 0
    if numContacts < batchSize:
       looper = 1
       batchSize = numContacts
    else:
       modulo = numContacts % batchSize
       looper = int(numContacts/batchSize)
       
    batch = 1
    if modulo > 0:
       looper = looper + 1

    for i in range(looper):
        print('processing batch %s' %  str(batch))
        if modulo > 0 and i == looper - 1:
            batchSize = modulo
        contactTrans = []
        for j in range(batchSize):
              firstName = strGenerator(random.randint(3, 7))
              lastName = strGenerator(random.randint(3, 7)) 
              contact = {
               "firstName":  firstName,
               "lastName": lastName,
               "transactionData": {
                  "product": products[random.randint(0, 2)],
                  "amount": round(random.uniform(100, 20000),2),
                  "saleDate": genDate(),
                  "bankBranch": bankBranch[random.randint(0, 2)] 
               },
               "externalReference": strGenerator(7),
               "email": firstName + "." + lastName + "@qualtrics.com" 
              }
              contactTrans.append(contact)
 
        data ={
            "transactionMeta": {
            "batchId": batchId,
            "fields":  [
               "product",
               "amount",
               "saleDate",
               "bankBranch"
            ] 
          },
           "contacts": contactTrans
         } 

        print (data)

        rsp = requests.post(url, json=data, headers=headers)
       
        checkProgress(rsp.json()["result"]["id"],mailingListId,directoryId,apiToken,dataCenter)
        batch = batch + 1

def checkProgress(importId,mailingListId,directoryId,apiToken,dataCenter):
    percentComplete = 0
    status = "in progress"

    headers = {
    "content-type": "application/json",
    "x-api-token": apiToken,
    }
    
    url = "https://{0}.qualtrics.com/API/v3/directories/{1}/mailinglists/{2}/transactioncontacts/{3}".format(dataCenter,directoryId,mailingListId,importId)

    j=""
    
    while percentComplete < 100 and status == "in progress":
        print ('checking progress of import %s mailing list %s ' % (importId, mailingListId))
        rsp = requests.get(url, headers=headers)
        percentComplete = rsp.json()["result"]["percentComplete"]
        status = rsp.json()["result"]["status"]
        print ("percentComplete =" + str(percentComplete) + " status=" + status)
        print ("====================================================")
        j = rsp.json()
        time.sleep(5)

    print (j)


def createDistribution(surveyId,libraryId,messageId,transactionBatchId,apiToken,dataCenter):
    
    url = "https://{0}.qualtrics.com/API/v3/distributions".format(dataCenter)

    onemonth = date.today() + timedelta(days=30)

    headers = {
    "x-api-token": apiToken,
    "Content-Type": "application/json"
    }

    data = {
        "surveyLink": {
          "surveyId": surveyId,
          "expirationDate": onemonth.strftime('%Y-%m-%dT%H:%M:%SZ'),
          "type":"Individual"
        },
        "header": {
          "fromEmail": "noreply@qemailserver.com",
          "fromName": "Survey User",
          "replyToEmail": "noreply@qualtrics.com", 
          "subject": "Volume Test"
        },
        "message": {
          "libraryId": libraryId,
          "messageId": messageId
        },
        "recipients": {
          "transactionBatchId": transactionBatchId
        },
        "sendDate": datetime.datetime.now().replace(microsecond=0).isoformat() + 'Z'
       }

    rsp = requests.post(url, json=data, headers=headers)
    print ("distribution %s created" % rsp.json()["result"]["id"])

def main():
    try:
        numContacts = sys.argv[1]
    except IndexError:
        print ("usage: number of contacts to create")
        sys.exit(2) 

    try:
        apiToken = os.environ['APIKEY']
        dataCenter = os.environ['DATACENTER']
    except KeyError:
        print("set environment variables APIKEY and DATACENTER")
        sys.exit(2) 
  
    config = yaml.load(open('config.yml')) 
    directoryId = config['directoryId']
    batchSize = config['batchSize']
 
    mailingListId, mailingListName = createMailingList(directoryId,apiToken, dataCenter)
    print("mailing list {0} created, name {1}".format(mailingListId,mailingListName))
 
    batchId = createTransactionBatch(directoryId,apiToken, dataCenter)
    print("transaction batch {0} created".format(batchId))

    createContactTransactions(int(numContacts),batchSize,batchId,directoryId,mailingListId,apiToken, dataCenter)

    surveyId = config['surveyId']
    libraryId = config['libraryId']
    messageId = config['messageId']

    createDistribution(surveyId,libraryId,messageId,batchId,apiToken,dataCenter)

if __name__ == "__main__":
    main()

Here is a sample of the JSON that is passed to the API.

{
  'contacts': [{
    'transactionData': {
      'product': 'Credit Card',
      'amount': 5477.77,
      'bankBranch': 'London',
      'saleDate': '2019-01-18 00:00:00'
    },
    'lastName': 'yntfyp',
    'externalReference': 'iwvcnlz',
    'email': 'qekr.yntfyp@qualtrics.com',
    'firstName': 'qekr'
  }, {
    'transactionData': {
      'product': 'Home Equity',
      'amount': 19567.72,
      'bankBranch': 'Los Angeles',
      'saleDate': '2019-01-31 00:00:00'
    },
    'lastName': 'vlyz',
    'externalReference': 'xoqyidu',
    'email': 'daj.vlyz@qualtrics.com',
    'firstName': 'daj'
  },
         ...
         
  }, {
    'transactionData': {
      'product': 'Car Loan',
      'amount': 16713.19,
      'bankBranch': 'Los Angeles',
      'saleDate': '2019-12-14 00:00:00'
    },
    'lastName': 'jftwish',
    'externalReference': 'ylhpgsh',
    'email': 'bsyc.jftwish@qualtrics.com',
    'firstName': 'bsyc'
  }],
  'transactionMeta': {
    'batchId': 'BT_bdRnzDOypaPboTX',
    'fields': ['product', 'amount', 'saleDate', 'bankBranch']
  }
}

Here is what the output looks like. Note the distribution ID is shown at the end.

checking progress of import PGRS_cPbbLjvlvss1Xi5 mailing list CG_86uhHTzhRyWm25n 
percentComplete =0.0 status=in progress
====================================================
checking progress of import PGRS_cPbbLjvlvss1Xi5 mailing list CG_86uhHTzhRyWm25n 
percentComplete =99.0 status=in progress
====================================================
checking progress of import PGRS_cPbbLjvlvss1Xi5 mailing list CG_86uhHTzhRyWm25n 
percentComplete =100.0 status=complete
====================================================
{'result': {'status': 'complete', 'percentComplete': 100.0, 'contacts': {'count': {'added': 100, 'updated': 0, 'failed': 0}}, 'transactions': {'count': {'created': 100}}}, 'meta': {'requestId': '515f5f2f-51c6-486b-8791-390af4182620', 'httpStatus': '200 - OK'}}
distribution EMD_ijU3ttFFNWFyd9Y created

This is the mailing it created. Screen Shot 2019-01-31 at 14.38.18.png

Retrying Failed Transactions

If any errors are found the Get Import Summary API will provide out a list of failed records like shown below. You can use that to look at the input data and determine what data is incorrect.

{
  "result": {
    "contacts": {
      "count": {
        "added": 0,
        "updated": 0,
        "failed": 0
      }
    },
    "transactions": {
      "count": {
        "created": 0
      }
    },
    "invalidEmails": ["yyd.ysxgyd.qualtrics.com", "upbid.emjlj.qualtrics.com", "fyg.hytrwdo.qualtrics.com", "cof.bvbr.qualtrics.com", "jgxeff.jhar.qualtrics.com", "qaesi.atawi.qualtrics.com", "qfa.dhtopm.qualtrics.com", "bss.qhhfmll.qualtrics.com", "jcdnxzj.qbnspv.qualtrics.com", "outkc.avoxhx.qualtrics.com", "njnbplr.ckwpxr.qualtrics.com", "lrtsycy.yvf.qualtrics.com", "mie.yrhnyk.qualtrics.com", "xiddo.okbldm.qualtrics.com", "mmdscbx.gtylwdj.qualtrics.com", "gkqapjv.jstrocg.qualtrics.com", "cimf.prscjb.qualtrics.com", "rkdc.tosxp.qualtrics.com", "itegbas.ohpgyx.qualtrics.com", "feunr.drovpd.qualtrics.com", "ero.tmsbaw.qualtrics.com", "ueyn.rcvlqqw.qualtrics.com", "avtca.xpko.qualtrics.com", "jgniqg.xth.qualtrics.com", "npqe.huohwx.qualtrics.com", "tyxak.ykft.qualtrics.com", "can.ptjj.qualtrics.com", "prnl.jiodvhj.qualtrics.com", "uczzx.elili.qualtrics.com", "loklxgq.uilq.qualtrics.com", "rag.mcwuy.qualtrics.com", "ynxujj.uovrnv.qualtrics.com", "golcq.luoh.qualtrics.com", "qidyz.ankpujm.qualtrics.com", "nzwueq.gwaw.qualtrics.com", "lsthrd.upma.qualtrics.com", "arcn.avmwvp.qualtrics.com", "fsvq.dyjbon.qualtrics.com", "ybpx.ugtdy.qualtrics.com", "xjavrgx.znsmo.qualtrics.com", "nplq.pno.qualtrics.com", "tqqhe.hyu.qualtrics.com", "psi.wlr.qualtrics.com", "vuaia.jpg.qualtrics.com", "oidcw.ntvfgrb.qualtrics.com", "xrohx.kitiz.qualtrics.com", "ybhwplm.sve.qualtrics.com", "xbxb.ceku.qualtrics.com", "irlkhbs.bwjtxd.qualtrics.com", "bzrj.plbgg.qualtrics.com", "nyeegl.fdyzkff.qualtrics.com",  "ciow.irmcjn.qualtrics.com", "hcx.qpoar.qualtrics.com"],
    "percentComplete": 100.0,
    "status": "complete"
  },
  "meta": {
    "httpStatus": "200 - OK",
    "requestId": "c18a86c8-877a-4989-92f3-9be0726a19bc"
  }
}

APIs Used

Contact Transactions JSON Data Structure

Limits​

De-dupe Settings​

Troubleshooting and Reasons for Failure​

Deduplication​

Sample Code

Code and Qualtrics Support​

Retrying Failed Transactions

Limits

De-dupe Settings

Troubleshooting and Reasons for Failure

Deduplication

Code and Qualtrics Support