Aggregate data load

martynwb — Sat, 16 Nov 2024 00:35:28 GMT

Hi, I'm trying to load some data from an API. The API documentation states that I have to use distinct trigger_id and use the latest testimestamp (the time stamp) to avoid duplication.

However, I can't get the load script to work. I'm given an unspecified error:

TESSIAN_DIRECT:
LOAD
user As TessianOpsDirUser,
testimestamp As TessianOpsDirTimestamp
FROM [lib://QVD_S1/InfoSec/OBJ_TESSIAN_DIRECT_OPS.QVD]
(qvd);
LEFT Join (TESSIAN_DIRECT)
LOAD
alert_type AS TessianOpsDirAlertType,
//anomalous_recipient AS TessianOpsDirAnomalousRecipient,
attachments_total_size As TessianOpsDirAttachmentSize,
changes_made As TessianOpsDirChangesMade,
check_performed_by As TessianOpsDirCheckPerformed,
//checkpoint As TessianOpsDirCheckpoint,
filter_name As TessianOpsDirFilterName,
final_outcome As TessianOpsDirFinalOutcome,
//flag_reason As TessianOpsDirFlagReason,
initial_response As TessianOpsDirInitialResponse,
message_id As TessianOpsDirMessageId,
message_shown_to_user As TessianOpsDirMessageShownToUser,
//misdirected_email_prevented As TessianOpsDirMisdirectedEmailPrevented,
module As TessianOpsDirModule,
number_of_attachments As TessianOpsDirNoOfAttachments,
priority As TessianOpsDirPriority,
salutation_extracted As TessianOpsDirSalutationExtracted,
subject As TessianOpsDirSubject,
subsequent_action As TessianOpsDirSubsequenbtAction,
suggested_recipient As TessianOpsDirSuggestedReceipient,
Max(testimestamp) As TessianOpsDirTimestamp,
//tracking_id As TessianOpsDirTrackId,
trigger_id As TessianOpsDirTrigId,
updated_at As TessianOpsDirUpdated,

//email_is_sensitive As TessianOpsDirEmailIsSensitive,
//justification_for_override As TessianOpsDirJustificationForOverride,
request_for_override As TessianOpsDirRequestOverride,
unauthorized_email_prevented As TessianOpsDirUnauthEmailPrevented,
//attachment As TessianOpsDirAttachment,
//bccrecipients As TessianOpsDirAttachmentbccRecipient,
__KEY_data
FROM [lib://QVD_S1/InfoSec/OBJ_TESSIAN_DIRECT_OPS.QVD]
(qvd)
WHERE module='enforcer'
//Resident TESSIAN_DIRECT
GROUP BY trigger_id;

Some help would be appreciated. Thanks

topic Aggregate data load in Integration, Extension & APIs

Aggregate data load