QvSource Load statement Add in

[marco_puccetti]

Hello i need to load some twitter tables (in this case the user lookup table) in order to get the necessary informations, but when i add this statement

LOAD

    $(page) as tpage, //Twitter page reference

    user_id as UserLookup_user_id,

  user_name as UserLookup_user_name,

       ............

The script ends correctly but the exectution and the application blocks.

Is there anything have i to change?

The page variable is 15 digits twitter ID.

Thanks

Marco

[marco_puccetti]

Hello, now i get this error for the followers table

Campo non trovato - <user_id>

TW_Followers:

LOAD

    276965569137896 as tpage,

    user_id as Followers_user_id,

  user_name as Followers_user_name,

  user_screen_name as Followers_user_screen_name,

  user_location as Followers_user_location,

  user_profile_image_url as Followers_user_profile_image_url,

  user_description as Followers_user_description,

  user_url as Followers_user_url,

  user_geo_enabled as Followers_user_geo_enabled,

  user_protected as Followers_user_protected,

  user_followers_count as Followers_user_followers_count,

  user_friends_count as Followers_user_friends_count,

  user_listed_count as Followers_user_listed_count,

  user_favourites_count as Followers_user_favourites_count,

  user_statuses_count as Followers_user_statuses_count,

  user_created_at as Followers_user_created_at,

  timestamp#(mid(user_created_at, 9, 2) & '-' & mid(user_created_at, 5, 3) & '-' & mid(user_created_at, 27, 4) & ' ' & mid(user_created_at, 12, 8), 'DD-MMM-YYYY hh:mm:ss') as Followers_user_user_created_at_timestamp,

  date#(mid(user_created_at, 9, 2) & '-' & mid(user_created_at, 5, 3) & '-' & mid(user_created_at, 27, 4), 'DD-MMM-YYYY') as Followers_user_user_created_at_date,

  time#(mid(user_created_at, 12, 8), 'hh:mm:ss') as Followers_user_user_created_at_time,

  user_utc_offset as Followers_user_utc_offset,

  user_time_zone as Followers_user_time_zone,

  user_verified as Followers_user_verified,

  user_lang as Followers_user_lang,

  user_follow_request_sent as Followers_user_follow_request_sent,

  user_is_translator as Followers_user_is_translator,

  user_following as Followers_user_following,

  user_notifications as Followers_user_notifications,

  status_id as Followers_status_id,

  status_created_at as Followers_status_created_at,

  timestamp#(mid(status_created_at, 9, 2) & '-' & mid(status_created_at, 5, 3) & '-' & mid(status_created_at, 27, 4) & ' ' & mid(status_created_at, 12, 8), 'DD-MMM-YYYY hh:mm:ss') as Followers_status_status_created_at_timestamp,

  date#(mid(status_created_at, 9, 2) & '-' & mid(status_created_at, 5, 3) & '-' & mid(status_created_at, 27, 4), 'DD-MMM-YYYY') as Followers_status_status_created_at_date,

  time#(mid(status_created_at, 12, 8), 'hh:mm:ss') as Followers_status_status_created_at_time,

  status_text as Followers_status_text,

  status_lang as Followers_status_lang,

  status_source as Followers_status_source,

  status_truncated as Followers_status_truncated,

  status_in_reply_to_screen_name as Followers_status_in_reply_to_screen_name,

  status_in_reply_to_status_id as Followers_status_in_reply_to_status_id,

  status_in_reply_to_user_id as Followers_status_in_reply_to_user_id,

  status_retweet_count as Followers_status_retweet_count,

  status_favorite_count as Followers_status_favorite_count,

  status_retweeted as Followers_status_retweeted,

  status_favorited as Followers_status_favorited,

  status_possibly_sensitive as Followers_status_possibly_sensitive,

  status_hashtag_count as Followers_status_hashtag_count,

  status_hash_tags as Followers_status_hash_tags,

  status_first_hash_tag as Followers_status_first_hash_tag,

  status_url_count as Followers_status_url_count,

  status_expanded_urls as Followers_status_expanded_urls,

  status_first_expanded_url as Followers_status_first_expanded_url,

  status_user_mentions_count as Followers_status_user_mentions_count,

  status_user_mentions as Followers_status_user_mentions,

  status_first_user_mention as Followers_status_first_user_mention,

  status_media_count as Followers_status_media_count,

  status_media_expanded_urls as Followers_status_media_expanded_urls,

  status_first_media_expanded_url as Followers_status_first_media_expanded_url,

  status_symbols_count as Followers_status_symbols_count,

  status_symbols as Followers_status_symbols,

  status_first_symbol as Followers_status_first_symbol,

  status_media_photo_count as Followers_status_media_photo_count,

  status_media_photo_urls as Followers_status_media_photo_urls,

  status_first_media_photo_url as Followers_status_first_media_photo_url,

  status_place_id as Followers_status_place_id,

  status_place_url as Followers_status_place_url,

  status_place_place_type as Followers_status_place_place_type,

  status_place_name as Followers_status_place_name,

  status_place_full_name as Followers_status_place_full_name,

  status_place_country_code as Followers_status_place_country_code,

  status_place_country as Followers_status_place_country,

  status_place_bounding_box_type as Followers_status_place_bounding_box_type,

  status_place_attributes as Followers_status_place_attributes,

  status_geo_type as Followers_status_geo_type,

  status_geo_latitude as Followers_status_geo_latitude,

  status_geo_longitude as Followers_status_geo_longitude,

  status_coordinates_type as Followers_status_coordinates_type,

  status_coordinates_latitude as Followers_status_coordinates_latitude,

  status_coordinates_longitude as Followers_status_coordinates_longitude

FROM

  [http://localhost:5555/QVSource/TwitterConnectorV2/?table=Followers&appID=Social Media Analizer&tokenEncrypted=&tokenSecretEncrypted=.....](qvx)

This is the row html return:

WebResponse	{"errors":[{"message":"Rate limit exceeded","code":88}]} [Headers - content-encoding: ;strict-transport-security: max-age=631138519;x-connection-hash: c1fe2b3b75dd52e62cea87ae3aa76533;x-rate-limit-limit: 15;x-rate-limit-remaining: 0;x-rate-limit-reset: 1431685523;x-response-time: 108;Content-Length: 81;Content-Type: application/json;charset=utf-8;Date: Fri, 15 May 2015 10:18:34 GMT;Set-Cookie: guest_id=v1%3A143168511440044621; Domain=.twitter.com; Path=/; Expires=Sun, 14-May-2017 10:18:34 UTC;Server: tsa_f;]
Message	Error requesting table Followers for TwitterConnectorV2
Exception	IndustrialCodeBox.APIConnector.APIConnectorException: Errore del server remoto: (429) Too Many Requests. ---> System.Net.WebException: Errore del server remoto: (429) Too Many Requests. in System.Net.HttpWebRequest.GetResponse() in IndustrialCodeBox.APIConnector.Interfaces.OAuth1APIHelper.MakeRequest(eHTTPMethod method, String urlWithoutQueryParams, String token, String tokenSecret, IDictionary`2 unencodedUrlParams, Object enclosedEntityEgPost, String[] trackingKeys) in IndustrialCodeBox.APIConnector.Interfaces.OAuth1APIHelper.MakeRequest(eHTTPMethod method, String urlWithQueryParams, String token, String tokenSecret, String[] trackingKeys) in TwitterConnectorV2.TwitterConnectorV2.writeFollowersOrFollowing(RequestPackage r, Inputs inputs, Boolean followers) in TwitterConnectorV2.TwitterConnectorV2.onRequestTable(RequestPackage r) --- Fine della traccia dello stack dell'eccezione interna --- in TwitterConnectorV2.TwitterConnectorV2.onRequestTable(RequestPackage r) in IndustrialCodeBox.QVSource.QVSourceWebServer.handlerCom(Object o)

Thanks

Marco

[chrisbrain]

‌HI Marco, in the error message you can see this is a rate limit error from Twitter.

I Believe I explained this in another response and linked to an example on our GitHub page for gathering all friends/followers.

[marco_puccetti]

I have a question related also to the normalization of the datas: Table normalization

Thanks

Marco