QVSource Load Error

[marco_puccetti]

Hello, i get this error during the Load process for the twitter follower and following i have noticed the 429 Error (Too Many Requests).

Have i to do any setting to avoid this?

Below the error from QvSource

[chrisbrain]

Twitter does have strict API call limits, usually scoped to 15 minute windows.

You can see these in detail in the RateLimit table in the connector and most of the other tables give an indication of how many API calls they use and how many are allowed per 15 minute window.

Getting all followers/friends is quite tricky - we have tried to put together a sample/demo here:
QVSource-Sample-Scripts-For-QlikView/Twitter Connector/Load All Followers And Friends at master · QV...

It uses a combination of the FollowerIds/FriendIds tables and the Post_XXXX_Info related tables to establish whether the previous calls were rate limited and then the UserLookupById table to get the actual details.

Please note though that the QlikView and Qlik Sense apps over at:
https://github.com/QVSource/

Are given away freely and unsupported, however if you find any specific issues with the QVSource tables themselves please let us know with steps to reproduce.

[marco_puccetti]

I've noticed the use of "Post_FollowerIds_Info" table instead of the "Followers", isn't it?

The "Follower" table structure is still available?

user_id as Followers_user_id,

  user_name as Followers_user_name,

  user_screen_name as Followers_user_screen_name,

  user_location as Followers_user_location,

  user_profile_image_url as Followers_user_profile_image_url,

  user_description as Followers_user_description,

  user_url as Followers_user_url,

  user_geo_enabled as Followers_user_geo_enabled,

  user_protected as Followers_user_protected,

  user_followers_count as Followers_user_followers_count,

  user_friends_count as Followers_user_friends_count,

  user_listed_count as Followers_user_listed_count,

  user_favourites_count as Followers_user_favourites_count,

  user_statuses_count as Followers_user_statuses_count,

  user_created_at as Followers_user_created_at,

  timestamp#(mid(user_created_at, 9, 2) & '-' & mid(user_created_at, 5, 3) & '-' & mid(user_created_at, 27, 4) & ' ' & mid(user_created_at, 12, 8), 'DD-MMM-YYYY hh:mm:ss') as Followers_user_user_created_at_timestamp,

  date#(mid(user_created_at, 9, 2) & '-' & mid(user_created_at, 5, 3) & '-' & mid(user_created_at, 27, 4), 'DD-MMM-YYYY') as Followers_user_user_created_at_date,

  time#(mid(user_created_at, 12, 8), 'hh:mm:ss') as Followers_user_user_created_at_time,

  user_utc_offset as Followers_user_utc_offset,

  user_time_zone as Followers_user_time_zone,

  user_verified as Followers_user_verified,

  user_lang as Followers_user_lang,

  user_follow_request_sent as Followers_user_follow_request_sent,

  user_is_translator as Followers_user_is_translator,

  user_following as Followers_user_following,

  user_notifications as Followers_user_notifications,

  status_id as Followers_status_id,

  status_created_at as Followers_status_created_at,

  timestamp#(mid(status_created_at, 9, 2) & '-' & mid(status_created_at, 5, 3) & '-' & mid(status_created_at, 27, 4) & ' ' & mid(status_created_at, 12, 8), 'DD-MMM-YYYY hh:mm:ss') as Followers_status_status_created_at_timestamp,

  date#(mid(status_created_at, 9, 2) & '-' & mid(status_created_at, 5, 3) & '-' & mid(status_created_at, 27, 4), 'DD-MMM-YYYY') as Followers_status_status_created_at_date,

  time#(mid(status_created_at, 12, 8), 'hh:mm:ss') as Followers_status_status_created_at_time,

  status_text as Followers_status_text,

  status_lang as Followers_status_lang,

  status_source as Followers_status_source,

  status_truncated as Followers_status_truncated,

  status_in_reply_to_screen_name as Followers_status_in_reply_to_screen_name,

  status_in_reply_to_status_id as Followers_status_in_reply_to_status_id,

  status_in_reply_to_user_id as Followers_status_in_reply_to_user_id,

  status_retweet_count as Followers_status_retweet_count,

  status_favorite_count as Followers_status_favorite_count,

  status_retweeted as Followers_status_retweeted,

  status_favorited as Followers_status_favorited,

  status_possibly_sensitive as Followers_status_possibly_sensitive,

  status_hashtag_count as Followers_status_hashtag_count,

  status_hash_tags as Followers_status_hash_tags,

  status_first_hash_tag as Followers_status_first_hash_tag,

  status_url_count as Followers_status_url_count,

  status_expanded_urls as Followers_status_expanded_urls,

  status_first_expanded_url as Followers_status_first_expanded_url,

  status_user_mentions_count as Followers_status_user_mentions_count,

  status_user_mentions as Followers_status_user_mentions,

  status_first_user_mention as Followers_status_first_user_mention,

  status_media_count as Followers_status_media_count,

  status_media_expanded_urls as Followers_status_media_expanded_urls,

  status_first_media_expanded_url as Followers_status_first_media_expanded_url,

  status_symbols_count as Followers_status_symbols_count,

  status_symbols as Followers_status_symbols,

  status_first_symbol as Followers_status_first_symbol,

  status_media_photo_count as Followers_status_media_photo_count,

  status_media_photo_urls as Followers_status_media_photo_urls,

  status_first_media_photo_url as Followers_status_first_media_photo_url,

  status_place_id as Followers_status_place_id,

  status_place_url as Followers_status_place_url,

  status_place_place_type as Followers_status_place_place_type,

  status_place_name as Followers_status_place_name,

  status_place_full_name as Followers_status_place_full_name,

  status_place_country_code as Followers_status_place_country_code,

  status_place_country as Followers_status_place_country,

  status_place_bounding_box_type as Followers_status_place_bounding_box_type,

  status_place_attributes as Followers_status_place_attributes,

  status_geo_type as Followers_status_geo_type,

  status_geo_latitude as Followers_status_geo_latitude,

  status_geo_longitude as Followers_status_geo_longitude,

  status_coordinates_type as Followers_status_coordinates_type,

  status_coordinates_latitude as Followers_status_coordinates_latitude,

  status_coordinates_longitude as Followers_status_coordinates_longitude

Thanks Marco

[chrisbrain]

‌Hi Marco. Sorry, I dont really understand the question.

[marco_puccetti]

The question is the following: Is it possible to get the Followers data without having error in retrinving them?

In your first answare i've seen that you have used the Post_FollowerIds_Info table.

Thanks

Marco