AWS s3 - GBQ Integration:

AWS s3 - GBQ Integration: 

Objective - Move data to GBQ, Identify the Tables within Google Big Query, Configure Permission(s) to update table lines/rows

1. Log into Source Interface (Example is AWS s3 bucket)

2. In GBQ Log into GBQ (Google Biq Query) Instance: Choose Data Transfers

3. Create a Transfer

4. Choose Source System 

5. Name your Transfer (*******************)

6. Set Schedule Options

7. Note under scheduling - If “Auto Schedule” is disable X’fers will be started ON-Demand VIA “API” etc

8. Enter your DATASET ID or Create a NEW Dataset | This requires some user technical acumen knowing the Dataset ID, or IAM in understanding the responsibility of securing Usernames/PW’s/& ID’s.

9. If creating a NEW DATASET Choose Availability Region, Expiration, & Encryption as desired.

10. Click Create Dataset

11. NAME Destination TABLE

12. Enter the source URI (Uniform Resource Identifier)

13. Enter Access Key for source as Required (Again, Personal Technical Acumen is MANDATORY!) Use Active Access Key ID’s / Secret Access Key’s Or Create New & store in secure repository for Org. Admins

14. Choose File Format

15. Leave Defaults

16. Click Save

17. RUN ????????‍♂️xfer

18. Click “Ok”

19. Check Status & Await Completion

20. Check Table Names

21. Assure Resources that are intended to be integrated EXIST & The Access (ACL) includes entities request access to data |Analyse data formats to match destination as required|.

22. Check Properties within YOUR TABLE SCHEMA to Identify Field name, Type, Mode, Description, etc .

Will UPDATE as info/tech changes...