The Google BigQuery connector for rudol allows you to connect your Google BigQuery data warehouse instances.
As Google BigQuery credits are charged per use we recommend you to choose a Scanning Frequency that matches your team's workflow to minimize your warehouse costs
|Service Account Keyfile||JSON Service Account key file content, see Generate Service Account keyfile|
|Dataset location||Dataset region, single and multiregion locations are supported (i.e. us-central-1 or US)|
Generate Service Account keyfile
The following steps provide examples for how to allow rudol reads to Cloud Storage buckets in
your-storage-project for a Cloud Composer environment deployed in the project ID
Create a service account in
- In the Google Cloud console, go to IAM & Admin -> Service Accounts page.
- Select your project.
- Click + CREATE SERVICE ACCOUNT.
- In the Service account name field, enter a name. The Google Cloud console fills in the Service account ID field based on this name.
- Click CREATE AND CONTINUE.
- Click Select a role field and add the required roles.
Minimum perssions for metadata scan require at least BigQuery User and BigQuery Metadata Viewer roles.
For Data Quality Validations the service account should also have the BigQuery Data Viewer role.
- Click Done to finish creating the service account.
- Do not close your browser window. You will use it in the next step.
Download a JSON key for the service account you just created
- In the Google Cloud console, click the email address for the service account that you created.
- Click KEYS.
- Click ADD KEY, then click Create new key.
- Click CREATE. A JSON key file will be downloaded to your computer.
Your JSON Service Account key file should look like this:
"private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\\n",
If you need more details on the previous steps you can check BigQuery Doc here