8/23/2020 Using the bq command-line tool | BigQuery | Cloud

Using the bq command-line tool

The bq command-line tool is a Python-based command-line tool for BigQuery. This page contains general information on using the bq command-line tool.

For a complete reference of all bq commands and ags, see bq command-line tool reference (//docs/reference/bq-cli-reference).

Before you begin

Before you can use the bq command-line tool, you must use the Google Cloud Console to create or select a project and install the Cloud SDK.

1. Sign in (https://accounts.google.com/Login) to your .

If you don't already have one, sign up for a new account (https://accounts.google.com/SignUp).

2. In the Cloud Console, on the project selector page, select or create a Cloud project.

 Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you nish these steps, you can delete the project, removing all resources associated with the project.

Go to the project selector page (https://console.cloud.google.com/projectselector2/home/dashboar

3. Install and initialize the Cloud SDK (/sdk/docs).

4. BigQuery is automatically enabled in new projects. To activate BigQuery in a preexisting project, go to Enable the BigQuery API.

Enable the API (https://console.cloud.google.com/ows/enableapi?apiid=bigquery)

5. BigQuery provides a sandbox (/bigquery/docs/sandbox) if you do not want to provide a credit card or enable billing for your project. The steps in this topic work for a project whether or not your project has billing enabled. If you optionally want to enable billing, see Learn how to enable billing (/billing/docs/how-to/modify-project).

https://cloud.google.com/bigquery/docs/bq-command-line-tool 1/10 8/23/2020 Using the bq command-line tool | BigQuery | Google Cloud

Instead of downloading and installing the Cloud SDK, you can use a pre-installed version of the Cloud SDK in Google Cloud Shell (/shell/docs).

General usage

Flag positioning

bq supports two kinds of ags — global ags and command ags. They should be used in the order shown here:

global_flag argument bq_command --command-specific_flag argument

Global ags (or common ags) can be used in all commands.

Command-specic ags apply to a specic command.

Separate multiple global or command-specic ags using a space. For example:

bal_flag argument \ bal_flag argument \ mmand \ mand-specific_flag argument \ mand-specific_flag argument

You can specify command arguments in one of the following ways:

--flag=argument

--flag='argument'

--flag="argument"

--flag argument

--flag 'argument'

--flag "argument"

https://cloud.google.com/bigquery/docs/bq-command-line-tool 2/10 8/23/2020 Using the bq command-line tool | BigQuery | Google Cloud

Each of these methods is used throughout the BigQuery documentation.

Some commands require the use of single or double quotes around arguments. This is often true when the argument contains spaces, commas, or other special characters. For example:

ery --nouse_legacy_sql \ CT UNT(*)

igquery-public-data`.samples.shakespeare'

Flags with boolean values can be specied without an argument. If you specify true or false you must specify =argument.

For example, this command species false for the boolean ag --use_legacy_sql by placing no at the front of the ag:

ery --nouse_legacy_sql \ CT UNT(*)

igquery-public-data`.samples.shakespeare'

Alternatively, to specify false as the ag's argument, you would enter:

ery --use_legacy_sql=false \ CT UNT(*)

igquery-public-data`.samples.shakespeare'

For a list of available global and command-specic ags, see bq command-line tool reference (/bigquery/docs/reference/bq-cli-reference).

Geing help

https://cloud.google.com/bigquery/docs/bq-command-line-tool 3/10 8/23/2020 Using the bq command-line tool | BigQuery | Google Cloud

You can enter the following commands to get help for the bq command-line tool:

For the installed version of bq, enter bq version.

For a full list of commands, enter bq help.

For a list of global ags, enter bq --help.

For help on a specic command, enter bq help command.

For help on a specic command plus a list of global ags, enter bq command --help.

Debugging

You can enter the following commands to debug bq:

To see requests sent and received

Add the --apilog=path_to_file ag to save a log of operations to a local le. bq works by

making standard REST-based API calls, which can be useful to see. It is also useful to attach this

log when reporting issues. Using - or stdout instead of a le path will print the log to the console. Setting --apilog to stderr outputs to the standard error le.

To help troubleshoot errors

Enter the --format=prettyjson ag when getting a job's status

(/bigquery/docs/managing-jobs#view-job) or when viewing detailed information about

resources such as and datasets. Using this ag outputs the response in JSON format, including the reason property. You can use the reason property to look up troubleshooting

steps (/bigquery/troubleshooting-errors).

Seing default values for command-line ags

You can set default values for command-line ags by including them in the bq command-line tool's conguration le — .bigqueryrc. Before you congure your default options, you must rst create a .bigqueryrc le. You can use your preferred text editor to create the le. After you create the .bigqueryrc le, you can specify the path to the le using the --bigqueryrc global ag.

https://cloud.google.com/bigquery/docs/bq-command-line-tool 4/10 8/23/2020 Using the bq command-line tool | BigQuery | Google Cloud

If the --bigqueryrc ag is not specied, the BIGQUERYRC environment variable is used. If that is not specied, the path ~/.bigqueryrc is used. The default path is $HOME/.bigqueryrc.

Creating a .bigqueryrc le using the bq init command is not recommended.

Adding ags to .bigqueryrc

To add default values for command-line ags to .bigqueryrc:

Place global ags (/bigquery/docs/reference/bq-cli-reference#global_ags) at the top of the le without a header

For command-specic ags (/bigquery/docs/reference/bq-cli-reference#command-specic_ags) , enter the command name (in brackets) and add the command-specic ags (one per line) below it in the following format:

nd mand-specific_flag=argument mand-specific_flag=argument

When you enter command-line ags in .bigqueryrc, you must specify the ag's argument in the following format =argument.

.bigqueryrc is read every time you run bq so changes should be updated immediately. When you run bq in interactive mode (#interactive) (bq shell), you must restart the shell for the changes to take effect.

Example

This example sets default values for the following global ags:

--apilog is set to stdout to print debugging output to the console.

--format is set to prettyjson to display command output in a human-readable JSON format.

--location is set to the US multi-region location.

https://cloud.google.com/bigquery/docs/bq-command-line-tool 5/10 8/23/2020 Using the bq command-line tool | BigQuery | Google Cloud

This example sets default values for the following query command-specic ags:

--use_legacy_sql is set to false to make standard SQL the default query syntax.

 Note: You cannot use --nouse_legacy_sql in .bigqueryrc.

--max_rows is set to 100 to control the number of rows in the query output.

--maximum_bytes_billed is set to 10,000,000 bytes (10 MB) to fail queries that read more than 10 MB of data.

This example sets a default value for the following load command-specic ag:

--destination_kms_key (/bigquery/docs/customer-managed-encryption) is set to projects/myproject/locations/mylocation/keyRings/myRing/cryptoKeys/myKey.

ntial_file = path_to_credential_file log=stdout mat=prettyjson ation=US

y] _legacy_sql=false _rows=100 imum_bytes_billed=10000000

] tination_kms_key=projects/myproject/locations/mylocation/keyRings/myRing/cryptoKeys/

To conrm your settings, enter the following command:

~/.bigqueryrc

Running bq in an interactive shell

You can run bq in an interactive shell where you don't need to prex the commands with bq. To start interactive mode, enter bq shell. After launching the shell, the prompt changes to the ID of

https://cloud.google.com/bigquery/docs/bq-command-line-tool 6/10 8/23/2020 Using the bq command-line tool | BigQuery | Google Cloud

your default project. To exit interactive mode, enter exit.

Examples

You can nd command-line examples throughout the How-to guides (/bigquery/docs/how-to) section of the BigQuery documentation. Below are links to common command-line tasks such as creating, getting, listing, deleting, and modifying BigQuery resources.

Creating resources

For information on using the bq command-line tool to create resources, see:

Creating a dataset (/bigquery/docs/datasets#bigquery-create-dataset-cli)

Creating an empty table with a schema denition (/bigquery/docs/tables#bigquery-create-table-cli)

Creating a table from a query result (/bigquery/docs/tables#creating_a_table_from_a_query_result)

Creating an ingestion-time partitioned table (/bigquery/docs/creating-partitioned-tables#creating_an_empty_ingestion- time_partitioned_table_with_a_schema_denition)

Creating a view (/bigquery/docs/views#creating_a_view)

For examples of creating a table using a data le, see Loading data (#loading_data).

Geing information about resources

For information on using the bq command-line tool to get information about resources, see:

Getting information about datasets (/bigquery/docs/dataset-metadata#getting_dataset_information)

Getting information about tables (/bigquery/docs/tables#getting_information_about_tables)

Getting information about views (/bigquery/docs/view-metadata#getting_view_information)

Listing resources

https://cloud.google.com/bigquery/docs/bq-command-line-tool 7/10 8/23/2020 Using the bq command-line tool | BigQuery | Google Cloud

For information on using the bq command-line tool to list resources, see:

Listing datasets (/bigquery/docs/listing-datasets)

Listing tables (/bigquery/docs/tables#listing_tables_in_a_dataset)

Listing views (/bigquery/docs/listing-datasets#listing_datasets_in_a_project)

Updating resources

For information on using the bq command-line tool to update resources, see:

Updating dataset properties (/bigquery/docs/updating-datasets)

Updating table properties (/bigquery/docs/managing-tables#updating_table_properties)

Updating view properties (/bigquery/docs/updating-views)

Loading data

For information on using the bq command-line tool to load data, see:

Loading Avro data from Cloud Storage (/bigquery/docs/loading-data-cloud-storage-avro#bigquery-import-gcs-le-cli)

Loading JSON data from Cloud Storage (/bigquery/docs/loading-data-cloud-storage-json#loading_json_data_into_a_new_table)

Loading CSV data from Cloud Storage (/bigquery/docs/loading-data-cloud-storage-csv#loading_csv_data_into_a_new_table)

Loading data from a local le (/bigquery/docs/loading-data-local#loading_data_from_a_local_data_source)

Querying data

For information on using the bq command-line tool to query data, see:

Running queries (/bigquery/docs/running-queries#bigquery-query-cli)

Writing query results (/bigquery/docs/writing-results#writing_query_results)

https://cloud.google.com/bigquery/docs/bq-command-line-tool 8/10 8/23/2020 Using the bq command-line tool | BigQuery | Google Cloud

Running parameterized queries (/bigquery/docs/parameterized-queries#running_parameterized_queries)

Using external data sources

For information on using the bq command-line tool to query data in external data sources, see:

Creating a table denition using a JSON schema le (/bigquery/external-table-denition#creating_a_table_denition_using_a_json_schema_le)

Querying Cloud data using permanent external tables (/bigquery/external-data-bigtable#permanent-tables)

Querying Cloud Storage data using permanent external tables (/bigquery/external-data-cloud-storage#permanent-tables)

Querying data using permanent external tables (/bigquery/external-data-drive#permanent-tables)

Expoing data

For information on using the bq command-line tool to export data, see:

Exporting data stored in BigQuery (/bigquery/docs/exporting-data#exporting_data_stored_in_bigquery)

Using the BigQuery Data Transfer Service

For information on using the bq command-line tool with the BigQuery Data Transfer Service, see:

Setting up a Campaign Manager transfer (/bigquery-transfer/docs/-campaign-transfer#setting_up_a_campaign_manager_transfer)

Setting up a Cloud Storage transfer (/bigquery-transfer/docs/cloud-storage-transfer#setting_up_a_cloud_storage_transfer)

Setting up a transfer (/bigquery-transfer/docs/doubleclick-publisher-transfer#setting_up_a_google_ad_manager_transfer)

https://cloud.google.com/bigquery/docs/bq-command-line-tool 9/10 8/23/2020 Using the bq command-line tool | BigQuery | Google Cloud

Setting up a Google Ads transfer (/bigquery-transfer/docs/adwords-transfer#setting_up_a_google_ads_data_transfer)

Setting up a Google Merchant Center transfer (/bigquery-transfer/docs/merchant-center-transfer#setting_up_a_google_merchant_center_transfer) (beta)

Setting up a transfer (/bigquery-transfer/docs/play-transfer#setting_up_a_google_play_transfer)

Setting up a Search Ads 360 transfer (/bigquery-transfer/docs/sa360-transfer#setting_up_a_search_ads_360_transfer) (beta)

Setting up a YouTube Channel transfer (/bigquery-transfer/docs/-channel-transfer#setting_up_a_youtube_channel_transfer)

Setting up a YouTube Content Owner transfer (/bigquery-transfer/docs/youtube-content-owner- transfer#setting_up_a_youtube_content_owner_transfer)

Getting information about transfer congurations (/bigquery/docs/working-with-transfers#getting_information_about_a_transfer_conguration)

Listing transfer congurations (/bigquery/docs/working-with-transfers#listing_transfer_congurations)

Viewing the run history (/bigquery/docs/working-with-transfers#viewing_the_run_history)

Viewing transfer run details and log (/bigquery/docs/working-with-transfers#viewing_transfer_run_details_and_log_messages)

Updating transfer congurations (/bigquery/docs/working-with-transfers#updating_a_transfer)

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License (https://creativecommons.org/licenses/by/4.0/), and code samples are licensed under the Apache 2.0 License (https://www.apache.org/licenses/LICENSE-2.0). For details, see the Site Policies (https://developers.google.com/site-policies). Java is a registered trademark of Oracle and/or its aliates.

Last updated 2020-07-06 UTC.

https://cloud.google.com/bigquery/docs/bq-command-line-tool 10/10