8/23/2020 Google Ad Manager transfers | BigQuery Data Transfer Service
Google Ad Manager transfers
The BigQuery Data Transfer Service for Google Ad Manager allows you to automatically schedule and manage recurring load jobs for Google Ad Manager (formerly known as DoubleClick for Publishers) reporting data.
Suppo ed repo s
The BigQuery Data Transfer Service for Google Ad Manager currently supports the following reporting options:
Data Transfer (Google Ad Manager DT) les (https://support.google.com/admanager/answer/1733124)
Data Transfer elds (https://support.google.com/admanager/table/7401123)
Match Tables as provided via PQL (https://developers.google.com/doubleclick-publishers/docs/pqlreference#matchtables)
Match Tables from CompanyService (v201908) (https://developers.google.com/doubleclick-publishers/docs/reference/v201908/CompanyService)
Match Tables from OrderService (v201908) (https://developers.google.com/doubleclick-publishers/docs/reference/v201908/OrderService)
Match Tables from PlacementService (v201908) (https://developers.google.com/doubleclick-publishers/docs/reference/v201908/PlacementService)
For information on how Google Ad Manager reports are transformed into BigQuery tables and views, see Google Ad Manager report transformations (/bigquery-transfer/docs/doubleclick-publisher-transformation).
The BigQuery Data Transfer Service supports the following delimiters for Google Ad Manager DT les: Tab ( \t ) aret ( ^ ), and Comma ( , ). The Thorn (þ) delimiter is NOT currently supported.
Reporting option Support
Supported API v201908
https://cloud.google.com/bigquery-transfer/docs/doubleclick-publisher-transfer/ 1/10 8/23/2020 Google Ad Manager transfers | BigQuery Data Transfer Service
version (https://developers.google.com/doubleclick-publishers/docs/rel_notes#v201908)
Schedule Every 8 hours, based on the creation time.
Not con gurable
Refresh window Last 2 days
Not con gurable
Maximum back ll Last 60 days duration Google Ad Manager retains Data Transfer les for up to 60 days. Files older than 60 days are deleted by Google Ad Manager.
Before you begin
Before you create a Google Ad Manager transfer:
Verify that you have completed all actions required to enable the BigQuery Data Transfer Service (/bigquery-transfer/docs/enable-transfer-service).
Create a BigQuery dataset (/bigquery/docs/datasets) to store the Google Ad Manager data.
Ensure that your organization has access to Google Ad Manager Data Transfer (Google Ad Manager DT) les. These les are delivered by the Google Ad Manager team to a Cloud Storage bucket. To gain access to Google Ad Manager DT les, complete the following steps. Additional charges from the Google Ad Manager team may apply.
Review the Overview of Data Transfer reports (https://support.google.com/admanager/answer/1733124).
In the upper right corner of the page, click Contact Us.
Click Reports.
Email or chat with a representative and ask about access to Google Ad Manager Data Transfer les.
After completing this step, you will receive a Cloud Storage bucket similar to the following:
https://cloud.google.com/bigquery-transfer/docs/doubleclick-publisher-transfer/ 2/10 8/23/2020 Google Ad Manager transfers | BigQuery Data Transfer Service
gdfp-12345678
The Google Cloud team does NOT have the ability to generate or grant access to Google Ad Manager DT les on your behalf. Contact, Google Ad Manager support (https://support.google.com/admanager/answer/3059042?hl=en&ref_topic=7519191), for access to Google Ad Manager DT les.
Enable API access (https://support.google.com/admanager/answer/3088588?hl=en) to your Google Ad Manager network.
If you intend to create transfers using the classic BigQuery web UI, allow pop-ups from bigquery.cloud.google.com so you can view the permissions window. You must allow the BigQuery Data Transfer Service permission to manage your Google Ad Manager data for the transfer to succeed.
If you intend to setup transfer noti cations, you must have pubsub.topics.setIamPolicy permissions for Pub/Sub. Pub/Sub permissions are not required if you just set up email noti cations. For more information, see BigQuery Data Transfer Service run noti cations (/bigquery-transfer/docs/transfer-run-noti cations).
Required permissions
BigQuery: Ensure that the person creating the transfer has the following permissions in BigQuery:
bigquery.transfers.update permissions to create the transfer
bigquery.datasets.update permissions on the target dataset
The bigquery.admin prede ned IAM role includes bigquery.transfers.update and bigquery.datasets.update permissions. For more information on IAM roles in BigQuery Data Transfer Service, see Access control reference (/bigquery-transfer/docs/reference/access-control).
Google Ad Manager: Read access to the Google Ad Manager DT les stored in Cloud Storage. Permissions for Google Ad Manager DT les are managed by the Google Ad Manager team.
https://cloud.google.com/bigquery-transfer/docs/doubleclick-publisher-transfer/ 3/10 8/23/2020 Google Ad Manager transfers | BigQuery Data Transfer Service
Se ing up a Google Ad Manager transfer
Setting up a BigQuery data transfer for Google Ad Manager requires a:
Cloud Storage bucket: The Cloud Storage bucket URI for your Google Ad Manager DT les as described in Before you begin (/bigquery-transfer/docs/doubleclick-publisher-transfer#before_you_begin). The bucket name should look like the following:
gdfp-12345678
Network Code: You'll nd the Google Ad Manager network code in the URL when you are logged into your network. For example, in the URL https://admanager.google.com/2032576#delivery, 2032576 is your network code. For more information, see Get started with Google Ad Manager (https://developers.google.com/doubleclick-publishers/docs/start).
To create a BigQuery Data Transfer Service transfer for Google Ad Manager:
ConsoleClassic UI (#classic-ui)bq (#bq)API (#api)
1. Go to the BigQuery web UI.
Go to the BigQuery web UI (https://console.cloud.google.com/bigquery)
2. Click Transfers.
3. Click Create Transfer.
4. On the Create Transfer page:
In the Source type section, for Source, choose Google Ad Manager.
In the Transfer con g name section, for Display name, enter a name for the transfer such as My Transfer. The transfer name can be any value that allows you to easily
https://cloud.google.com/bigquery-transfer/docs/doubleclick-publisher-transfer/ 4/10 8/23/2020 Google Ad Manager transfers | BigQuery Data Transfer Service
identify the transfer if you need to modify it later.
In the Schedule options section, for Schedule, leave the default value (Start now) or click Start at a set time.
For Repeats, choose an option for how often to run the transfer.
Daily (default)
Weekly
Monthly
Custom
On-demand
For Start date and run time, enter the date and time to start the transfer. If you choose Start now, this option is disabled.
In the Destination settings section, for Destination dataset, choose the dataset you created to store your data.
https://cloud.google.com/bigquery-transfer/docs/doubleclick-publisher-transfer/ 5/10 8/23/2020 Google Ad Manager transfers | BigQuery Data Transfer Service
In the Data source details section:
For Cloud Storage bucket, enter the name of the Cloud Storage bucket that stores your Data Transfer les. When you enter the bucket name, do not include gs://.
For Network code, enter your network code.
(Optional) In the Noti cation options section:
Click the toggle to enable email noti cations. When you enable this option, the transfer administrator receives an email noti cation when a transfer run fails.
For Select a Pub/Sub topic, choose your topic (/pubsub/docs/overview#types) name or click Create a topic. This option con gures Pub/Sub run noti cations (/bigquery-transfer/docs/transfer-run-noti cations) for your transfer.
5. Click Save.
ng: If you change the schema of a report, all les on that day must have the same schema, or the transfer for t day will fail.
Troubleshooting Google Ad Manager transfer setup
If you are having issues setting up your transfer, see Google Ad Manager transfer issues (/bigquery-transfer/docs/transfer-troubleshooting#google_ad_manager_transfer_issues) in
https://cloud.google.com/bigquery-transfer/docs/doubleclick-publisher-transfer/ 6/10 8/23/2020 Google Ad Manager transfers | BigQuery Data Transfer Service
Troubleshooting BigQuery Data Transfer Service transfer setup (/bigquery-transfer/docs/transfer-troubleshooting).
Querying your data
When your data is transferred to BigQuery, the data is written to ingestion-time partitioned tables. For more information, see Introduction to partitioned tables (/bigquery/docs/partitioned-tables).
If you query your tables directly instead of using the auto-generated views, you must use the _PARTITIONTIME pseudo-column in your query. For more information, see Querying partitioned tables (/bigquery/docs/querying-partitioned-tables).
Google Ad Manager sample queries
You can use the following Google Ad Manager sample queries to analyze your transferred data. You can also use the queries in a visualization tool such as Google Data Studio (https://www.google.com/analytics/data-studio/). These queries are provided to help you get started on querying your Google Ad Manager data with BigQuery. For additional questions on what you can do with these reports, contact your Google Ad Manager technical representative.
If you query your tables directly instead of using the auto-generated views, you must use the _PARTITIONTIM o-column in your query. For more information, see Querying partitioned tables uery/docs/querying-partitioned-tables).
In each of the following queries, replace variables like dataset with your values. For example, replace network_code with your Google Ad Manager network code.
Impressions and unique users by city
The following sample query analyzes the number of impressions and unique users by city over the past 30 days.
Consolebq (#bq)
https://cloud.google.com/bigquery-transfer/docs/doubleclick-publisher-transfer/ 7/10 8/23/2020 Google Ad Manager transfers | BigQuery Data Transfer Service
# START_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY) # END_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY) SELECT City, _DATA_DATE AS Date, count(*) AS imps, count(distinct UserId) AS uniq_users FROM `dataset.NetworkImpressionsnetwork_code` WHERE _DATA_DATE BETWEEN start_date AND end_date GROUP BY City, Date
Impressions and unique users by line item type
The following sample query analyzes the number of impressions and unique users by line item type over the past 30 days.
Consolebq (#bq)
# START_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY) # END_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY) SELECT MT.LineItemType AS LineItemType, DT._DATA_DATE AS Date, count(*) AS imps, count(distinct UserId) AS uniq_users FROM `dataset.NetworkImpressionsnetwork_code` AS DT LEFT JOIN `dataset.MatchTableLineItem_network_code` AS MT ON DT.LineItemId = MT.Id WHERE DT._DATA_DATE BETWEEN start_date AND end_date GROUP BY LineItemType, Date ORDER BY Date desc, imps desc
Impressions by ad unit
https://cloud.google.com/bigquery-transfer/docs/doubleclick-publisher-transfer/ 8/10 8/23/2020 Google Ad Manager transfers | BigQuery Data Transfer Service
The following sample query analyzes the number of impressions by ad unit over the past 30 days.
Consolebq (#bq)
# START_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY) # END_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY) SELECT MT.AdUnitCode AS AdUnitCode, DT.DATA_DATE AS Date, count(*) AS imps FROM `dataset.NetworkImpressionsnetwork_code` AS DT LEFT JOIN `dataset.MatchTableLineItem_network_code` AS MT ON DT.AdUnitId = MT.Id WHERE DT._DATA_DATE BETWEEN start_date AND end_date GROUP BY AdUnitCode, Date ORDER BY Date desc, imps desc
Impressions by line item
The following sample query analyzes the number of impressions by line item over the past 30 days.
Consolebq (#bq)
# START_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -31 DAY) # END_DATE = DATE_ADD(CURRENT_DATE(), INTERVAL -1 DAY) SELECT MT.Name AS LineItemName, DT._DATA_DATE AS Date, count(*) AS imps FROM `dataset.NetworkImpressionsnetwork_code` AS DT LEFT JOIN `dataset.MatchTableLineItem_network_code` AS MT ON DT.LineItemId = MT.Id WHERE DT._DATA_DATE BETWEEN start_date AND end_date
https://cloud.google.com/bigquery-transfer/docs/doubleclick-publisher-transfer/ 9/10 8/23/2020 Google Ad Manager transfers | BigQuery Data Transfer Service
GROUP BY LineItemName, Date ORDER BY Date desc, imps desc
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License (https://creativecommons.org/licenses/by/4.0/), and code samples are licensed under the Apache 2.0 License (https://www.apache.org/licenses/LICENSE-2.0). For details, see the Google Developers Site Policies (https://developers.google.com/site-policies). Java is a registered trademark of Oracle and/or its a liates.
Last updated 2020-07-21 UTC.
https://cloud.google.com/bigquery-transfer/docs/doubleclick-publisher-transfer/ 10/10