"Exceeded quota: too many free query bytes scanned for this project" in Google BigQuery

Question

I'm getting the above error since yesterday, after trying out Google BigQuery with JasperReports Server. I couldn't find any info on this, since the first 100GB should be free and the BigQuery Access Report is saying:

BigQuery Resource Usage for API Project Data Analysis Jan 31, 2013 – Feb 27, 2013

Data Processed Per Day 0 MB Peak 0.00 MB Average

I haven't reached the free query/user/day limit either: API Per-User Limit Used Billable Limit Courtesy Limit
BigQuery API 5.0 requests/second/user
0% 200,000 requests/day 10,000 requests/day

and

Traffic Reports for API Project Total requests 935 Requests/day 856 peak 33.39 average

Start Date Jan 31, 2013

Sample Period 28 days

I'm trying to use the natality sample data in a project for my university and it's pretty urgent. It was working fine for a day or so. The queries are made by the JasperSoft BigQuery Connector (http://code.google.com/p/jaspersoft-bigquery-datasource/) using an Ad-hoc-View (based on a JasperReport as Topic).

I am using the following statement:

SELECT
    STRING($P!{dimension1}) as dimension_1,
    STRING($P!{dimension2}) as dimension_2,
    SUM(record_weight)  as count_group,
    AVG(weight_pounds)  as avg_weight_pounds,
    AVG(apgar_1min) as apgar_1min,
    AVG(apgar_5min) as apgar_5min
FROM publicdata:samples.natality
WHERE
    year >= $P!{year_first}
    AND year <= $P!{year_last}
GROUP BY dimension_1, dimension_2
ORDER BY dimension_1, dimension_2

Any help would be highly appreciated! Thanks in advance!

Exothermic

What is your project ID? If you include that we can look up your project history. – Windburn

The project ID should be 793058927795. Thanks! – Exothermic

Our logs have you doing 125GB of processing in the last 30 days. If it isn't being reported correctly in the developer console that sounds like a bug. I'll investigate. – Windburn

Since it sounds like this is a university project, and your usage isn't being represented correctly, I've reset your quota. You should have the full 100GB left. If you need more, please enable billing. – Windburn

Thank you for your answer and for the quota reset!! There must be a problem with the reporting, since I just get 0 GB traffic shown. Nevertheless, I still don't know how I could have made that much processing with that only some selected fields, restricted periods of time & about 20-30 queries in total. – Exothermic

There are 137 million entries in that table -- so if you are reading 5 fields per query, that is 5 GB per query, or 20 queries until you hit the 100 GB limit. – Windburn

@JordanTigani - I have the same problem, again just querying the publicdata. Given that my reports page also shows 0 and I also do not have billing enabled, is the correct solution to this to enable billing AND to assume there is a bug in the reports page? I'm running a training course for 12 people in 2 weeks, so multiple quota resets would be required and that's probably not great for you! – Auramine

We report unbilled usage in a different bucket that doesn't show up in the developer console page. We're working on the fix for that. Your best bet is to enable billing. If you do, the usage in the developer console should be correct. – Windburn

Answer 1

Summarising the comments above and below for anyone else's benefit:

• The quota for your usage doesn't show up in the developer console if you don't enable billing.
• Looking at the reports page will show 0GB used if you don't enable billing.
• Unless you enable billing, you have a limit of 100GB to query per month.
• Enabling billing will enable you to query more than 100GB of data.
• The BigQuery team are working on a fix for the reports page not including queries on data when you haven't enabled billing

Auramine

There are some issues with the proposed answer in my opinion: not only queries on 'publicdata' will not show in the traffic reports, but all traffic, that is not billed. So all free, unbilled traffic won't be reported. The 100GB quota is a monthly, not a daily quota. – Exothermic

I'd change this slightly: the problem is that when you don't enable billing, your usage doesn't show up in the developer console. We're working on a fix for this. It isn't related to the querying of public datasaets. If you enable billing and query public datasets, your usage will show up. – Windburn

Answer 2

Even in 2021 still a relevant question. Now with PowerBi connected to BigQuery (Direct Queries). It shows an error message, in my case with code 31780: quota exceeded free query bytes scanned.

Here's the problem

• When the data is connected BQ seems to not connect to the correct project.

Here's how to solve it:

• Click the table with the BQ connection
• Click on the transform data (it's on the home tab)
• Within transform data go to the view tab
• Click on advanced editor
• You'll see this line:

    Source = GoogleBigQuery.Database(null),

• Change it to:

    Source = GoogleBigQuery.Database([BillingProject="project-id"]),

Note: not the project name, the project ID

Do this for all tables that fetch data from BQ.

source: https://learn.microsoft.com/en-gb/power-bi/connect-data/desktop-connect-bigquery#considerations-and-limitations

Maeganmaelstrom