Mgiyuk

How to copy a partitioned table to another partitioned table in BigQuery I'm trying to copy one partitioned table into another one. According to the docs, this should be possible. If you want to copy a partitioned table into another partitioned table, the partition specifications for the source and destination tables must match. To test, I created two partitioned tables ( partition1 & partition2 ) with the same schema. I pushed 3 records into partition $20170101 on the partition1 table: partition1 partition2 $20170101 partition1 echo '{"a":1, "b":2}' | bq insert '<removed>.partition1$20170101' echo '{"a":1, "b":2}' | bq insert '<removed>.partition1$20170101' echo '{"a":1, "b":2}' | bq insert '<removed>.partition1$20170101' That looks good: Next, I pushed 2 records into the partition2 table and to the same partition ( $20170101 ): partition2 $20170101...

Multiple Left Joins in BigQuery I'm trying to make a currently working SQL query that I have in BigQuery more streamlines and am running into the following issue: Error: ON clause must be AND of = comparisons of one field name from each table, with all field names prefixed with table name. Consider using Standard SQL .google.com/bigquery/docs/reference/standard-sql/), which allows non-equality JOINs and comparisons involving expressions and residual predicates. Below is the query that is giving the error above. The first LEFT JOIN works. When I added the second one, right below, I started getting the error. What I'm trying to do is get the human readable own.o.firstname and own.o.lastname values rather than the owner_id value of the deal record (o.properties.hubspot_owner_id.value), but in order to do so I need to join some tables. I had to use CAST on the ON clause of the second JOIN because the fields are of different types in each table's respective schema. If I don'...

Google Cloud Bigquery Library Error I am receiving this error Cannot set destination table in jobs with DDL statements Cannot set destination table in jobs with DDL statements When I try to resubmit a job from the job.build_resource() function in the google.cloud.bigquery library. job.build_resource() google.cloud.bigquery It seems that the destination table is set to something like this after that function call. 'destinationTable': {'projectId': 'xxx', 'datasetId': 'xxx', 'tableId': 'xxx'}, 'destinationTable': {'projectId': 'xxx', 'datasetId': 'xxx', 'tableId': 'xxx'}, Am I doing something wrong here? Thanks to anyone that can give me any guidance here. EDIT: The job is initially being triggered by this query = bq.query(sql_rendered) We store the job id and use it later to check the status. We get the job like this job = bq.get_job(job_id=job_id) If it meets a condition, i...