Multiple Left Joins in BigQuery


Multiple Left Joins in BigQuery



I'm trying to make a currently working SQL query that I have in BigQuery more streamlines and am running into the following issue:



Error: ON clause must be AND of = comparisons of one field name from each table, with all field names prefixed with table name. Consider using Standard SQL .google.com/bigquery/docs/reference/standard-sql/), which allows non-equality JOINs and comparisons involving expressions and residual predicates.



Below is the query that is giving the error above. The first LEFT JOIN works. When I added the second one, right below, I started getting the error. What I'm trying to do is get the human readable own.o.firstname and own.o.lastname values rather than the owner_id value of the deal record (o.properties.hubspot_owner_id.value), but in order to do so I need to join some tables.



I had to use CAST on the ON clause of the second JOIN because the fields are of different types in each table's respective schema. If I don't do that, I get the following error: Error: Join keys o.properties.hubspot_owner_id.value (string) and o.ownerid (int64) have types that cannot be automatically coerced.



The WHERE clause is just a suppression list to not return entries that have been deleted from the database.


SELECT o.*
FROM (
SELECT
o.dealid,
o.properties.dealname.value,
stages.Label,
o.properties.closedate.value,
o.properties.hubspot_owner_id.value,
own.o.firstname,
own.o.lastname,
o.properties.amount.value,
o.properties.createdate.value,
o.properties.pipeline.value,
o.associations.associatedcompanyids,
ROW_NUMBER() OVER (PARTITION BY o.dealid ORDER BY o._sdc_batched_at DESC) as seqnum
FROM [sample-table:hubspot.deals] o
LEFT JOIN [sample-table:hubspot.sales_stages_lookup] stages ON o.properties.dealstage.value = stages.Internal_Value
LEFT JOIN [sample-table:hubspot.owners_reporting] own ON CAST(o.properties.hubspot_owner_id.value AS INTEGER) = CAST(own.o.ownerid AS INTEGER)) o
WHERE o.dealid NOT IN (SELECT objectid FROM [sample-table:hubspot_suppression_list.data] WHERE subscriptiontype = 'deal.deletion') AND seqnum = 1




1 Answer
1



Use standard SQL in BigQuery instead, which supports expressions as part of the ON clause:


ON


#standardSQL
SELECT o.*
FROM (
SELECT
o.dealid,
o.properties.dealname.value AS dealname_value,
stages.Label,
o.properties.closedate.value AS closedate_value,
o.properties.hubspot_owner_id.value AS hubspot_owner_id_value,
own.o.firstname,
own.o.lastname,
o.properties.amount.value AS amount_value,
o.properties.createdate.value AS createdate_value,
o.properties.pipeline.value AS pipeline_value,
o.associations.associatedcompanyids,
ROW_NUMBER() OVER (PARTITION BY o.dealid ORDER BY o._sdc_batched_at DESC) as seqnum
FROM `sample-table.hubspot.deals` o
LEFT JOIN `sample-table.hubspot.sales_stages_lookup` stages ON o.properties.dealstage.value = stages.Internal_Value
LEFT JOIN `sample-table.hubspot.owners_reporting` own ON CAST(o.properties.hubspot_owner_id.value AS INT64) = CAST(own.o.ownerid AS INT64)) o
WHERE o.dealid NOT IN (SELECT objectid FROM `sample-table.hubspot_suppression_list.data` WHERE subscriptiontype = 'deal.deletion') AND seqnum = 1



For more on the differences between legacy and standard SQL in BigQuery, see the migration guide.





thank you for the answer. Is the expression what the problem is in the query?
– Felipe Da Cruz
2 days ago





Yes. Using standard SQL is recommended instead since it doesn't have this or other limitations.
– Elliott Brossard
2 days ago





I just ran the query using standard SQL and got the following error... Duplicate column names in the result are not supported. Found duplicate(s): value. I've tried a bunch of modifications to the query, but I still can't figure it out.
– Felipe Da Cruz
2 days ago



Duplicate column names in the result are not supported. Found duplicate(s): value





See if this edit helps. The problem is that the query produces multiple columns in the result named value, but you can give them different names to resolve the error.
– Elliott Brossard
2 days ago


value





That did it! I also realized that the inner query was returning a lot of columns named value and ended up using aliases for each of them via AS. It fixed it and ended up making the column names more meaningful anyway. Thank you for all of the help - it is sincerely appreciated!
– Felipe Da Cruz
2 days ago


value


AS






By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Comments

Post a Comment

Popular posts from this blog

paramiko-expect timeout is happening after executing the command

paramiko-expect timeout is happening after executing the command Can someone help me why timeout is happening after executing the command .I am trying to SSH to a machine and execute a command.I want to store the result of the executed command . I have referred paramiko-timeout but i didn't get my expected result. import traceback try: import paramiko except ImportError: raise Exception("Please install paramiko , pip install paramiko") import pexpect from paramiko_expect import SSHClientInteraction def main(): HOSTNAME = "minion5.net" USERNAME = "root" PASSWORD = "help@432" PROMPT = "root@*:~$s+" try: client = paramiko.SSHClient() client.load_system_host_keys() client.set_missing_host_key_policy(paramiko.AutoAddPolicy()) client.connect(hostname=HOSTNAME, username=USERNAME, password=PASSWORD) with SSHClientInteraction(client, timeout=20, display=True) as interact: ...
Read more

Opening a url is failing in Swift

Opening a url is failing in Swift breakpointing inside the callback and logging success shows false. I added itms-apps to my plist: LSApplicationQueriesSchemes and put this in my app delegate: func application(_ app: UIApplication, open url: URL, options: [UIApplicationOpenURLOptionsKey : Any] = [:]) -> Bool { return true } func moreTapped() { let url = NSURL(string: "itms-apps://itunes.com/developer/quantum-productions/id979315877") if #available(iOS 10.0, *) { UIApplication.shared.open(url! as URL, options: [:], completionHandler: {(success: Bool) in }) } else { UIApplication.shared.openURL(url! as URL) } } Are you testing on a real device? – rmaddy Jun 21 at 23:57 I try on a device, get "Cannot Connect to App Store". Device is online ...
Read more

Export result set on Dbeaver to CSV

Export result set on Dbeaver to CSV Normally I use Dbeaver for windows and always export my result set like this: run my query --> select the result --> export the result set --> select export to clipboard --> done This step by step puts my result set in my clipboard and I can paste it wherever I want to work with it. The problem is that now I am using dbeaver for mac and this guide is not working. I can go on until the moment that I select my result set like in the image below: exporting data set But once I go further in the process, in the last step I get: no query Note that in "source" it was suppose to show the query that originated the result set, but instead it says just "select. As a result it does't select my result or anything (besides being "successful"). Normally my query would show up there automatically and I couldn't find any option that corrects this problem in the menus. Please help me! Thanks. Aft...
Read more