0

I am working on Tutorial 4 of this doc from Azure Team where this section in a Dedicated SQL pool (that actually is also a database - as item 2 of Tutorial states) creates a database named nyctaxi with a table nyctaxi.trip as follows:

%%pyspark
spark.sql("CREATE DATABASE IF NOT EXISTS nyctaxi")
df.write.mode("overwrite").saveAsTable("nyctaxi.trip")

Then Tutorial 5 creates another table NYCTaxiTripSmall. After completing these tutorials, I can see (on the Data Hub of the Synapse studio) my Dedicated SQL Pool as shown below. But when I click on any database object folder it does not show any db object (tables, external tables etc.) and instead, it shows a red cross sign (as shown below).

Question: Why I am not seeing the db objects (described above) in the Dedicated SQL pool below.

Remarks: Please note that I also created DataExplorationDB db and an external source in tutorial 2 using serverless SQL pool - and, as shown in yellow below, I can see that db and its objects. So why same is not true for Dedicated SQL Pool db? I have also restarted the Dedicated SQL pool and it's online. But still no db objects are showing.

enter image description here

1 Answer 1

1

When you create a Spark database the tables aren't automatically added to your Dedicated SQL Pool. You can add them as External Tables if you want, but there's no automatic metadata sync between Spark and Dedicated SQL Pool.

Synapse does create a serverless "Lake Database" corresponding to your Spark database, which you can use from SQL Scripts or access with SQL Server reporting tools.

Sign up to request clarification or add additional context in comments.

2 Comments

Thank you for pointing it out - as I do see those tables inside Lake Database section that is created above SQL Database section in the Data Hub of Synapse studio. But, in what cases, one would be able to see the db tables (for example) inside Tables folder of DedicatedSQLPool folder (shown in yellow in image above?
Only when you create a table using TSQL in a SQL Script or from SQL Server Management Studio.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.