Reading Row values in DataBricks Notebook

Question

I have a Databricks python notebook reading ADL configurations from a JSON file on DBFS. I am able to read the file fine and the DF generated shows the values with the below schema.

testJsonData:pyspark.sql.dataframe.DataFrame

clientId:string
oauth2Credential:string
oauth2RefreshUrl:string
providerType:string

I am having challenges in reading a specific row value. I am getting a row object but doesn't give me the value of the row.

testJsonData.select('clientId').collect()
Out[65]: [Row(clientId='6xxxx08vvvvvvvvv7f')]

Appreciate any pointers helping with this.

I found the solution in another article - stackoverflow.com/questions/38610559/… — Satya Azure
– Satya Azure, Commented Mar 3, 2019 at 22:18

Satya Azure · Accepted Answer · 2019-03-03 22:47:36Z

0

Hope this helps someone having a similar issue, I read the config values like this

   configlist = testJsonData.select("*").collect()
   dataProviderType = configlist[0].providerType
   dataClientId = configlist[0].clientId
   dataOAuthCredential = configlist[0].oauth2Credential
   dataOAuthUrl = configlist[0].outh2RefreshUrl

edited Mar 3, 2019 at 22:47

answered Mar 3, 2019 at 22:22

Satya Azure

4891 gold badge11 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Reading Row values in DataBricks Notebook

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related