Databricks–Using Databricks Secrets

Here are all the steps needed to setup a secret in databricks (not key-vault) in Databricks. This works with the standard version of DB:

Install Databricks CLI:
1. install databricks cli (needs python)
     pip install databricks-cli
2. Setup databricks token (needs token from user-settings in Databricks. Also needs the host url: eg Host: https://northcentralus.azuredatabricks.net)
     databricks configure --token
3. Create a scope
     databricks secrets create-scope --scope dbSecretsScope --initial-manage-principal "users"
4. Add a key-value to scope
     databricks secrets put --scope dbSecretsScope --key mydbPassword --string-value myPasswordValue
5. List keys in scope
     databricks secrets list --scope dbSecretsScope

Create a Python notebook and add the following code in a cell and run it

sqlserver = xxxxx.database.windows.net'
port = '1433'
database = 'myFirstDb'
user = 'iamspecial'
pswd = dbutils.secrets.get("dbSecretsScope","mydbPassword")
print "password is", pswd #will display redacted – very important!
table = 'myFirstTable'

## Load Data Frame ##
df1 = spark.read \
   .option('user', user) \
   .option('password', pswd) \
   .jdbc('jdbc:sqlserver://' + sqlserver + ':' + port + ';database=' + database, table)

df1 #to display the dataframe to make sure we connected