
<p>This is pretty exciting! We have had the privilege of working with the fantastic ResourceSmart Schools team at&nbsp;<a href="https://www.linkedin.com/company/sustainability-victoria/">Sustainability Victoria</a>&nbsp;on a new platform for schools to manage their efforts toward becoming more sustainable. We have just found out the CRN has recognised us as a finalist for Platform Innovator of the year. The award recognises our innovation with our Digital Experience Runtime toolkit, which has let us build an amazing React / next.js based user experience for schools on top of a Microsoft Dynamics CRM.</p>



<p>ResourceSmart Schools (RSS) is a leading, internationally-recognised sustainability and climate change education program.</p>



<p>A BIG thank-you to the team at ResourceSmart Schools for giving us the opportunity to work with you on such an important program.</p>



<p>More about our work here: https://www.crn.com.au/news/meet-the-platform-innovator-finalists-in-the-2022-crn-impact-awards-582997</p>



<p></p>



<p></p>


CRN_Impact_FinalistTiles_PlatformInnovator

CRN Impact Awards


<p>If you have been paying attention to your data circles or attended the Data + AI Summit 2022, you might have heard that Delta Sharing is about to be released into the production offering for Databricks. This is exciting for users &amp; organisations who want to explore how Databricks can expand their solutions in secure, open data sharing.</p>



<h2><a></a>What is Databricks Delta Sharing?</h2>



<p>Delta Sharing is an open protocol for the secure real-time exchange of data in an attempt to streamline sharing solutions between data providers and their recipients. Delta Sharing is built around achieving 4 primary goals:</p>



<p><strong>Share real-time data directly</strong> without the requirement to copy it.</p>



<p><strong>Support a wide range of clients</strong>, allowing the ability for users to use data consumption tools of their choice.</p>



<p><strong>Strong security, auditing and governance</strong>. Grant, track and audit access points from a single platform.</p>



<p><strong>Efficient Scalability solutions. </strong>Delta Sharing facilitates the ability to scale to large datasets whilst leveraging the cost and elasticity benefits of cloud storage systems.</p>



<p></p>



<h2>So what?</h2>



<p>Whether you are looking to resolve headaches around data availability in multi-tenant user organisations, want to escape data sharing solutions being locked into a single vendor’s chosen computing platform, or simply want to get involved with <a href="https://databricks.com/blog/2022/06/28/introducing-databricks-marketplace-an-open-marketplace-for-all-data-and-ai-assets.html">Databricks Marketplace</a>; Databricks is building an exciting solution that is flexible to meet multiple use cases.</p>



<p><strong>Data Cleanrooms: </strong>The concept of data clean rooms is not new but is quickly gaining traction as businesses are coming to terms with how valuable their data is and how it can be efficiently monetised.</p>



<p>A clean room is a ‘location’ where two or more parties can collaborate on data and utilise the benefits of joint resources in a matter that is still private, secure and has governance measures in place. However, modern solutions still face problems with scalability, the movement &amp; availability of data and exclusivity supporting Structured Query Language (SQL). On top of the already incredible arsenal, the Databricks Lakehouse has on-demand, the rollout of the Unity Catalog (For building a single point of governance enforcement) &amp; Delta Sharing, it seems that Databricks aims to resolve the main concerns companies face when dealing with other modern data cleanroom solutions.</p>



<p><strong>Multi-tenant user organisations:</strong> It is not uncommon nowadays for larger companies to utilise multi-tenant architecture, especially when considering administration, configuration, and resource concerns. Previously, when looking to quickly and securely share data between ‘isolated’ business units may have arisen (change in configuration, adoption of new computing platforms to match different BU, etc.). With Delta Sharing, this task can be streamlined without taking away from BU benefits that would normally be associated with a multi-tenant architecture.</p>



<figure class="wp-block-image size-large"><img src="https://www.databricks.com/wp-content/uploads/2022/06/delta-sharing-img-5.png" alt=""/></figure>



<p><strong>Freedom of choice when choosing computing platforms: </strong>We’ve just considered multi-tenant architecture, which highlights a multitude of solutions that Delta Sharing could solve. However, what happens when you have partner organisations using different computing platforms that you’d like to share with? What if one of your data providers is on a different platform? Do you adopt new technology? Do you spend hours devising a crafty solution to pipeline the data across? Instead, Delta sharing opens the possibility for you to share the data in real-time, with no need to copy the data from a vast range of supported computing platforms.</p>



<p><strong>Databricks Marketplace:</strong></p>



<p>The Databricks Marketplace aims to provide an open community resource for users to openly share, collaborate and create both Datasets and Data solutions. This platform&#8217;s powered by Delta sharing, which is incredibly exciting for both data providers and data consumers alike, and will be an amazing resource for the data community as a whole. I highly suggest you go check out the announcement from Databricks <a href="https://databricks.com/blog/2022/06/28/introducing-databricks-marketplace-an-open-marketplace-for-all-data-and-ai-assets.html">here</a>.</p>



<h2><a></a>Delta Sharing Cheat Sheet, Some basic commands</h2>



<p>As of the time of writing this post, Delta Sharing is in open preview, so if you have previously signed up and have access, you can get started today! Otherwise, you will have to wait for general access or set up a local stand-alone environment.</p>



<p>If you are interested in getting access to exciting new Databricks features ahead of the crowd to play around with before release, I recommended signing up to the public preview <a href="https://databricks.com/product/delta-sharing#signup">here</a>.</p>



<p>Alternatively, if you feel like setting up a stand-alone Delta Sharing Server, you can check out the GitHub Repo (<a href="https://github.com/delta-io/delta-sharing">https://github.com/delta-io/delta-sharing</a>).</p>



<p>For this post, I will exclusively outline some basic config and commands to be run on the Databricks platform.</p>



<h3><a></a>Prerequisites</h3>



<ul><li>Ensure that you are an account or metastore admin (If you are not, you will need an admin to complete the “Enabling Delta Share” step on your behalf.</li><li>Have the unity catalogue enable with at least one metastore</li></ul>



<h3><a></a>Limitations:</h3>



<ul><li>As of the Open Preview, you <strong>cannot</strong> share views</li><li>`SELECT` is the only statement permission you can grant to a user</li></ul>



<h3>Enabling Delta Share</h3>



<figure class="wp-block-image size-full is-resized"><img loading="lazy" src="https://cms.excopartners.com/wp-content/uploads/2022/08/DataMetastores.png" alt="" class="wp-image-3094" width="390" height="147"/></figure>



<ol type="1"><li>Log into the account console: h<a href="https://accounts.cloud.databricks.com/">ttps://accounts.cloud.databricks.com/</a>, and head to the data pane to view your unity catalogue metastores.</li></ol>



<ol type="1" start="2"><li>Select the metastore which you want to share tables from and select the configuration tab</li><li>Check &#8220;Enable Delta Sharing to allow a Databricks user to share outside their organization&#8221;</li><li>(Optional) If you would like to curate how long a data recipient has access to the shared data. Make edits to the&nbsp; &#8220;Delta Sharing recipient token lifetime&#8221; to meet your use case.</li></ol>



<h3><a></a>Share Objects</h3>



<p>A `share` object is a collection of tables in a metastore that you want to share as a single group.</p>



<p>Note: Tables added to a share must all be from the same unity catalog metastore.</p>



<p>Firstly, boot up a notebook with `SQL` code blocks to get started.</p>



<h4>Creating a Share</h4>



<pre class="wp-block-code"><code><strong>CREATE</strong> &lt;share_name&gt;;</code></pre>



<h4>Show all existing Shares</h4>



<pre class="wp-block-code"><code><strong>SHOW</strong> SHARES;</code></pre>



<h4>List all tables in the target share</h4>



<pre class="wp-block-code"><code>```sql<br><strong>SHOW</strong> ALL <strong>IN</strong> <strong>SHARE</strong> &lt;share_name&gt;;<br>```</code></pre>



<h4>Delete a Share</h4>



<pre class="wp-block-code"><code>```SQL
<strong>DROP</strong> <strong>SHARE</strong> &lt;share_name&gt;;
```</code></pre>



<h4>Adding a table to a share</h4>



<pre class="wp-block-code"><code>```SQL
ALTER SHARE
    &lt;share_name&gt;
ADD TABLE
    &lt;database_name&gt;.&lt;table_name&gt;;
```</code></pre>



<h4>Removing a table from share bucket</h4>



<pre class="wp-block-code"><code>```SQL
<strong>ALTER</strong> <strong>SHARE</strong>
&nbsp;&nbsp;&nbsp; &lt;share_name&gt;
REMOVE <strong>TABLE</strong>
&nbsp;&nbsp;&nbsp; &lt;database_name&gt;.&lt;table_name&gt;;
```</code></pre>



<h4>Partition Data to share? No Problem</h4>



<pre class="wp-block-code"><code>```SQL
ALTER SHARE
    &lt;share_name&gt;
ADD TABLE
    &lt;database_name&gt;.&lt;table_name&gt;
PARTITION
    (&lt;expr&gt;),
    (&lt;expr_n&gt;),
    (...);
```</code></pre>



<p>Example:</p>



<pre class="wp-block-code"><code>ALTER SHARE
    my_share
ADD TABLE
    db.students
PARTITION
    (GPA &gt;= 3.0);</code></pre>



<h3><a></a>Recipients</h3>



<p>Recipients are a generated set of credentials that represents a user or set of users who you wish to share data with.</p>



<h4>Create Recipient</h4>



<pre class="wp-block-code"><code>CREATE RECIPIENT
    &lt;recipient_name&gt;;</code></pre>



<p>Note: Review the output created from this code block, as it will display the shareable link you are to provide, which will include the shareable link you will share with data recipients. <strong>Caution: </strong>This Crediential file can only be downloaded once.</p>



<figure class="wp-block-image size-full is-resized"><img loading="lazy" src="https://cms.excopartners.com/wp-content/uploads/2022/08/DownloadCredentialFile.png" alt="" class="wp-image-3097" width="316" height="198"/></figure>



<p>The recipient can then use this share file to upload into their desired computing platform, eg: Power Bi, Tableau, QLIK</p>



<h4><a></a>View Recipient Information</h4>



<p>This command will pull all relevant information about a named recipient.</p>



<p>I found this command particularly useful when I did not catch the activation link in the initial account creation.</p>



<pre class="wp-block-code"><code><strong>DESCRIBE</strong> RECIPIENT &lt;recipient_name&gt;;</code></pre>



<h4>List all recipients</h4>



<pre class="wp-block-code"><code><strong>SHOW</strong> RECIPIENTS;</code></pre>



<h4>Remove a recipient</h4>



<pre class="wp-block-code"><code>DROP RECIPIENT &lt;recipient_name&gt; ;</code></pre>



<h4>VIEW ACCESS</h4>



<pre class="wp-block-code"><code>SHOW GRANTS TO RECIPIENT &lt;recipient_name&gt;;</code></pre>



<h4><a></a>&nbsp;</h4>



<h3><a></a>PERMISSIONS</h3>



<p>The below commands will run through some core functionality which will manage access and permissions across your share objects</p>



<pre class="wp-block-code"><code><strong>GRANT</strong> <strong>REVOKE</strong> <strong>SELECT</strong> <strong>ON</strong> <strong>SHARE</strong> &nbsp;&nbsp;&nbsp; &lt;share_name&gt; <strong>TO</strong> RECIPIENT &nbsp;&nbsp;&nbsp; &lt;recipient_name&gt;;</code></pre>



<p>Note: At this point, SELECT is the only privilege you can grant.</p>



<h2><a></a>Read More:</h2>



<p>This blog was written from excerpts from the following posts. If you want to read further or see the original sources, please check out the links below:</p>



<ul><li><a href="https://docs.microsoft.com/en-us/azure/databricks/data-sharing/delta-sharing/">https://docs.microsoft.com/en-us/azure/databricks/data-sharing/delta-sharing/</a></li><li><a href="https://delta.io/sharing/">https://delta.io/sharing/</a></li><li><a href="https://databricks.com/blog/2021/05/26/introducing-delta-sharing-an-open-protocol-for-secure-data-sharing.html">https://databricks.com/blog/2021/05/26/introducing-delta-sharing-an-open-protocol-for-secure-data-sharing.html</a></li><li><a href="https://docs.microsoft.com/en-us/azure/databricks/data-sharing/delta-sharing/recipient">https://docs.microsoft.com/en-us/azure/databricks/data-sharing/delta-sharing/recipient</a></li></ul>
