PnP Provisioning

Extract and apply templates with Office Dev PnP Provisioning PowerShell

Earlier I introduced the Office Dev PnP Program and Office Dev PnP Provisioning. I mentioned the great functionality to define a template of a site or site collection in XML. In this blog post I will explain how to extract and apply XML templates with Office Dev PnP Provisioning using the Office 365 Developer Patterns and Practices PowerShell Cmdlets, from now on referred to as PnP Provisioning cmdlets.

PnP Provisioning cmdlets?

Basically the PnP Provisioning cmdlets are an extension to the standard CSOM functionality Microsoft offers. PnP Provisioning cmdlets are specifically designed to provision and modify artifacts in SharePoint Online using CSOM.

Layers of CSOM and PnP

Preparing you environment

Of course, you first need to prepare your environment before you are able to use the PnP Provisioning cmdlets. Although the cmdlets are designed for SharePoint Online, it is also possible to use the cmdlets on an on-premises environment. The on-premises variant of the cmdlets is another build, because SharePoint 2013 uses an older version (15) of CSOM whereas SharePoint Online uses the newest version (16) of CSOM. Therefore you first need to determine the right build of the PnP Provisioning cmdlets:

  • msi for SharePoint 2013
  • msi for SharePoint Online

Next download the latest release of the right build from the release page and run the install. Erwin van Hunen created a blog post about running the install and connecting to the SharePoint environment with Connect-SPOnline. Complete the installation as described in his blog post.

Connect and get the context

As mentioned earlier you can use Connect-SPOnline to connect to SharePoint. Connect-SPOnline creates the context you need to acquire or apply the XML template.

To ease your life, use the Windows Credential Manager to store your credentials. This way you don’t have to fill in your credentials each time you connect to SharePoint with Connect-SPOnline without having to hardcode your credentials in the script.

Almost ready to extract

Once you have the necessary context you are almost ready to extract a template. One thing is missing: the source to extract from. The source is a SharePoint site which is designed as the blueprint of to be provisioned sites. Create this source site exactly as you want this blueprint to be. Do this in a root site (of a separate site collection) for best results. In a subsite it is possible that some artifacts are not extracted properly. Things like site columns are only obtained correctly when the source is a root site. Once you created the source site, you are ready to extract.

Extract a XML template with Get-SPOProvisioningTemplate

Now you have a source site as blueprint, you need the context of the source site. So if /sites/MySourceSite is the source site your code should look like something like this:

# Define variables
$tenant = "tenantName";
$sourceSite = "/sites/MySourceSite";
$path = "C:\MyPnPTemplate.xml";

# Get context
$webUrl = "https://{0}.sharepoint.com{1}/" -f $tenant, $sourceSite;
Write-Output $("Connecting to {0}..." -f $webUrl);
Connect-SPOnline -Url $webUrl -Credentials WCMStoredCredentials;
Write-Output "Context obtained";

To make the script easier to use with different environments, I used a $tenant and a $sourceSite variable to define the name of the SharePoint tenant and the source site respectively. Later these variables are used in $webUrl to define the complete URL of the source site. The $path variable is to define the location where the XML template should be saved. Last thing to do is the actual extraction with Get-SPOProvisioningTemplate and save the XML template to the given path:

# Extract template
Write-Output "Creating PnP template...";
Get-SPOProvisioningTemplate -Out $path;
Write-Output $("Template saved to {0}" –f $path);
Combined the code looks like this:

# Define variables
$tenant = "tenantName";
$sourceSite = "/sites/MySourceSite";
$path = "C:\MyPnPTemplate.xml";

# Get context
$webUrl = "https://{0}.sharepoint.com{1}/" -f $tenant, $sourceSite;
Write-Output $("Connecting to {0}..." -f $webUrl);
Connect-SPOnline -Url $webUrl -Credentials WCMStoredCredentials;
Write-Output "Context obtained";

# Extract template
Write-Output "Creating PnP template...";
Get-SPOProvisioningTemplate -Out $path;
Write-Output $("Template saved to {0}" –f $path);

Apply a XML template with Apply-SPOProvisioningTemplate

Applying a XML template is as easy as extracting a XML template. Obtaining the context works the same, but instead of a defining source site, you define a target site. With Get-SPOWeb you get the context of the subsite. Now it’s time to apply the XML template with Apply-SPOProvisioningTemplate:

# Define variables
$tenant = "tenantName";
$targetSite = "/sites/MyTargetSite";
$path = "C:\MyPnPTemplate.xml";

# Get context
$webUrl = "https://{0}.sharepoint.com{1}" -f $tenant, $ targetSite;
Write-Output $("Connecting to {0}..." -f $webUrl);
Connect-SPOnline -Url $webUrl -Credentials WCMStoredCredentials;
Write-Output "Context obtained";

# Get web object
$web = Get-Web;

# Apply template
Write-Output $("Applying PnP template [{0}] to site [{1} ({2})]..." -f $path, $web.Title, $web.Url);
Apply-SPOProvisioningTemplate -Web $web -Path $path;
Write-Output "Template created";

After some time (seconds or minutes, depending on the size of the template and the connection speed) your template will be successfully applied to the site.

In the sample a $web context is used as input for Apply-SPOProvisioningTemplate, but when no web context is given, Apply-SPOProvisioningTemplate will use the context of Connect-SPOnline (which in this sample is indeed the same context as $web). The code will then look like:

Apply-SPOProvisioningTemplate -Path $path;

The XML template is now applied on an existing site. It could also be applied to a new site by creating the new site first with New-SPOWeb using the standard blank (STS#1) template.

# Define variables
$tenant = "tenantName";
$targetSite = "/sites/MyTargetSite";
$path = "C:\MyPnPTemplate.xml";
$siteTitle = "My Target Site";
$siteDescription = "My Target Site Description";
$siteUrl = "MyTargetSite";

# Get context
$webUrl = "https://{0}.sharepoint.com/{1}" -f $tenant, $targetSite;
Write-Output $("Connecting to {0}..." -f $webUrl);
Connect-SPOnline -Url $webUrl -Credentials WCMStoredCredentials;
Write-Output "Done";

# Create new subsite
$web = New-SPOWeb -Title "$siteTitle" -Url "$siteUrl" -Description "$siteDescription" -Locale 1043 -Template "STS#1"

# Apply template
Write-Output $("Applying PnP template [{0}] to site [{1} ({2})]..." -f $path, $web.Title, $web.Url);
Apply-SPOProvisioningTemplate -Web $web -Path $path;
Write-Output "Done";

This is also a good example of a situation when you actually need the $web context. In the coming blog posts I will explain how to create a XML template yourself.

Further information

PnP Provisioning

Getting started with Office Dev PnP Provisioning

Office Dev PnP provides a site provisioning engine to create sites and site collections within a SharePoint Online tenant or a SharePoint farm based on a given definition. This definition is an object describing the to be provisioned components like list, libraries and features. But also SharePoint security groups, users, site property bags entries and Composed Looks.

With the template provider the definition object can be transformed to JSON or XML. This works in both ways, so you can transform a XML definition into an object. Therefore you are able to define the structure of a site or site collection in a XML template. The definition object will be referred to as PnP template.

Classic site provisioning techniques

Below are a few classic site provisioning techniques. Vesa Juvonen explains these techniques in more detail.

Site Definition
Requires full trust solutions, which means it is not possible to use in Office 365. Also it will be difficult to migrate site definitions to newer versions of SharePoint.

Site Templates
Not possible to create site collections without workarounds and is difficult to upgrade. This method is however possible on Office 365.

Web Templates
Not possible to create site collections without workarounds. This method is however possible on Office 365.

Server side provisioning
Provides standardization of site creation in code and is easy to upgrade. Requires full trust solutions, which means it is not possible to use in Office 365.

Introducing remote provisioning and PnP Provisioning

The modern way of site provisioning is remote provisioning. Remote provisioning uses the Client Side Object Model (CSOM) or the REST API to create a site and fill this site incrementally with CSOM/REST requests. This model is suitable for both on-premises and Office 365 since we control the customizations outside of the SharePoint farm in an independent solution.

To improve the experience using remote provisioning, the Office Dev PnP team created PnP Provisioning. As mentioned earlier, a really great part of PnP Provisioning is defining a site template in a XML using the PnP Provisioning Schema. In my next blog post I will explain how to create a site based on a template defined using the PnP Provisioning Schema.

Further information

Introduction to the Office Dev PnP Program

Many organizations using SharePoint have created legacy solutions on the platform. This lead to all kinds of migration problems. Since these legacy problems prevented customers to migrate to the cloud, Microsoft started an initiative to ease these problems. And so in 2013 the Office Dev Patterns and Practices Program was born, shortly Office Dev PnP.

Office Dev PnP originally started as a Microsoft initiative to help customers migrate their on-premises farms to the Microsoft cloud. Soon it introduced many helper code and examples to provide the basic plumbing and optimize your workflow.

Components

All components of Office Dev PnP are placed on GitHub. It started in a single repository, but soon the volume of the program became so large, the Office Dev PnP team decided to split the individual components in separate repositories.

PnP
The original repository which now only contains samples.

PnP Sites Core
Contains the core library and the PnP provisioning engine. The PnP provisioning engine helps to deploy customizations to remote SharePoint environments.

PnP PowerShell
Contains PowerShell wrappers to the functionalities of PnP Core.

PnP Partner Pack
Contains the PnP Partner Pack, a starter kit for customers and partners and combines numerous patterns and practices demonstrated in the PnP samples to one reusable solution, which can be deployed and used in any Office 365 tenant.

PnP Guidance
Contains the documentation and guidance of the program. The contents of the master branch are published to dev.office.com and MSDN.

PnP Office Add-ins
Contains samples of how to work with the HTML and JavaScript based Office add-in model.

PnP Tools
Contains more tools, code packages and scripts for the on-premises and hybrid setup, than purely for the Office 365 operations. Using classic terminology, this could be considered more targeted for IT pro’s, architects and administrators in on-premises deployments.

PnP Transformation
Contains the farm solution to add-in model transformation tooling and guidance.

PnP Provisioning Schema
Contains the provisioning schema for PnP Provisioning, a community driven effort for designing one schema which can be used to define elements in the Office 365.

Resources

Office Dev PnP has many resources. In the picture below are the official channels.

Overview Office Dev PnP Resources

Contribute to PnP

Office Dev PnP is completely open source. Microsoft and the Office Dev PnP team encourage you to contribute to the program. When using the Office Dev PnP resources, you will probably encounter bugs or come up with improvements. Please report the issues to the PnP projects on GitHub or contribute your own fixes by creating pull requests.

Restore the default SharePoint groups for a SharePoint site

A SharePoint group is a collection of users who all have the same set of permissions, or permission level. Certain groups are created by default when you create a SharePoint site or activate certain features, these are known as the default SharePoint groups. The default groups use default permission levels to grant users rights and access.

Associated groups

Each SharePoint site has three associated groups (AssociatedOwnerGroup, AssociatedMemberGroup and AssociatedVisitorGroup) who define which SharePoint groups are designated as Owners, Members or Visitors groups of a site respectively. When role inheritance is broken, by the default these SharePoint groups are “<sitename> Owners”, “<sitename>” Members and “<sitename> Visitors”.

The problem

When these associated groups are not created on site creation or when one or more of the associated groups are deleted, you need to (re)create those associations (if you want the default Owners, Members and Visitors groups as SharePoint creates them). Unfortunately SharePoint doesn’t offer an intuitive interface to do this.

Create the missing SharePoint groups

To restore the default SharePoint groups, you first need to make sure the default groups exist. Go to <site URL>/_layouts/15/groups.aspx to make sure the following groups are present:

  • <site name> Owners
  • <site name> Members
  • <site name> Visitors

If not you have to create the missing manually (New > New Group) with the corresponding permission levels, as shown below.

Default groups of SharePoint site

Set the SharePoint groups a default groups

When the default groups are present, you have to create the associations. Go to <site URL>/_layouts/15/permsetup.aspx and select the corresponding default groups in the form.

Set group associations of SharePoint site

Finally, press OK to create the associations. Now the default SharePoint groups of the site are restored.

Authenticate an Office 365 user with ADAL JS

In single page applications (SPAs) you consume REST endpoints to retrieve and change data. With Web API you can secure these endpoints using Azure AD and authenticate to these endpoints with OAuth 2.0. Active Directory Authentication Library for JavaScript (ADAL JS) helps you to use Azure AD for handling authentication in your SPAs. The sources can be found on GitHub. In this blog post I describe how to authenticate an Office 365 user (which is an Azure AD user) to a Web API endpoint with ADAL JS.

First things first

Make sure your Office 365 tenant is associated with your Azure AD tenant. When your tenants are associated you need to sign in to the Azure management portal and register your application principal in the Azure AD tenant. The next step is to enable the OAuth 2 implicit grant for your application principal. Make sure the application principal has at least Read user files and Read items in all site collections permissions for the Office 365 SharePoint Online application.

Basics to authenticate to an endpoint with ADAL JS

When using ADAL JS, the library will first create an authentication context based on a configuration object in the constructor. Next ADAL JS will check if the user is authenticated. If the user is not yet authenticated, ADAL JS will redirect the user to the Azure AD login page. When the user is authenticated (within the right Azure AD tenant), ADAL JS provides a function to acquire an access token for an endpoint defined in the configuration object. Within this function you use this access token to authenticate to the endpoint.

Using ADAL JS to authenticate with Office 365 user

An Office 365 user is also a Azure AD user. To get a better understanding of how to authenticate an Office 365 user to multiple endpoints with ADAL JS, I will demonstrate how to get the OneDrive documents of the current user and a list of items within a given SharePoint list.

First we create the configuration object as input for the authentication context. This object specifies the Azure AD tenant, ClientId of the application principal we registered earlier in Azure AD, the URL to which the user should be redirected after login, the endpoints you want to connect to and the cache location.

When the authentication context is created, we make sure the user is authenticated.

If the user is authenticated, we acquire the access token for OneDrive to access the OneDrive files of the current user. The acquireToken function returns an access token for the given endpoint. Since we passed in the endpoint of OneDrive, we can use this token in the REST call to OneDrive. Now the REST call will return a list of all items in the OneDrive root folder.

To obtain an access token for SharePoint, we just use the acquireToken function again with the SharePoint endpoint as input. The function returns an access token which we use in the call to get the titles of all files in a given SharePoint list. Probably you will receive a CORS error the first time you open the page. Refreshing the page should solve this.

Conclusion

You have seen that with ADAL JS you can authenticate an Office 365 user to the endpoints of multiple Office 365 applications. This is not limited to only Office 365 applications. You can authenticate to the endpoints of any other application when the application principal in Azure has received the required permission.

Don’t forget to checkout the complete code on GitHub.

Further information