What is it?

It is a GUI application that looks at exports from your Orchestrator environment and allows you to perform multiple actions on them before re-importing them into the same environment or another Orchestrator Environment

Primary Functions

1. Display a Graphical Representation of an Export
   1. We use this internally while doing change control to ensure that all Global Configurations are properly configured in the destination Orchestrator Environment
2. Sanitize Export
   1. This will remove all un-referenced global Settings and Configurations
   2. http://blogs.technet.com/b/orchestrator/archive/2012/05/15/3497985.aspx
   3. http://opalis.wordpress.com/2013/04/11/parse-orchestrator-export-tool/
3. Apply Link Best Practices
   1. Modifies the links of an export to give them appropriate names and colors
   2. http://opalis.wordpress.com/2013/04/12/orchestrator-link-best-practices/
4. Set Max Parallel
   1. Allows for Bulk setting of Runbook Job Concurrency
   2. http://opalis.wordpress.com/2013/04/17/bulk-set-orchestrator-max-policy-requests/
5. Turn On / Off Generic Logging
   1. Turns Off or On Generic Logging for All Runbooks
   2. http://opalis.wordpress.com/2013/04/11/parse-orchestrator-export-tool/
6. Turn On / Off Object Specific Logging
   1. Turns Off or On Object Specific Logging for All Runbooks
   2. http://opalis.wordpress.com/2013/04/11/parse-orchestrator-export-tool/
7. Analyze Export
   1. Iterates through the Export File and checks for common misconfigurations
      1. Multiple Link objects between the same source and destination objects
      2. Trigger Policies that do not have all of the input parameters for the target policy defined
8. Modify Name
   1. Changes the name of any Runbook / Folder / Object / Global Configuration

Modify Name

This is one of the most exciting functions of this tool in my opinion. This, coupled with “Set Max Parallel” makes it very simple to set Job Concurrency in bulk. It also makes it very easy to ‘duplicate’ runbooks and have them maintain their own unique sets of global settings / configurations for connecting to resources

Max Parallel Example

Say I wanted I had a Runbook that is called from SCCM task sequences through our Web Service. I probably want each one of my Runbooks to be able to be called by different machines being re-imaged. By default though, each Runbook has it concurrency set to 1. This means that all of my requests will ‘stack up’ behind one another (ie if I have 5 machines that make a call to my workflow for shutting off Port Authentication they would be handled one at a time in the order they were received even though they could all run at the same time). To change this behavior we can go in with the Runbook designer to each Runbook and set ‘Job Concurrency’ to something other than one

Or, if we want to do it in bulk we can use the Export to to change a folder name then click ‘Set Max Parallel’ and it will change the setting for all of our Runbooks!

And now all the Runbooks under that folder have their concurrency set to 5! More information about ‘Set Max Parallel’ can be found at http://opalis.wordpress.com/2013/04/17/bulk-set-orchestrator-max-policy-requests/

Runbook Duplication

The most interesting use of ‘Modify Name’ I think though is the ability duplicate a Runbook in such a way that it can live in the same environment as its base Runbook without sharing any references. Imagine if you will a scenario where we have a Runbook that will be called by a task sequence. The first version of these Runbooks *should* be pointed at ‘development’ infrastructure – a test Active Directory domain, a test SCCM instances etc etc – the though becomes how do we make a ‘production’ version of this without much effort and while keeping our development version? Modify Name to the rescue!

Step 1: Export your Runbook

Step 2: Sanitize your Runbook with the tool

Step 3: Modify Global Names to Ensure Uniqueness

If there is anything in your export folder that you need to have a ‘separate’ instance of for the ‘new’ version then simply change its name using Modify Name (it will update both the element name and all references to that name). We usually will re-name the Root Runbook Folder, the Root Variable Folder and all Global Configurations

Renaming Root Runbook Folder

Supply the New Name and Click Ok

Renaming Root Variable Folder

We also rename the root Variable folder, this means that we will have two totally separate versions of all variables and we can update them independently

Renaming Global Configurations

We can also rename Global Configurations (those things under ‘Options -> Integration Pack’) meaning we don’t have to go in and ‘re-point / reconfigure’ all of our objects to change a global configuration

Now we have unique versions in our export file of everything that we want to be independent of the other version. We can no re-import and go about our day!

Hopefully everyone finds this useful!

Setting from a Runbook in the Disable in AD folder – Defaults to 20

Setting from a Runbook in the Disable in AD folder – Defaults to 5

Setting from a Runbook in the Disable in AD SCOM – Automated Process does not change value

How to use the automated process?

This functionality has been built into the Parse Orchestrator Export Tool (available at https://scorch.codeplex.com/releases/view/104915)

And into the Orchestrator Administration Integration Pack (Available at https://scorch.codeplex.com/releases/92063)

Enjoy!

What are the Best Practices

We defined the best practices as follows

1. If a link comes from an object that is flattened, modify the name of the link to reflect
2. If a link has a delay modify the name to show the delay and if a color is not set change it to blue
3. If a link is a ‘success’ of previous object change it to green and add success to the name
4. If the link is a ‘always continue’ (success, warning, failure) of previous object update link name to always and change color to green width 2
5. If the link is a ‘failure’ or ‘failure or warning’ of previous object update link name to failure and set color to red
6. If there are other criteria (multiple criteria or based on property of previous object) set link to orange and add custom criteria to the end

What does it look like?

Here is an example of a runbook pre-export and link modification

And post modification

How to set it up on the export runbook object

Set the optional property

How to use it on the Parse Orchestrator Export Tool

Enjoy!

What does this tool do?

That’s a great question! It looks at a Orchestrator Export File – point it to one and click ‘Parse’

Now you can see all of the cool things that were included in your export and some additional features are shown

Sanitize Export

This will modify the export file and remove all of the unreferenced global settings and configurations (as you can see above this export has a lot of them)

Now that the clean is done lets re-parse!

Turn On / Off Object Specific Logging

This will turn on or off the Object Specific Logging flag for all runbooks

Turn On / Off Generic Logging

This will turn on or off the Generic Logging flag for all runbooks

Enjoy!

How to Benchmark

Orchestrator runbook activities can be thought of as having two distinct types of code: platform code and domain code. The term domain code is used to identify code within a runbook activity that is typically not associated with the Orchestrator platform itself (with notable exceptions, such as Invoke Runbook, Junction, and others). For example, the Invoke Web Service standard activity would contain Orchestrator platform code (the “plumbing” of the activity) as well as domain code unique to invoking a SOAP-based web service. The platform code will be very similar for most activities, since it is built on a common framework. However, there will potentially be great variation in domain code for different activities.

Data Logging

Another aspect of runbook performance is data logging. For the purpose of understanding performance consider two logging configurations: Default logging and Common Published Data logging. Default logging results in approximately 524 bytes of data being written to the Orchestrator database each time an activity is run. Logging of common published data writes approximately 6,082 bytes of data (12 times the default logging level). There is a notable difference in performance between these logging levels.

Consider the scenario where the same runbook activity is run twice, once with data logging at the default level and once with logging of common published data enabled. The domain code should take the same amount of time to complete. However, the platform code will take longer to run with common published data logging enabled. Essentially, the platform code has to support logging 12 times more data with common published data enabled than it did when running at the default logging level.

The Standard Activity Compare Values can been used to create benchmarks of an Orchestrator environment.

To create a runbook that can be used to benchmark your Orchestrator environment

This simple one-activity runbook will run a Compare Values activity 10,000 times. Compare Values is a very simple activity whose domain code is quite minimal. This runbook can be invoked under a variety of circumstances to characterize the overall performance of a given Orchestrator runtime environment.

This runbook can be used to experiment with different configurations of Orchestrator. For example, supposed you wanted to determine the performance of four Runbook Servers deployed to different data centers.

Notice the significant decrease in platform performance caused by logging of common published data. The worst scenario appears to be logging of common published data at Location 2. On the surface, this appears to be a clear and relevant conclusion.

However, it should be noted that these figures reflect the overhead of the platform code, not the domain code. Domain code runtimes can be significantly longer. For example, the Create VM from Template activity in the Virtual Machine Manager Integration Pack may run for several minutes as the VM is created. Expanding on the previous example, consider the platform code costs on a runbook activity that takes 1 minute to run (1 minute = 60,000 milliseconds) regardless of location.

A clearer picture begins to emerge from the data. The scenario where logging of common published data is enabled at Location 2 continues to be the worst performer. However, the platform code and logging only accounts for 6% of the total runtime. While this is a significant figure, the best-case scenario is 1.4%. Essentially, the time spent in the domain code in the example far outweighs the time spent running platform code. To put this in perspective, if you were able to completely eliminate the platform code costs, you would only see runbook performance improvements in the range of 1.4 to 7.4%.

Of course most real-world scenarios will be different. Activity behavior may change depending on what the domain code is told to do. For example, a Clone VM from Template activity may take one minute to clone a VM from Server Template A, but take 5 minutes to clone a VM from Server Template B. Also, Runbook Servers may reside on different networks with different performance characteristics which can potentially impact both domain code performance as well as Orchestrator data logging performance.

To summarize:

    Make careful decisions about when to log published data.

    Carefully consider the impact of logging common published data. Remember that the number of times activities run determines the volume of logged data. A runbook with a small number of activities run many times can result in more data logging than a larger runbook run a small number of times.

    Do not enable logging of activity specific published data in production environments.

    Develop an understanding of how much time your runbooks spend running domain code compared to running platform code.

    Estimate platform code costs using the techniques outlined in this document. Use as a reference in considering where to make improvements in runbook performance.

    Use the techniques outlined in this document to gain a deeper understanding of the relative performance of your different runtime environments. Identify opportunities for improvement by making normalized comparisons of your measurements.

Using this information

Given these considerations we have internal policies that say we will turn on logging (Object Specific) in our central QA environment and turn it off in our Central Production environments to maximize performance. We do two things to make sure we have logging turned on in development environment exports and turned off from QA environments. We do this by changing the logging flags on our nightly backup jobs (discussed at http://opalis.wordpress.com/2012/08/06/automating-the-export-and-tfs-check-in-of-workflows/)

Development Environment

QA Environment

We also have jobs that run in our production environments that periodically run a SQL query to disable logging on all Runbooks in the environment

Query to find Runbooks with Logging Turned on

--Find all Runbooks with Object Specific Logging Enabled 
select * from POLICIES where LogSpecificData = 1

--Find all Runbooks with Common Data Logging Enabled
select * from POLICIES where LogCommonData = 1

Query to turn off logging

--Turn Off Common Logging for All Runbooks

Use Orchestrator 
DECLARE @id UNIQUEIDENTIFIER 
DECLARE @name NVARCHAR(MAX)
DECLARE curs CURSOR FOR
SELECT 
UniqueId, Name
FROM
POLICIES
WHERE
LogCommonData = 1
OPEN curs
FETCH NEXT FROM curs
INTO @id, @name
WHILE @@FETCH_STATUS = 0
BEGIN
PRINT 'Turning Off Common Data Logging For : ' + @name
UPDATE POLICIES
SET LogCommonData = 0
WHERE UniqueID = @id
FETCH NEXT FROM curs 
    INTO @id, @name
END
CLOSE curs
DEALLOCATE curs

--Turn Off Object Specific Logging for All Runbooks

Use Orchestrator 
DECLARE @id UNIQUEIDENTIFIER 
DECLARE @name NVARCHAR(MAX)
DECLARE curs CURSOR FOR
SELECT 
UniqueId, Name
FROM
POLICIES
WHERE
LogSpecificData = 1
OPEN curs
FETCH NEXT FROM curs
INTO @id, @name
WHILE @@FETCH_STATUS = 0
BEGIN
PRINT 'Turning Off Specific Data Logging For : ' + @name
UPDATE POLICIES
SET LogSpecificData = 0
WHERE UniqueID = @id
FETCH NEXT FROM curs 
    INTO @id, @name
END
CLOSE curs
DEALLOCATE curs

Step 1: Accidently Delete Something (this could be a Runbook, Folder, Variable etc.)

Step 2: Find the Folder you deleted

Useful SQL Query to help you along

Use Orchestrator
Select UniqueID, ParentID, LastModified, Deleted from FOLDERS where Name like 'Central Workflows'

Step 3: Recover the deleted folder

Use Orchestrator
UPDATE FOLDERS set Deleted = 0 where UniqueID = '97627FD3-2CD0-4B88-BA90-6B1BA8C6E4EC'

Step 4: Recover child items

Find Folders deleted around this time

use Orchestrator
select * from FOLDERS where Deleted = 1 and LastModified > '2013-04-10 19:38:00.000' and LastModified < '2013-04-10 19:40:00.800'

Changed their deleted bit to 0

use Orchestrator
UPDATE FOLDERS Set Deleted = 0 where Deleted = 1 and LastModified > '2013-04-10 19:38:00.000' and LastModified < '2013-04-10 19:40:00.800'

Find the deleted Runbooks

use Orchestrator
select * from POLICIES where Deleted = 1 and LastModified > '2013-04-10 19:38:01.000' and LastModified < '2013-04-10 19:38:04.800'

Recover the Deleted Runbooks

use Orchestrator
UPDATE POLICIES Set Deleted = 0 where Deleted = 1 and LastModified > '2013-04-10 19:38:00.000' and LastModified < '2013-04-10 19:40:00.800'

Find the Deleted Objects and Recover Them

Rejoice that Orchestrator doesn’t delete things!

The Service Request

Step one is creating the SR and adding the activities. We add 4 activities, an OrchestratorRunbook (our way of calling out to Orchestrator discussed at http://opalis.wordpress.com/2013/04/03/triggering-orchestrator-runbook-from-service-manager/) and 3 Manual activities

• OrchestratorRunbook
  
  • Used for initiating the Orchestrator Runbook
    
  • Has 40 ‘input property’ fields to map values to from the Request Offering
    
• Manual Activity one: Managed By User Selection
  
  • Used for holding the user selected in the managed by Query Result control
    
• Manual Activity two: End User Selection
  
  • Used for holding the end user selected in the end user Query Result control
    
• Manual Activity three: Computer Selection
  
  • Used for holding the computer selected in the Computer Selection Query Result control
    

Request Offering

In the request offering on the Query Result Controls we set them to relate their outputs to the correct Manual Activity

Runbook

Now in our Runbook we can get all of the related Manual activity, filter down to an individual one and select the object output!

And now we can uniquely identify which user is which and which computer is which J

The first idea floated around the office was to simply overwrite the max number of results returned for this control to 30,000. This is doable and explained at http://blogs.technet.com/b/servicemanager/archive/2011/11/08/advanced-query-results-customization-for-request-offerings.aspx but puts a large load on the SM database. It also pulls back the full dataset (30,000 user CI items) to the end users web page; this means that the control takes a long time to load. What we really wanted was a people picker type control like users are used to from SharePoint that does server side filtering based on their inputs and returns smaller datasets.

After a fair amount of reading / searching the blogosphere I came across a post by Nathan Lasnoski about using tokens from previous selection in query results to lower the number of results loaded by the query result object http://blog.concurrency.com/infrastructure/service-manager-request-query-result-filtering/ which got us really close. We no longer needed to increase the number of query results returned to 30,000 so the control loaded a bit faster (still, loading 3,000 results initially takes a while so it wasn’t optimal) but we still had a rather large User Experience problem results initially loaded into the control. This tempted our users to believe that all results were being loaded and to just use the ‘search for instances’ functionality of the Query Result Control instead of specifying any filter criteria (ie the user name). My initial thought was to just set the default values of the text controls to something invalid, but, as far as I know this isn’t possible with text controls.

At this point a whiteboard became involved. I wrote up the properties that we wanted from the control and information we knew and tried to figure out how to accomplish it

Desired Properties:

• A single text input box that an end user places the name (ID or friendly) into.
• A query result control that only display’s results based on the input in the above box
• The query result control should be blank if the input box is blank

Solution

The ultimate solution designed was based on Nathan’s idea and an extension of the AD User class. The class was extended to include a property that is always blank in our environment. This allows us to add an additional filter criteria to our query results control, only display AD Users who AD User Property [Blank] does not equal the token from the text box (ie only display AD Users when the text box is not blank). Our requests now load quickly (no data is loaded into the Query Result initially) and our users are not confused by seeing ‘partial’ datasets not based on their filter criterion.

Wrinkle

The one problem with this model is that by default blank properties are defined in the SM database as ‘null’ not the SQL empty string. This means that doing any comparisons between them and a text field (ie, the token from Filter Users) will result in NO results being returned. To work around this we have a nightly job that updates our extension database class field ‘blank’ and sets it to the SQL empty string. For us this query is below, it will be different in your environment but if you dig around in your SQL DB I am sure you can figure it out. As always, direct modification of the SQL tables is a ‘at your own risk’ operation but, for our use case it seems to work without adverse effect.

update MT_ClassExtension_378d3dfe_23d5_4f39_8218_5335167ad8c9 set Blank_60EC6C94_0BE6_2C63_90B7_4FB40CB9255D = '' where Blank_60EC6C94_0BE6_2C63_90B7_4FB40CB9255D is null

To do this refresh we have an Orchestrator Runbook running the above query 1 time per day after our ‘AD Sync’ connector has finished

The default syncing mechanism for integrating Service Manager and Orchestrator is based on discovery through connectors and initiation by Runbook Guid. This is great for demos and simple implementations of Service Manager and Orchestrator, you just connect the two, click on a Runbook and say create Runbook Automation Activity template and you are off to the races.

The problems begin to arise if your environment is a bit more complicated. In our environment we have 8 Orchestrator development environments, 1 Orchestrator quality assurance environment and 2 mirrored production environments (web services behind a F5 VIP that points to the currently active environment). In this sort of staged out environment things become less clear, how do you update your Runbook activity templates when the environment moves from a development environment to the QA environment while minimizing the configuration changes? How can you initiate a Runbook in a mirrored production environment where the Guids between the two environments are different but everything else is the same?

Solution: A new Activity Type to replace Runbook Activity Templates

The cool part about most Microsoft products lately is that if you don’t like the default functionality you are not stuck with it. What we did was create a new class (OrchestratorRunbook) that is based on the activity class. This class has a number of properties that are specific to the Runbook, what Web Service URL should be used to initiate it, what is its path, what is the path to a command line tool for initiating the Runbook and 20 string properties for mapping input values to.

There is then a workflow that fires off when an instance of the activity is changed to the ‘In Progress’ or ‘Re-Run’ status

This workflow runs a PowerShell script that calls the ScorchLauncher command line utility for initiating Runbooks through the Orchestrator web service http://scorch.codeplex.com/releases/view/102637 which has built-in retry logic if the web service is temporarily un-available.

The script it is running is

#——————————————————————————————————————————————————————————————————————————————————–

write-host “Scorch Launcher Path: $scorchLauncherPath”

write-host “Webserver URL: $orchestratorSVCUrl”

write-host “Runbook Path: $runbookPath”

write-host “ReferenceID: $ReferenceID”

if($userName -and $userDomain -and $userPassword) {

Write-Host “Passed Username [$Username] Domain [$userDomain] and Password”

Write-Host $scorchLauncherPath -WebServerURL “$orchestratorSVCUrl” -RunbookPath “$runbookPath” -InputParameters “ReferenceID|$ReferenceID” -Username “$userName” -Domain “$userDomain” -Password “Password” -WaitForExit $true

& $scorchLauncherPath -WebServerURL “$orchestratorSVCUrl” -RunbookPath “$runbookPath” -InputParameters “ReferenceID|$ReferenceID” -Username “$userName” -Domain “$userDomain” -Password “$userPassword” -WaitForExit $true

}

else {

Write-Host “Not Passed User Name”

Write-Host $scorchLauncherPath -WebServerURL “$orchestratorSVCUrl” -RunbookPath “$runbookPath” -InputParameters “ReferenceID|$ReferenceID” -WaitForExit $true

& $scorchLauncherPath -WebServerURL “$orchestratorSVCUrl” -RunbookPath “$runbookPath” -InputParameters “ReferenceID|$ReferenceID” -WaitForExit $true

}

#——————————————————————————————————————————————————————————————————————————————————–

As you can see the script passes a single parameter to the target Runbook called “ReferenceID” (you need to have a initialize data on your connecting Runbook with this input parameter defined). The unique Guid of this activity will be passed to your Runbook, from there you can use objects from the SCSM 2012 integration pack to get the rest of your data. Specifically you can use the ‘Get Object’ to read the rest of the input values or ‘Get Related Object’ to get the related Service Request and its properties.

Creating the SCSM Request Offering Suite

Now that we have the class created in a sealed management pack we can create instances of it in any other management pack! Internally we have a best practice to make a new management pack per service offering, so, for testing and demo purposes I will create a new unsealed management pack. Each unsealed management pack will need one template instance of this OrchestratorRunbook class created initially, after that you simply add an instance of that template to each new Service Request Template instance and configure it to the Service Request specifications (target it at the correct Runbook and Web Service)

Creating the Initial OrchestratorRunbook Activity Template

Since I am making a new Service Offering for this example I need to create a new OrchestratorRunbook template instance

In our base template we set the Activity status to Pending initially, this is to avoid a race condition inside of Service Manager (discussed here http://social.technet.microsoft.com/Forums/en-US/administration/thread/0f4a187d-ad74-473e-ab25-5f21f8ec5218). We can also change the default Orchestrator Web Service URL or Runbook Path, both of these properties can and be over written when an instance of this template is added to a Service Request.

Creating the service request

Now that we have the activity template created we can create our Service Requests and add in instances of this OrchestratorRunbook activity to trigger off our runbooks!

Just like normal, when we want to trigger a Runbook we go to the ‘Activities’ portion of the SR Template and add in a new instance the OrchestratorRunbook Activity template class we created. We can now define, on a SR to SR basis (and update easily) what web service it is initialized through (huge for multi-instance Orchestrator deployments) and the path of the Orchestrator Runbook!

Creating the Service Request Offering

Now that we have the Service Request setup we can create a Service Request Offering to display the SR on the self-service portal

Setup your input prompts as normal

The OrchestratorRunbook Activity has 20 fields to map inputs to.

Use the Request Offering!

Now that the Request Offering is created you can do all of your normal activities, add it to a Service Offering, secure who can access it and initiate it!

Conclusion

So, what does all this mean? We have a new way to trigger Runbooks that is not tied to the ‘connector’ framework and Runbook Activity Template model from Microsoft.

• This gives us a more flexible solution for environments with Multiple Orchestrator and / or Service Manager environments
  • All that is stored in these classes is strings, not references to connectors which are Service Manager and Orchestrator environment specific).
• We can reduce the number of steps to create a Request offering down to two

1. Create Service Request
   1. add and configure OrchestratorRunbook activity
2. Create Request Offering

• If we need to change what environment a Runbook is triggered from we don’t have to re-create anything. Just find the Service Request that references it, open up the OrchestratorRunbook Activity instance and change the web server URL
• We can utilize ‘Cool’ functions inside of Service Manager, like Copy Request Offering to make QA / Production versions of our Request Offerings
  • This creates a new Request Offering and New Service Request
    • We can point the new Service Request to a QA / Production Orchestrator environment by just switching the web service URL
    • Easy way to create ‘dev / qa / production’ versions of everything from the UI (Service Manager Request Offerings) to tie to our Orchestrator Tiers

    

    

    

Setup

So, how do you go about getting all of this setup? The management pack / InitiateRunbook.dll file is available on the Orchestrator public integration pack codeplex site at https://scorch.codeplex.com/releases/view/104608 — as with everything this is a use at your own risk solution, try it in a development / test environment first.

Step 1: Register MP

• Just like normal register the integration pack into your environment, both the sealed and unsealed versions are available on the codeplex site
  • Register a sealed version so you can reference it from unsealed management packs

Step 2: Deploy the InitiateRunbook.dll

• Since this MP relies on a workflow there is a .dll file that needs to be deployed to your service manager management server.
• Stop your Management Service

  

• Copy the .dll file out to the Service Manager installation folder

  

• Start your Management Service

  

Step 3: Deploy the scorchLauncher Utility

Copy the scorchLauncher utility and its dependencies to the path that you will reference with your OrchestratorRunbook template. By default this is c:\Orchestrator\ScorchLauncher.exe on your Service Manager Management server

• scorchLauncher and its dependencies are available at https://scorch.codeplex.com/releases/view/102637

  

Step 4: Enjoy and Give Feedback

Feedback can be given on the http://scorch.codeplex.com project!