Active / Passive Production Environment
Purpose
- Running of production workflows
Environment Description
- Mirrored Multi Server Environment
- Scale Out Points
- Web Tier – Add additional Servers to this role (and to F5 VIP) if Web Service load becomes Heavy
- Action Server
- Monitor CPU and Memory consumption on Action Server
- Underutilization: Increase concurrent policy number using asp
- Overutilization: Decrease concurrent policy number using asp
- Monitor total environment current concurrent policy number
- Create new action servers when utilization demands it
- Should always have one action server worth of ‘free’ policy spots
Failover Steps
- Failover Web Service Calls
- Set Passive Node as Active in Production Pool
- Failover Monitor Workflows
- List active Runbooks with isMonitor Flag in Active Environment
- Start Mirror Policy in Passive Environment
- Stop Runbook in Active Environment
- Wait for all Ad-Hoc policies to stop on all actions servers
Policy Migration
- Runbooks will be sanitized before import
- Runbooks will be imported with the same folder structure in both environments to facilitate a ‘mirror policy’ relationship
- Runbooks will be imported into both environments at the same time
- No Runbook design will be done in central production
- Only Runbooks in this environment will be exports from Central QA
Reporting
- Runbook CI Hours will be tracked in external database
- This will be handled on a per-Workflow basis
Runbook Promotion
Workflow Properties
- Workflows will be self contained under a root folder
- Any reference global settings will be contained in a folder of the same name
- All folder names will be prefaced with the team name they are developed by
Promotion Phases
- Workflow is designed in Development
- Workflow is exported and sanitized
- Workflow is imported into QA Environment
- QA Is preformed on workflow for best practice analysis
- If this fails it is removed from QA and re-enters the development phase
- Workflow is run in QA environment and performance impact is evaluated
- If workflow is highly impactful and properly designed expand production action server count
- Workflow is exported from QA and sanitized
- Workflow is imported into TFS
- Workflow is imported into Active and Passive Environments
Production Environment Integration Pack Deployment
- Deploy integration pack to passive environment
- Fail Active environment over to Passive Environment
- Test Workflows
- If failure rollback to active, remediate
- If successful deploy integration pack to new passive environment
