Update 1909 for Cloud Platform System (CPS) Standard Dell Hybrid Cloud System for Microsoft Dell Engineering January 2020
Revisions Date July 2016 August 2016 August 2016 October 2016 November 2016 December 2016 January 2017 February 2017 March 2017 May 2017 May 2017 June 2017 August 2017 September 2017 October 2017 November 2017 January 2018 January 2018 March 2018 April 2018 May 2018 June 2018 July 2018 August 2018 September 2018 October 2018 November 2018 December 2018 January 2019 March 2019 April 2019 April 2019 June 2019 January 2020 Description Initial release 1605 Release 1606 Release 1607 Release 1608 Release 1609 Re
Dell Hybrid Cloud System for Microsoft Cloud Platform System Standard
Table of contents Revisions ............................................................................................................................................................................. 2 1 Overview of the Patch and Update framework ............................................................................................................. 6 2 Update 1909—Summary ..............................................................................................................................
WARNING: You cannot run the 1909 Patch & Update framework—1.5.14—directly without first upgrading your environment to 1803 Patch & Update framework—1.5. You can directly upgrade to 1909 only after the DHCS stamp is at the 1.5 version, P&U 1803. Also be advised that the addition of any non-DHCS hardware to your system will cause the Patch & Update process to fail.
1 Overview of the Patch and Update framework The Dell Hybrid Cloud System for Microsoft includes the Patch and Update (P&U) framework. This framework enables you to easily update the infrastructure components of the Dell Hybrid Cloud System for Microsoft stamp with minimal or no disruption to tenant workloads. The framework automates the installation of software, driver, and firmware updates on the physical hosts and the infrastructure VMs. Note: The P&U framework does not update tenant VMs.
2 Update 1909—Summary Update 1909 for CPS Standard includes updates for Windows Server. This update includes the following components: 1909 update. This is the main package. It can contain Windows Server, System Center, and SQL Server updates. (See payload details in Chapter 5.) IMPORTANT: Update 1803 is a prerequisite for installing update 1909. IMPORTANT: Update 1812 contains a security related WAP update (See CVE-2018-8652).
2.2 How to check which update package is installed To check the version of the update package that is currently installed on the stamp, do the following: 1. On the Console VM, open the DeploymentManifest.xml file at the path: C:\Program Files\Microsoft Cloud Solutions\DeployDriver\Manifests. 2. At the top of the file, look for the following entries: • • “Version=”: This is the version of the Dell-provided update package.
3 1909 Patch and Update Prerequisites You must do the following in order to run the P&U successfully. 3.1 Prepare the patching environment You must first prepare the environment.
3.5 Step 4: Ensure that LaJollaDeploymentService is not running in the background on the Console VM You can ensure that the service LaJollaDeploymentService is stopped by doing the following: 1. On the Console VM, open up the services MMC console that is located under Control Panel>System and Security->Administrative Tools->Services. 2. Look for LaJollaDeploymentService. 3. Ensure that Status is Stopped. 3.
3.7 Step 6: (Optional): Exclude external SOFS storage clusters from P&U IMPORTANT: This procedure applies only if you attached external Scale-Out File Server (SOFS) storage clusters to the CPS Standard stamp. If you attached external Scale-Out-File-Server (SOFS) storage clusters to the CPS Standard stamp (for additional workload capacity), you must exclude them from P&U. If you do not, P&U will fail. To exclude external storage clusters, do the following: 1. Open the VMM console. 2.
4 1909 Patch and Update Process IMPORTANT: Be sure to follow the prerequisites listed in the previous section before you run the 1909 Patch and Update process. You must first prepare the environment. This section covers the preparation steps. In the "Update the computers" section of the CPS Standard Administrators Guide, complete "Step 1: Restart the Console VM" and "Step 2: Run a health check and fix any discovered issues." This includes functionality to check for and disable any running backup jobs.
\\CON01\PUShare\PU_DellEMC1909\PU\Framework\PatchingUpgrade\InvokePURun .ps1 -PUCredential (Get-Credential) -ScomAlertAction "Continue" 4. When prompted, enter the account credentials of the account that you used to log into the ConsoleVM. 5. The Invoke-PURun script performs a one-time environment setup and may prompt you to restart Windows PowerShell on its for invocation, for example: PowerShell environment settings have changed. Please restart the PowerShell console before proceeding.
update for a while. You can use the following steps to view the progress of cluster updates in Failover Cluster Manager. i. Open Failover Cluster Manager. ii. Connect to the cluster. a. In the navigation pane, right-click Failover Cluster Manager, and then click Connect to Cluster. b. In the Select Cluster dialog box, click Browse. c. Click the desired cluster, and then click OK two times. iii.
4.2 Step 2: Upgrade the Intel NIC Driver IMPORTANT: You must manually upgrade the NIC driver on all your physical servers. Failure to do so could result in Blue Screen of Death Errors. 1. Extract the NIC driver from the zip package a. Unzip the NIC drivers from the file Network_Driver_M1P35_WN64_18.8.0_A00_01.zip which is located in the folder C:\PUShare\DellEMC1909\Payload\PatchingUpdates\DellUpdate\CAUHotfix_All\Binaries to the folder C:\PUShare\NIC 2.
d. Verify that the driver was successfully updated i. Go back to the PowerShell window and run the following commands: ii. Enter-PSSession C iii. Get-WmiObject Win32_PnPSignedDriver | Select-Object -Property devicename, driverversion | Where devicename -like 'Intel(R) Ethernet*' Note: The driver version should now be 3.14.78.0 e. Repeat steps a through d on every Compute cluster node in the stamp. 4. Upgrade the Intel NIC Driver on the File Servers a.
4.3 Step 3: Run the 1909 Microsoft P&U package (DHCS_Update_1909_Run_Second) IMPORTANT: You must run the DHCS_Update_1909_Run_First package before you run the 1909 Microsoft P&U package (DHCS_Update_1909_Run_Second). Because of the size of this package, estimates for deployment duration are 12 to 18 hours. Run the 1909 Microsoft P&U update package by doing the following: 1.
6. DPM agents on the DPM servers are in an enabled state. If this is the case, the health check output indicates that you must run the Set-DPMBackupMode script to cancel the jobs and disable the agents. The PowerShell output looks similar to the following screenshot: 7. To cancel the jobs and disable the agents, do the following: a. From an elevated Windows PowerShell session, run the following commands. Press Enter after each command: cd \\CON01\PUShare\\PU\Framework\PatchingUpg
In the ClusterName – Cluster-Aware Updating dialog box, click the Log of Updates in Progress tab to monitor what is happening. Note: After Cluster-Aware Updating (CAU) completes, you can click Generate a report on past Updating Runs to view details about what was installed through CAU. iv. • If you have the VMM console open, and it reconnects, patching of the VMM server may be in progress. This is expected behavior. 2.
4. If you disabled DPM agents on the DPM servers earlier, do the following to restart any canceled jobs and enable the DPM agents: a. On the Console VM, make sure that you are logged on as the account that is a member of -Setup-Admins. b. Open an elevated Windows PowerShell session, and run the following commands. Press Enter after each command. cd "\\CON01\PUShare\\PU\Framework\PatchingUpgrade" Import-Module .\PatchingUpgrade\DPM.
4.3.1 Run an optional compliance scan If you want to run a compliance scan, pass the following flag: \\SU1_InfrastructureShare1\Framework\PatchingUpgrade\InvokePURun.ps1 -PUCredential $cred -ComplianceScanOnly The compliance scan output is written to the following location, the place where the update package was extracted. For example, the following shows output written to: "PURoot"\MissingUpdates.
b. The updated MPs should show with the names of “Microsoft System Center Service Management Automation Library”, “Microsoft System Center Service Management Automation Dashboards”, and “Microsoft System Center 2012 R2 Service Management Automation”. The version should be 7.2.102.0 is they have been imported correctly. Post-update clean up After you have verified that patching has completed successfully, do the following to clean up the environment. 1.
IMPORTANT: We recommend that you leave the latest update package in the PUShare in case diagnostics or debugging is needed. Also, do not remove the artifacts that were created during patching; for example, the VMM artifacts such as custom resources, and any associated log files, Windows Installer packages (.msi files), or patch files (.msp files).
5 Microsoft payload for Update 1909 Payload for Update 1909 Update Details KB Number Title CVE / ADV 890830 Windows Malicious Software Removal Tool x64 - August 2019 N/A 4514604 2019-09 Security and Quality Rollup for .NET Framework 3.5, 4.5.2, 4.6, 4.6.1, 4.6.2, 4.7, 4.7.1, 4.7.2, 4.8 for Windows 8.
5.1 Troubleshooting the P&U process Issue 1 Symptoms: The P&U install process fails with an SMA MAX Timeout Error: Exception calling "InvokeRunbook" with "2" argument(s): "Max Timeout reached for SMA runbook 'Import-OmManagementPack'. P&U fails after a two-hour timeout waiting for the Runbook to complete. Description: SMA Service is hanging when processing runbooks for P&U, specifically the “Import-OmManagementPack” Runbook.
Issue 2 Symptoms: The P&U process updates the console, including reboots, but does not finish final P&U processing. Description: This can include examining the Deployment Manifest and running compliance checks. Detection: Run the following script (updating the $prefix variable before running with the prefix of your stamp) $prefix = "" (Get-SmaVariable -WebServiceEndpoint ("https://{0}APA01" -f $prefix) -Name PUSubsystemVersions).Value If any of the values for “MicrosoftVersion” are not "1.0.1909.
Detection: 1. Open Failover Cluster Manager. 2. Right-click on the CCL cluster and choose More Actions, and then choose Cluster-Aware Updating. 3. Once the Cluster-Aware Updating dialog opens, select Analyze cluster updating readiness.
4. The analyzer runs for a minute or two, and then shows you the results, as illustrated by the following graphic: Under the Title “A firewall rule that allows remote shutdown should be enabled on each node in the failover cluster” you should see a green ‘Passed’ result. If there are any compute nodes that are members of this CCL cluster listed as having failed this test, you have run into this issue. Resolution: Reboot the affected nodes.
Resolution: Browse to the location where the Dell EMC Patch and Update package has been extracted. Under C:\PUShare\PU_DellEMC1909\ Subsystems\PU, you can find the Test-PUHealth.ps1 script, and in the following snippet, add the highlighted workaround: Write-HealthLog -TelemetryInfo $TelemetryInfo -EventType "Progress" -Message "Checking PU custom property for '$($server.ComputerName)'.
Issue 5 Symptoms: From the Console VM, the CPS Administrator cannot access the OEM OOB (Out-of-Band Management) webpage through Internet Explorer. The error will be similar to the following: This page can’t be displayed Turn on TLS 1.0, TLS 1.1, and TLS 1.2 in Advanced settings and try connecting to https://URL again. If this error persists, it is possible that this site uses an unsupported protocol or cipher site such as RC4 (link for details), which is not considered secure.
Cause: Microsoft Update KB# 4093492, that impacts CredSSP authentication protocol and RDP functions. All servers in a CPS environment are now forcing the registry key “HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System\CredSSP\Parameter s\AllowEncryptionOracle” to a value of “1”. Workaround #1: Open a Microsoft Management Console and add the Group Policy Editor Snap for the WSUS server.
6 Dell EMC Payload for Update 1909 Dell EMC Update 1909 for CPS Standard includes the following driver and firmware updates. WARNING: Once the Intel NIC firmware is updated to 18.8.x, do not downgrade to previous A-Rev versions below 18.8.x. • • • 32 Dell Server PowerEdge BIOS R630/R730/R730XD Version 2.10.
o • • 33 Fixes: - Added HTTPS support for Firmware Update feature o Enhancements: - Security/IPS fixes Dell PERC H330 Mini/Adapter RAID Controllers firmware version 25.5.6.
• Intel NIC Family Version 18.8.0 Firmware for I350, I354, X520, X540, and X550 adapters Fixes & Enhancements Note: Firmware downgrade from 18.8.x to 18.5.x or older versions is not supported. o • • 34 Fixes -Resolved an issue that may cause the link to stay down with both port LEDs off on Intel(R) 10G X520 LOM, Intel(R) 10G X520 rNDC and Intel(R) 10G 4P X520/I350 rNDC when firmware is upgraded to version 18.5.17.
