APPENDIX A Troubleshooting and Cyber-Reference Security Center Logs and Cybersecurity Vocabulary Azure Security Center Diagnostics Troubleshooting This appendix contains tips for troubleshooting Security Center that you may need during configuration and when trying to understand how data flows from the endpoint protection–extended client to Security Center. This appendix was written in response to technical questions that I have gotten in the course of my work; it helps security analysts understand how the communications and log data are handled in Security Center. Finally, cloud administrators in businesses large or small that are using Azure Security Center can gain insight into how to collect and share cybersecurity information. You should begin by searching for data in the Azure Audit Logs view after logging into the Azure Resource Manager (ARM) portal. When you installed and configured Security Center, you configured the diagnostic log that collects information separately from the virtual machines (VMs) and other platform as a service (PaaS) services. The Azure portal provides a Monitor journey with information collected for more than Azure Security Center. Review the information in Figure A-1 to gain a better understanding of the Azure Monitoring view. © Marshall Copeland 2017 165 M. Copeland, Cyber Security on Azure, DOI 10.1007/978-1-4842-2740-4 APPENDIX A ■ TROUBLESHOOtING aND CYBER-REFERENcE Figure A-1. Azure “Diagnostics logs” collection portal service The log files for Windows Server contain records of information specifically used to write events on the services provided by the operating system (OS). The security log in Windows Server provides details for the Local Security Authority Subsystem Service (LSASS.exe), and this log is used to investigate any attempt to gain access by unauthorized intruders. If you are taking advantage of the Azure infrastructure as a service (IaaS), the specific logs for the Windows OS VM include the following: • Windows Log • Application • Security • Setup • System • ForwardedEvents • Applications and Services • Admin • Operation • Analytic • Debug 166 APPENDIX A ■ TROUBLESHOOtING aND CYBER-REFERENcE Now that a VM is a supported OS type (in the Azure Marketplace) running in Azure, additional logs have been added to the OS. These logs can also be used in collecting and identifying health issues beyond the standard OS logs. Some of the Azure logs include Azure platform logs, which includes logs that might be related to Azure guest agents, the Azure extensions, and other Azure components. ■■Note You can find Azure log details at https://azure.microsoft.com/en-us/blog/simplifying- virtual-machine-troubleshooting-using-azure-log-collector, and you can find the Azure VM extension details at https://docs.microsoft.com/en-us/azure/virtual-machines/windows/log-collector- extension. The Event-Logging service controls tracked events in the OS, and when the service starts, you can track activities such as user actions and system resource events. The generic data that is provided includes auditing data in categories by event audit, including the following: • Information: Events that are informational, not a toggle between success and failure • Success Audit: Events related to successful action • Failure Audit: Failed execution of an action • Warning: Details useful in future system actions • Error: Failure for a specific action When you look at log information, there is specific data that log files can provide including the following: • Source: What application or service logged the action • Category: Future description related to the action • Event: Identifies the specific action • User: User account that was logged during the event action • Computer: Computer name during the event action • Description: Details of the event action • Data: Any data or error codes output by the event action You now have a good understanding of the type of data provided in the Window OS event logs and the extended logs from the Microsoft Azure event logs. The logs and data can be searched for the action, who enabled it, when the action accrued, and other statuses or values of the event to help with troubleshooting. In this book, you were introduced to three Azure Security Center agents that are installed on VMs, and these services all provide log file data. You should enable the monitoring of the agents’ data collection when you configure the diagnostic service. The three services are as follows: • ASMAgentLauncher.exe: Azure Monitoring Agent • ASMMonitoringAgent.exe: Azure Security Monitoring extension • ASMSoftwareScanner.exe: Azure Scan Manager The extensions are needed to integrate the Azure-installed endpoint protection agent and to complete the automated removal of cybersecurity threats. If open a Task Manager or Sysinternals Process Explorer, you will see something similar to Figure A-2. 167 APPENDIX A ■ TROUBLESHOOtING aND CYBER-REFERENcE Figure A-2. ASM agents installed by Azure Security Center SYSINTERNALS FROM A WEB BROWSER Often IT professionals must configure tools to run on systems and provide details; for example, you might need a tool to review log files and run processes in memory and CPU utilization. However, installing the product is a post-configuration build step that is sometimes missed. Rather than install the Sysinternals tools on each server, with the proper firewall security in place, you can use the entire toolset available at https://live.sysinternals.com, as shown in Figure A-3. These Sysinternals tools, with proper administrative permissions, run on any server or workstation with a web browser. 168 APPENDIX A ■ TROUBLESHOOtING aND CYBER-REFERENcE Figure A-3. View through a browser of https://live.sysinternals.com 169 APPENDIX A ■ TROUBLESHOOtING aND CYBER-REFERENcE If you scroll down to the Process Explorer and click the text to start the executable, you may see a browser pop-up window to allow the processes, as shown in Figure A-4. Figure A-4. IE browser pop-up to run or save the Sysinternals procexp.exe program Click Run; the installation starts and requires permission to continue, as shown in Figure A-5. Figure A-5. Sysinternals license terms 170 APPENDIX A ■ TROUBLESHOOtING aND CYBER-REFERENcE Click Agree, and the Process Explorer program starts to run. Process Explorer is a feature-rich view, more than the built-in Windows Task Manager, as shown in Figure A-6. Figure A-6. Process Explorer running after selecting the procexp.exe from the web browser The Azure Security Monitoring extension does the scan for event information, and each agent has a different responsibility, such as patch scanning. You can look at the log file information on the Azure VM in the installation directory’s agent log folder. • %systemdrive%\windowsazure\logs (i.e., C:\WindowsAzure\Logs). One of the “unwritten” rules of cybersecurity analysis is to understand, based on log file data, the amount of data and correlation of events to classify work as “normal.” If log file information is not updating as expected or is not what you would normally expect to see from the agent log files, you need to restart the VM because there is currently no PowerShell command to stop or restart the agent. After the system reboots, you can remove the agent and then reinstall it to validate that any issues were resolved. Typically, on large installations, identifying a single Security Center agent installation that is incomplete is common. To uninstall the agent and then automatically reinstall it, follow these steps: 1. From the Azure portal, click the virtual machine. 2. Right-click Microsoft Azure Security Monitoring and click Uninstall. The Security Monitoring extension will automatically reinstall (it takes about ten minutes). The Security Center agent installation in Linux requires a different but similar log file location. The following are the log file locations: • /var/log/mdsd.err • /var/log/azure/ On a Linux system, the extension is downloaded to the following location: /var/lib/waagent/. Then you need to run the command cat /var/log/waagent.log. 171 APPENDIX A ■ TROUBLESHOOtING aND CYBER-REFERENcE Linux systems have daemons (similar to Windows OS services) and require listening via connections for specific ports based on the TCP or UDP requirements. The Linux agent communicates on port 29130 using TCP. From a command prompt, run the following: netstat -plantu | grep 29130 General networking troubleshooting processes may include the endpoint protection of the anti- malware extension on the Azure agent. ■■Security Tip This tip is important to repeat. The System Center Configuration Manager agent installation is not compatible with Azure Security Center because it does not, currently, have the needed extensions. Even if you plan to support all Azure VMs through the Configuration Manager console, the integration with Azure Security Center must be installed from the Azure System Center portal. The following are additional key points for troubleshooting: • Remove and reinstall the agent: uploaded, custom image, and new installs. • The Linux agent is different from the Windows agent. • New agents update automatically. Older versions do not update correctly; you must reinstall them. • Validate that the agent is running (Figure A-2); some software may disable the agent. • A change request process may help identify when incorrect settings for an Azure network security group (NSG) were changed. Block network traffic to and from the guest agent. There are other Azure-specific sites that you can review, including the Windows Azure – Troubleshooting & Debugging site. (The current site refers to Windows Azure still.) You can find it at https://blogs.msdn.microsoft.com/kwill/, as shown in Figure A-7. 172 APPENDIX A ■ TROUBLESHOOtING aND CYBER-REFERENcE Figure A-7. Azure troubleshooting blog Another location for helpful information on all Azure cloud subjects is the Microsoft Azure forum at https://social.msdn.microsoft.com/forums/en-US/home?category=windowsazureplatform, azuremarketplace,windowsazureplatformctp, as shown in Figure A-8. The top-right search option allows you to search for Security Center topics. 173 APPENDIX A ■ TROUBLESHOOtING aND CYBER-REFERENcE Figure A-8.