By James Reynolds Disclaimer

VMware ESXi By James Reynolds Disclaimer

• I'm new to this stuﬀ • I'm still new to the jargon • I'm pretty inexperienced • This presentation barely scratches the surface • I'm assuming you've never done this before • I'm focusing on Mac OS and VMware • This is non-critical servers Agenda

• Why • vSphere, ESXi, and VMware History • Hardware support and licensing • Installing and conﬁguring ESXi • Datastores and creating VM's • Backup Why?

Prepare for changes to macOS Server - January 2018 https://support.apple.com/en-us/HT208312

"In fall 2018, Apple will stop bundling open source services such as Calendar Server, Contacts Server, the Mail Server, DNS, DHCP, VPN Server, and Websites with macOS Server. Customers can get these same services directly from opensource providers."

In other words, the open source server software that Apple distributes is really old, and it's really easy to get it on your own, so Apple isn't going to distribute it anymore. The writing has been on the wall since 2011

• Xserve, 2002-2011 • Mac OS X Server OS, 1999-2011 • OS X Server App, 2011-2018? Advantages

• Run 10.5 - 10.14 • Run old apps • Run any virtually any OS, Haiku OS (OpenBeOS), React OS (Win XP clone)

• Apple says to use open source software to replace OS X Server • This software is easy on Linux • All of the advantages of VMs • Snapshots, consolidated hardware, easier backups, easier to move to new computers, etc Remote Only

• This isn't a replacement for dual booting • This is not Fusion, VirtualBox, Parallels, Wine, etc • Remote only • This is a replacement for servers in closets that are accessed remotely

• None of these VM's can be interacted with by plugging in a keyboard, mouse, and display into the computer Mac, Linux, Windows ESXi - The 4th OS? That rules them all? Jargon

• There are lots of VMware ﬂash cards with 100's of terms • I've been doing IT for almost 20 years and I haven't heard of half of these terms

• I'll avoid most of them in this presentation What is vSphere?

• Umbrella term for the package that includes • ESXi • vCenter Server • vCenter Client (moving to a web client) • VMFS vCenter Server and Client

• vCenter Server • Windows installer • Linux appliance (VCSA) • vCenter Client • "vSphere Web Client (FLEX)" moving to "vSphere Client (HTML)"

• Centralized Management of a ﬂeet of hosts What is ESXi?

• ESXi = Elastic Sky X (integrated) • Type 1 hypervisor, "bare metal" • VMware Fusion and Workstation are type 2 hypervisors What is ESXi?

• ESX (not available anymore) • Red Hat • After booting control passed to VMware What is ESXi?

• ESXi • Tiny footprint compared to ESX • Can be placed on 16 GB ﬂash media (takes 4 GB) • 10.3 was 1.5 GB, 10.4 was 3 GB, 10.6 was 5 GB • ESXi doesn't need to be rebooted or patched that often • ESXi boots much faster than ESX did What is ESXi?

• "ESXi is not Unix" • Similar but diﬀerent enough to not qualify as "Unix" • Replaced Linux kernel with "POSIX-like" VMkernel • Has Linux-compatible storage and networking drivers • BusyBox, other opensource software (ﬁrewall, ssh) • Software Freedom Conservancy is suing VMware for GPL violations

VMware's Architecture VMware History

• 1999 - VMware Workstation released • 2001 - VMware ESX Server • 2004 - EMC acquires VMware • 2007 - Mac OS X 10.5 Server allows virtualization • 2007 - VMware Fusion for Mac OS • 2008 - ESX free version VMware History

• 2010 - VMware vSphere 4.1 (ESXi released) • 2011 - vSphere 5.0 • 2012 - vSphere 5.1 • 2013 - vSphere 5.5 • 2014 - vSAN • 2015 - vSphere 6.0 VMware History

• 2016 - Dell acquires EMC • 800 US Workstation & Fusion developers ﬁred • 2016 - vSphere 6.5 • August 2018 - 15th VMworld conference held in Las Vegas, 23,000+ attendees

• 2018 - vSphere 6.7 Dell

• In 2016 Dell bought EMC for $67b to become Dell EMC • Largest tech merger & largest private tech company • VMware is the crown jewel, but is public & independent • Dell EMC owns 80% (or something) of VMware • In January 2018 rumor was VMware would buy Dell • But in July 2018 Dell bought DVMT, went "sort of" public, and gained more control of VMware Dell PowerEdge

• Dell's server line • #1 server at 19% - 20% marketshare • HPE closely behind (by decimals) • Starting from the $1,200's (upper limit? $100,000's?...) Mac OS VM's on Dell?

• Apple's license does not allow it • VMware will only run Mac OS VM's on Apple hardware Mac OS VM's on Dell on Macs?

• What if Mac OS VM's are *literally* (instead of virtually) on Apple hardware? Ofﬁcially Supported Mac Hardware

• Xserve3,1 (Early 2009) - ESXi 5.0

MacPro5,1 (2010-2012) - ESXi 6.0 • MacPro5,1 6.0 • MacPro6,1 (Late 2013, the trash can) - ESXi 6.7 • 1 Ivy Bridge CPU; 4, 6, 8 or 12 Cores • 64 GB (128 GB via 3rd party) Max RAM • $3000 - $7000 retail MacPro6,1 6.7 Unsupported Hardware

• ESXi will not run at all on iMacs or laptops • Mac Mini unoﬃcial support • They are not in the Hardware Compatibility Guide (HCL) • Not all versions of ESXi work • New Mac Mini's will lack support • It will take a year or so to add support in ESXi Mac Mini's & ESXi 6.7

• Macmini6,2 (Late 2012) • Dual Core i5 2.5 GHz or Quad Core i7 2.3 or 2.6 GHz • Max 16 GB RAM (upgradable) • Macmini7,1 (Late 2014, the 4 year old current model) • Dual Core i5 2.6 or 2.8 GHz or Core i7 3 GHz • 8 GB or 16 GB RAM (not-upgradable) Macmini7,1 Mac Mini Pros & Cons

• Pros • CHEAP • Cons • Limited to internal drives (more on this later) • Internal hard drives are a pain to swap (mod the case?) • ESXi support is not guaranteed (not all versions work) • New Mac Mini soon? How long before it is supported? What I'm using

• Macmini6,2 (Late 2012) • Quad Core i7 2.6 GHz • Max 16 GB RAM • 1 TB SSD • I don't run more than 4 Mac OS VM's at the same time • Most VM's are test images VMware Licensing

• Confusing • USHE has an agreement until 12/28/2021 • Email OSL to get a license

• vSphere licensing is per CPU (not core) • vCenter licensing is per vCenter server instance • I'm pretty sure nobody here needs more than 1 • vSAN not covered by campus agreement vSphere Licensing

• ESXi Free ("you should never use for free") • No vCenter, max 2 CPUs, max 8 vCPUs, no support • Essentials or Essentials Plus, 3 hosts max

• Standard (part of campus agreement) • Enterprise Plus (part of campus agreement) • Operations Management Enterprise Plus (vRealize Operations) • Not part of the campus agreement vSphere Standard

• Data Protection • High Availability • vMotion • Storage API's for backup vSphere Enterprise Plus

• DRS (Distributed Resource Scheduler) • vSphere Distributed Switch • Host Proﬁles and Auto Deploy • Storage and Network I/O Control • VM encryption • vGPU Installing ESXi

• Create an account on my.vmware.com and download "VMware vSphere Hypervisor (ESXi ISO) image (Includes VMware Tools)"

• Use UNetbootin and burn that image to a USB flash disk • https://unetbootin.github.io • (Format flash as Fat32/MBR) • Boot the computer from USB disk and finish installation Embedded vs Installable

• ESXi has 7 partitions • ESXi Installable • All 7 partitions on the boot drive (only 2 are writable) • ESXi Embedded • USB ﬂash, SD card, PXE boot • 5 read-only partitions (unless there's a core dump) • Uses a RAM disk or your 1st datastore for logs Darwin Tools Image

• Go to my.vmware.com, "View & Download Products" • Find and download darwin.iso* (3 ﬁles) • Copy them to /usr/lib/vmware/isoimages • symlink to one of the partitions packages/ vmtoolsRepo/vmtools Direct Console UI

• Use to conﬁgure networking, passwords, low-level debug • Can't manage VM's ESXi Host Client

• Point a web browser at your host's IP

• This is not the vSphere web client

Conﬁguring ESXi

• Autostart • License • Services (ntpd, snmpd, ssh, syslog, X.Org server, etc) • Users & Groups • Datastores Datastores

• Where the VM images are stored • Datastores can be networked or local Networked Datastores

• Storage Area Network (SAN) • Network block level access • Fibre Channel or iSCSI • Network Attached Storage (NAS) • Network ﬁle level access • NFS • Not CIFS (SMB) Local Storage (LS)

• Internal drives

• Directly Attached Storage (DAS)

• SATA or SAS (Serial Attached SCSI)

• Shared SAS, (Dell VRTX)

• "Hyperconverged"

• E.g. datacore.com starts at ~$750/TB Mac Hardware & Datastores

• Internal drives: Yes • Mac Mini and the data doubler • MacPro5,1 has 4 bays! Except ESXi 6.0 only :( • NAS: Yes (more about this later) • iSCSI & Fibre Chanel requires PCIe slots for HBA's: No • USB: No (USB is passed through to VM's) • That leaves Thunderbolt to the rescue! Thunderbolt...

• Requires drivers... • ATTO's ThunderLink® - Thunderbolt Fibre Channel adapters are the only supported Thunderbolt product

• $1095-$3495 for Fibre Channel adapters Stuff others got to work with ESXi 6.0

• Old Promise STEX driver • scsi-stex-4.07.0000.75-1OEM.500.0.0.472560.x86_64 • Sonnet Thunderbolt PCIe Expansion Systems • Sata-xahci driver (adds PCI ID's to the generic ahci driver) • Areca SAS RAID driver • Laci Rugged Thunderbolt drives PSOD Local Datastore: VMFS

• Versions match the version of ESXi • Multiple servers can read/write to this disk • Can span multiple volumes • ESXi is the only OS that can read & write this format • 3rd party (read) FUSE drivers: ﬂuidOps, & glandium New Datastore

• Click "Storage" and "New datastore" • Create new VMFS datastore My Internal SSD Creating VM's

• Just like VMware Fusion or VirtualBox How to Install VM OS

• Non-Mac OS'es • Upload ISO's to datastore and set it to the DVD drive • Mac OS installer app... • Convert to bootable installers • Use VMware Fusion and upload the VM • Create an image with AutoDMG and Netboot the VM Other Mac OS Things

• Copying VM's • Change uuid.bios • Display size • ESXi Web Console • /L/AS/VMware Tools/vmware-resolutionSet 1920 1080 • github.com/MarLoe/VMware.PreferencePane Sound Driver

• Mac OS VM's need sound driver added if you use `say` Linux Appliances

• Import free virtual appliances (OVF/OVA format) • Bitnami • Wordpress, Joomla, Redmine, Drupal, WAMP, ExternalDNS, Moodle, Magento, etcd, ownCloud, Kubewatch, LAMP, NATS, PrestaShop, GitLab CE, MediaWiki, Redis Sentinel, Ruby, phpBB, ELK, Django, Tomcat, Jenkins, phpMyAdmin, MODx, Roundcube, Node.js, Diaspora, Mantis, Hadoop, etc

• Turnkey Linux

Controlling VMs

• vCenter web console • ESXi Host web console • VMware control • VNC (ARD) • SSH • etc Transferring Files (VM's)

• Enable ssh on host and use scp • Just to the datastore • ESXi web client datastore browser • NFS datastore • VMware Fusion • VMware Fusion's ovftool • vSphere web client NFS 3 (Mac OS X)

• NFS 3, authenticates using IP, no encryption, requires root • Be sure to read "NFS Protocols and ESXI" at docs.vmware.com • Edit /etc/exports (BSD and Linux syntaxes diﬀer) • Each line is an export point followed by allowed clients

• /path [options] client1 [client2...]

/path -maproot=root:wheel 10.0.1.2 10.0.1.3 /path -maproot=root:wheel -network 10.0.0.1 -mask 255.255.255.0

• sudo nfsd enable ESXi Datastore Browser

• Upload • Download • Delete • Move • Copy VMware Fusion

• Connect to Server... • Download to Server... • Snapshots... • Upload to Server... • Show Windows • Settings vSphere Client Capabilities of Each Tool

VMware Fusion VMware Fusion

ESXi Host Client ESXi Host Client

vCenter Server Backup

• VMware vSphere Storage APIs (VADP) • Not part of ESXi free • Creates and backs up snapshots • Full backups • Change Block Tracking (CBT) • Keeps track of which blocks have changed • Incremental backups Backup

• Backing up the VM's from inside the guest OS • Amanda • Bacula • etc Backup

• Backing up the VM's from the host • Unitrends • Acronis • CloudBacko • Thinware vBackup • VSquare Backup

• Backing up the VM's from the host • Veeam • Free scripts that work with the free ESXi • XSIBackup • GhettoVCB Veeam Backup & Replication

• Free and paid version • Installs on Microsoft Server, requires SQL Server • File level backup and restore • Full VM backup and restore • Installs on vCenter XSIBackup

• Free and paid version • Large feature set • Very quick to set up • Installs on host • Requires a free license key • Requires basic shell skills ghettoVCB

• Free • Large feature set and mature • Easy to install (it's a VIB) • Installs on host • Good documentation • Deployment is not self-explaining • Requires basic shell skills Troubleshooting

• VMFS • Mac Mini internal hard drives • Can't erase disk, because I formatted it as GPT? • vim-cmd hostsvc/maintenance_mode_enter • esxcli • esxtop (TERM=xterm before ssh'ing) What about Hyper-V?

• Microsoft Hyper-V does not run Mac OS • If you don't need Mac OS • The free version is highly recommended Things I didn't talk about

• Auto deploying and conﬁguring hosts • Auto deploying and conﬁguring VMs • Docker, Vagrant, Salt stack, etc • Redundancy and zero-downtime • Data de-duplication • Virtual Networking Review

• Apple is exiting the server market completely

• vSphere is the umbrella term, ESXi is the host OS, vCenter Server manages multiple hosts

• Datastores: SAN, NAS, SAS, LS, DAS, Converged, Hyperconverged; Mac datastores are limited

• Creating VM's is just like in Fusion • Backing up can cost or is free • I've barely scratched the surface Links

• communities.vmware.com • virtuallyghetto.com • labs.hol.vmware.com • www.v-front.de • community.spiceworks.com/virtualization/vmware • reddit.com/r/vmware • esxi at macadmins.slack.com Questions