Solaris

Why to use Solaris10+є (e.g. for AFS file server)

September 2009

Mathias Feiler Universität Hohenheim

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 1 Speed up

For faster walk through

You may want to

follow the (approx. 35) ­tagged slices only

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 2 Why to use Solaris 10+є ?

● Solaris on Sparc is rather solid – We don't use to boot each year – HW and software provided by same company ● Development can stay more focused in limited HW options ● Iimho. the most innovative U*.*X at present – Zfs (the only contemporary disk file system) – Zones (a very effective way of virtualization) – SMF (faster boot , better normalized management) – Some more I don't use (too much) ● FMA, Dtrace, Role Based Access(RBAC), ...

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 3 Solaris

>>> SMF

ZFS

Zones

Solaris + AFS

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 4 Known Issues

● Service dependency is hell. Isn't it ? – Not solved by RC­order numbers:/etc/rc?.d/SNNxxx ● Works in serial and thus rather slow ● No real knowledge of dependencies – just “sooner or later” ● Is it possible to unify management somewhat? ● w/o fiddling the link names and rc­files ● What if a service dies in middle of day? ● Restart service? Reboot system? – Initd cares half­hearted , but rc does not at all ● What to do with dependent tasks ? – Guess why AFS­BOS was introduced ● Can't a machine come up faster?

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 5 What is SMF

● Service Management Facility – Part of “Predictive Self­Healing” ● “Fault Management Architecture” (FMA) is an other one. – A method to describe, handle and monitor a service – Much more powerful than inittab, runlevel and the /etc/rc?.d mechanism – LiteSQL database aided

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 6 What is SMF doing

● Outperforms / replaces runlevel & /etc/rc?.d ● Populates database from file (profile & manifest) ● Keep track of maintained services (Nanny) ● Provides helpful assistance for management ● Works in parallel as far as possible

Think of a booting machine as defective since expected functionality is not given. A healing mechanism (smf) detects the issue and tries to repair it honouring dependencies.

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 7 SMF first touch

● /etc/rc?.d is alarming empty ● Did the installation really succeed ? ● No /etc/rc3.d/S89sshd ­ but its running anyway – Who started it – and how – and why ?

● /etc/inittab is incredibly melt down ● Where had all the runlevel been gone to?

● I kill a daemon but something strikes back ● It is restarted immediately

● How can I stop this dirty service I don't like ? ● except of shut down the machine

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 8 My first expirience with SMF

● Remote (SSH) tuning of /etc/ssh/sshd_config > # Take care ! Don't saw the limb you are sitting on! > ps ­ef | grep "[Rr]oot.* 1 .*sshd" | cut ­c9­15 123 > kill 123 > /usr/lib/ssh/sshd ; tail /var/adm/messages ... sshd 2345 ... error: Bind to port 22 on :: failed: ... > # Hell, it won't die – try harder > kill ­9 123 kill: (123) ­ No such process > # ??! ­ Ok, obviously meanwhile it died > /usr/lib/ssh/sshd ; tail /var/adm/messages ... sshd 2356 ... error: Bind to port 22 on :: failed: ... > # ? ... ??? ... Something is bloody strange .. ??? > ps ­ef | grep "[Rr]oot.* 1 .*sshd" Root 2340 1 0 13:45:16 ? 0:00 /usr/lib/ssh/sshd

● HEY! Some one is doing my job w/o asking me – and even faster

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 9 SMF components

● svc.startd – The SMF core started by initd ● svc.configd – database­interface & management ● LiteSQL – Database to handle all the knowledge ● Manifest – A full service description (XML) ● Also contains default properties and values, methods,... ● Method file – similar to the old RC­file ● Program (or scripts) to start / restart / stop services ● Use of contract­fs ● A kernel fs for maintaining process­”families” ● Administrative commands (see below)

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 10 SMF terminology

● FMRI ● The URI of SMF & FMA (see below) ● Service ● A big polymorphic object SMF is working with ● Milestone ● Used in a similar way like the old runlevel : “name­service:default” , “devices:default” , “single­user:default” , “multi­user:default” , “multi­user­server:default” ● Properties ● Initially created on behave of manifest ● Stored in DB, can hold all sort of adjustments ●

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 11 How SMF comes in?

Kernel initd inittab LiteSQL SMF configuration Contract­fs repository svc.startd(1M) svc.confd

svc:/application/print/server

delegated restarter delegated restarter svc:/system/cron ... svc:/network/inetd svc:/network/openafs/server:bos

svc:network/ftp svc:/network/login:rlogin ...

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 12 FMRI ­ the naming scheme

● Fault Management Resource Identifier ● URI used by “Predictive Self­healing” ● Which SMF is part of ● Describes all sort of system objects ● services (svc) , devices(dev) , packages(pkg), memory(mem) , diagnostic engine(fmd), .... ● In general: fmri­scheme://[authority]/path ● Example of a SMF­FMRI: svc:/network/ssh:default usually parts of FMRI can be omitted e.g.: network/ssh

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 13 What is a service in terms of SMF?

From Sun admin sudent guide (SA­225­S10) :

... “A service is an entity which provides a known list of capabilities to other local and remote services. Services are represented as instance nodes which are children of service nodes ... One service might have many instances. ...Both...can have properties...“

In my words: A abstract bunch of all the parts you need to describe + maintain + yield function.

Other, more formal, definition are available on the net

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 14 SMF­administrativ commands > svcs – Monitoring of SMF ● Dependence , dependencies , problems , further hints > svcadm ● Switch on/off service , set tgt­milestone (confer runlevel) > svccfg ● Database­ and file­ centric management , properties, manifest,... > svcprop ● Print properties e.g. used by scripts to learn options > inetadm ● Management of the delegated restareter “inetd”

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 15 > svcs

> svcs ­a STATE STIME FMRI legacy_run Sep_14 lrc:/etc/rcS_d/S29wrsmcfg legacy_run Sep_14 lrc:/etc/rc2_d/S10lu ... disabled Sep_14 svc:/network/iscsi_initiator:default disabled Sep_14 svc:/network/smtp:sendmail ... online Sep_14 svc:/milestone/multi­user­server:default online Sep_14 svc:/network/openafs/client:default ... offline 21:21:03 svc:/application/print/rfc1179:default

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 16 > svcadm + svcs

> svcadm enable svc:/application/print/rfc1179:default > svcs ­xv svc:/application/print/server:default (LP print server) State: disabled since 14 September 2009 11:13:34 CEST Reason: Disabled by an administrator. See: http://sun.com/msg/SMF­8000­05 See: man ­M /usr/share/man ­s 1M lpsched Impact: 1 dependent service is not running: svc:/application/print/rfc1179:default > svcs ­l svc:/application/print/rfc1179:default fmri svc:/application/print/rfc1179:default name BSD print protocol adapter enabled true state offline next_state none state_time 24 September 2009 21:21:03 CEST restarter svc:/network/inetd:default dependency require_all/refresh svc:/application/print/server (disabled) > svcadm disable svc:/application/print/rfc1179:default > svcs ­x

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 17 > svcs dependencies

> # the ­D shows what services or milestone bank on this on > svcs ­D svc:/application/print/server:default STATE STIME FMRI disabled Sep_14 svc:/application/print/ipp­listener:default online Sep_14 svc:/milestone/multi­user:default offline 21:21:03 svc:/application/print/rfc1179:default

> # the ­d shows what is needed to run this service > svcs ­d svc:/application/print/server:default STATE STIME FMRI online Sep_14 svc:/system/filesystem/usr:default online Sep_14 svc:/system/identity:domain online Sep_14 svc:/system/filesystem/local:default online Sep_14 svc:/system/system­log:default

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 18 > svcprop

>svcprop ­p config svc:/network/openafs/client:default config/ExtraAfsdOpts astring "" config/LegacyCfg astring "" config/Notice astring V0.1/Feiler config/SizeModel integer 3 config/afsdb boolean true config/backuptree boolean true config/blocks astring "" config/cachedir astring "" config/confdir astring "" config/dynroot boolean false config/fakestat boolean true config/fakestat­all boolean false config/memcache boolean false config/mountdir astring "" config/rootvol astring root.afs.readonly config/ThisCell astring uni­hohenheim.de ....

> svcprop ­p config/ThisCell openafs/client uni­hohenheim.de

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 19 Modify properties

– Set a singe value > svccfg -s openafs/client setprop config/SizeModel = 3 – Delete a property­value > svccfg -s openafs/client delpropval config/LegacyCfg \*

– Delete a property > svccfg -s openafs/client delprop config/LegacyCfg – Stdout of methode shall make it to the log file > svccfg -s openafs/client set -v – Force changes to become effective > svcadm -v refresh openafs/client

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 20 SMF:Manifest Example 1

. ­­>

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 21 SMF:Manifest Example 2

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 22 SMF:Manifest Example 3

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 23 You might think now..

● Why for heaven's sake a “registry” for Solaris ? How else to keep track of and manage dependencies? ● Why on earth XML ? It is oblong­ yes, but it has also DTD's1) which helps a lot ● Cant it be made more easy, native , evident ? SMF is just a part of a hole structure design. You surely want to have a (and keep) a unified way to talk with it. ● How to switch off the beast ? Recreate the RC­files, refill inittab , disable most of the smf­ services , recompile.... – stop, no, don't do it !

1) DTD = document type definition = what is where allowed/required

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 24 Why not to switch off SMF?

● Automated fault detection and repair – usually w/o boring You. – Even dependent services are considered. ● All get logged but does not flood screen ● boot works in parallel most of the way ● Switching on/off a service persistently is easy svcadm disable ssh ; svcadm enable ssh ● It is aware of Solaris RBAC. – SMF­tasks can be delegated ● Configuration roll back possible (DB­snapshot) ● ....

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 25 Solaris

SMF

>>> ZFS

Zones

Solaris + AFS

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 26 What is ZFS

● Zeta File system (not “Zetta”=10²¹ or Zebi=270

● New file system implementation of Sun – Full dynamical file system – Reminds to MMU in terms of RAM

● Like file system + volume managaer – e.g UFS + SVM or VxFS + VxVM ... – However much better than that

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 27 ZFS big picture

Operating system / Application

7 ZFS­datasets Zfs aZfs Zfs bZfs Zfs1Zfs Zfs2Zfs (~Filesystems) Zfs cZfs Zfs XZfs Zfs3Zfs

2 Zpools

2.3t 1t 1t 0.3t 4 Block­ 2t USB 2 striped disks storage­ Mirrored with a partition devices

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 28 Layer overview

Taken from I don't know where on the Internet ­ seams to be property of Sun microsystems

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 29 Problems of common file systems

● Data corruption is always possible – Journalling helps much , but by far not all ● Often does not cover payload data blocks ● Focus poor problems: power outage , system crash – Reread is important (disk firmware?) but takes time – check sums are usually somewhat week ● Said to detect typically only 2⁸ ­1 of 2 ⁸ failure ● Covers data on disk but not data on the run ● Fsck – Assumes faults to occur on system start only – Usually not designed to work while in service – On larger FS it can take ages while not in service

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 30 Problems of common FS / .*VM

● No much optimization possible – Volume manager does not know data structure ● Trouble with write optimization and error detection ● Suitable compression is rather difficult – File system does not know about mirror ● Trouble with error correction using mirror ● (Brain based) human intervention required

● inconvenient & time consuming administration get PV's , build PVG's holding PE's of each PV create LV's by LE's using of PE's of different PVG's, get FS on LV­Partition, configure vfstab, tread boot parti. –­­ did I miss something of the nightmare?

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 31 Some ZFS design goals

● Administration – easy and evident ● No config file at all , only 2 commands ● Pooled storage ● File system “knows” of its multiple disks ● Hierarchical space distribution + inherited parameter ● Treat disk space dynamically, almost like RAM ● End to end data integrity ● Reasonable check sum for every block/file ● Automatically access mirror disk + self healing ● Transactions only ● No fsck at all , scrubbing is possible while on line ● Get better performance , less downtime

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 32 ZFS and the future

● The only 128 Bit file system I know – Max file system size 2¹²⁸ ( 2 ⁶⁴ by Posix API limit) ● UFS can do 2⁴⁴ ­ 2 ⁷³ Byte ( = 16 TiB­8ZiB )

● Easy Migration of ZFS – To different machines – To different architecture – NFS ready/compliant – ISCSI ready/prepared

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 33 ZFS data migration

● ZFS open eggs either way ¹⁾ ● ZFS interprets either endian (e.g. like TIFF) ● Writes in host byte order ● Thus you can cross mount w/o problems for ZFS ● ZFS stores all management info along with data ● Label , device path , mount point , NFS­share,... ● One central point to manage ZFS (in ZFS) ● ZFS converges even on change of disk device file – e.g. when moved to a different controler / system ● Disk­mig: zfs export , , zfs import

1) Jonatah Swift: Guliver's travel : Likiput vs. Blefuscu

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 34 ZFS and storage

● ZFS accepts (loves cheep) block­devices ● disk , partition , usb­stick , .... and files ● no partitioning needed when on hole disk ● You can set mirroring posthumous ● Dynamically spend datasets (ZFS) when needed – Think in terms of work flow and logical grouping ● Stop thinking in terms of bytes and partition – Much like AFS volumes ● No fragmentation, no garbage , no waste. ● Error handling works coordinated on all layers – Self healing is possible easily

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 35 Common Volume/FS Modell

1. Read fails 2a. Wrong meta data : 2a. Wrong user data : FS driver got troubled ! App got troubled ! Perform system crash ? Calculation reliable ? Elect George W. Bush? Application Application Application

FS FS FS

*xVM *xVM *xVM

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 36 ZFS and self healing

1. Read fails 2. ZFS reads mirror 3. App gets good data due to false detection by and CRC check sum test ZFS fixes bad blocks

Application Application Application

ZFS mirror ZFS mirror ZFS mirror

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 37 ZFS uses copy on write

● Basically well known from AFS ● ZFS introduces the “Uberblocks” ● 128 in cyclic use, 4 copies each, CRC and SN ensured ● Recall – Payload is written to free blocks on disk – Associated meta data is written to free blocks too ● Pointing to the newly written Payload – Now changes (transaction) get committed ● Writing new version of Uberblocks pointing at the new tree – The original data tree is still there. – Occasionally used by snapshots – Clean up if neither used nor referenced

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 38 ZFS and memory

● Per default ZFS takes all RAM but 1GB – Occasionally 1GB of RAM could be insufficient ● Virtual machines or zones (see prstat ­Z' ) ● Applications keen on memory ● Cache of other file systems ● Perhaps limit ZFS' s ARC – ARC = Adaptive Replacement Cache ● See : Solaris­internals Wiki – Such a limit can ● reduces swapping ● increase speed of other file systems ● Example: / etc/ system: set zfs:zfs_arc_max = 0x780000000

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 39 ZFS integrity

● Transactions ● Make use of “copy on write” + “uberblock” ● Implies no need for fsck ● Scrub can check & repair on line (using transactions) ● CRC check sum ● All data ● End to end – Not only while on disk bur also while on way ● Never trust the cat to keep the cream – Check sum is not held by the summed data itself – Check sum is held within the referring structure ● Raidz ● Raid w/o “write hole” ability ... by transactions

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 40 ZFS got also...

● Almost all works on line ● Compression on/of , Block size adjustment , stripe size adjustment , Storage modification , FS­resizing,... ● Modern I/O optimization ● Pipelined I/O , priority ● multi­user prefetch – something cool for streaming people ● Snapshot and cloning ● NT­ and NFSv4 ACLs ● Encryption (Solaris Security Framework) ● NFS and CIFS support ● Booting from ZFS is ok now (on either endian)

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 41 ZFS conclusion

● Easy ● Just does what admin expects to ● Powerful ● Incorporates most modern concepts ● Save and secure ● Detects and repairs silent data corruptions ● Fast ● Dynamic striping, less Disk overhead , ... ● Open ● No company can wash away (like Linux) ● For free

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 42 ZFS for short

● No limits ahead (128 bit) ● Eazy administration

● No endian issue ● Integrated volume manager

● Robust (self healing ) ● Dynamic storage distribution

● Quite fast ● Scores of ZFS in zero time

● Transactions ● No vfstab / dfstab

● CRC end to end ● Quota & Reservation

● No fsck ● Snapshot & cloning

● On line compression ● NT­ACLs

● On line tuning /configuation ● No solid locking (SAN)

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 43 Solaris

SMF

ZFS

>>> Zones

Solaris + AFS

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 44 Introduction : Solaris Zones

● There is no modern Solaris w/o a main zone – The “global” zone is always there ● The Solaris isolation primitive – Application execution environment – Isolated process hierarchy (process­tree) ● A Solaris way of virtualization (partitioning) – Change­root environment – Got its roots in BSD­Jail (a decade ago) ● Advanced and improved quite a lot since then

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 45 Solaris Zone flavours

● Four main flavour available – Global zone ● System management – Sparse zone ● shares as much as possible – Branded zone (whole root zone / brandz) ● shares as less as possible ● Can hold/run other OS as well ● Logically not really to be deemed special or unique – Container ● also adds resource management to the zone concept

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 46 Solaris Global Zone

● The primary zone of the host system ● formerly: ' the system ' ● Hardware boots into the global zone ● There is exact one global zone per phy.Host ● Always called “global” ● System wide administration ● Management of all none global zones – eg. configure,install, boot, shut down, ...

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 47 Solaris Sparse Zone

● Share most of the host OS installation ● Thus sticks to the host OS and OS level ● Limited configuration ability ● Got only a few bits in private ● /etc , /var , /tmp and such – the image stays small ● IP­address ● Share most of the kernel ● Only one dispatcher, ● Only one memory management, ● Only one driver­set ● Spot on : This means only one AFS­Kernel module ! ● Needs minimum of system power to run

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 48 Solaris Zones – Virtualization

● Imho leanest virtualization possible – „application virtualization“ or „Light weight virtual machine“ ● Virtual Page Management & swapd ● Scheduler ● Helper App 1 App 2 Helper Helper App 1 App 2 Helper App 1 App 2 Helper OsA OsB Zone1 Zone2 OS1 OS2 Helper VMware Solaris special hardware hardware hardware Application virtualization Hardwarevirtualisierung OS Virtualization

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 49 Solaris Zone – physical view

● 2­3 Configuration files within the host ● A dedicated “root” file system (file set) – Prepared with some important files – Most files blend in from the Host­installation (RO) ● Solaris resource management framework ● A global process for management zoneadmd ● Some (7­10) system processes within the zone zsched /sbin/init /lib/svc/bin/svc.startd /lib/svc/bin/svc.configd /usr/lib/utmpd /usr/lib/saf/ttymon /usr/lib/saf/sac /usr/sbin/syslogd

see 'ps ­efZ' from within the globals zone

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 50 Sparse Zones : Pro & Contra

● Wanted – Modern most economical way of virtualization ● Avoid superfluous redundancy = minimum overhead – Easy to replicate ● To a second instance ● To a second host (with same OS) ● Traded in – closely tied to the host system ● All zones got virtually the same patch level ● All zones share most of the binaries & libraries – Patching might take significant longer. – Defining a zone is somewhat strange – ask Google

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 51 Why do I use Solaris

● Because it

– Performs well enough in terms of my spots

– Tend to solve my problems a reasonable way

– Does not bring in much of secondary problems

● e.g. no meta­db­partitions for mirroring

– Make me feel comfortable

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 52 Solaris

SMF

ZFS

Zones

>>> Solaris + AFS

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 53 Solaris

What is the point in using all of that

Solaris stuff with

AFS file server

Since I talked on that last year in Graz I don't go into that too deep now

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 54 Known AFS charakteristics

● Afs favour 'many small ' over 'one huge' – design goal (see e.g. 'vos move' ) – method of load distribution ● SysID , Dirty­Flag , ... (meta data) – logically belongs to the payload (vice partition) ● e.g. a SysID should tag a set of data not a host – however it is stored on a server directory ● Data and meta data can accidentally end up on different hosts ● Or can be left behind

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 55 AFS client and Solaris

● Afs cache cannot use native ZFS – Use a “zvol” (raw­dataset) instead and it works just fine. (how ever this is where ufs and vfstab steps in again)

● Sparse zones share the kernel with the host – Tokens held in Kernel are virtually available on all zones. – Root personally may differ over zones – Thus take care to use pag­shell when root

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 56 AFS and SMF

● Since I got a correct AFS­manifests and suitable AFS­methods it works like a charm for both, the server and client. ● Configuration is stored within SMF properties ● Optionally cacheinfo and ThisCell is written out using those property­values ● afsd get options according to properties ● The logically same is true for the server side ● However some more is needed also

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 57 AFS file server + ZFS

● Compression : speed up + save storage ● Compression ratio is 1.1 ... 1.3 for user volumes ● No waste on fsck (and no salvage in my case) ● No fiddling with device files or vfstab (in order to mount partitions always and on invariant place) ● The hierarchical concept of zpools/datasets allows to have the interpreter stored along with associated payload. (e.g. Openafs / OSD)

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 58 AFS fileserver + Zones + ZFS

● Example 1: Cut one 5TiB server into 10 virtual: = 10 * (500 GiB + OpenAFS server installation) Always keep at least one instance totally empty

● Now You can ... while on service: ● Suppress all but AFS ports on the server IP address ● Update afs version of a zone (with vols moved aside) ● Convert format by vos move (e.g. common to osd) ● Test a construct on a real live server ● Starting can be designed to be parallel or unfair ● Give real server training space to novices ● Add vice partition by using a further zone

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 59 AFS­fs + Zones + ZFS from SAN

● Example 2: like example 1 but also – Storage is on SAN as handled by x Zpools. – Use two machines (twins), zones are installed on both concurrently – Interpreter+config (e.g. sysid, NetInfo) is stored together with data (same zpool but different zfs) ● Now You can : ● Do some load distribution / balancing ● Recover from a host hardware crash much faster – No truble with IP, SysID, VLDB ● ....

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 60 Yes ­ I got it running

● Yes , I got that running. ● 4 * Hardware = 2 * twins ● 20 * 2 Zones ● 20 * Zpool ● 2 * Netapp filer + 1* Infotrend­Eurostor raid ● ZFS mirroring anyway ● Works for more than a year now – Got one case due to a system disk outage ● Trouble in spit of using “Solaris Disk Suite” ● Affected zones started on the twin – all works fine ● 3 of 4 Server now relay on ZFS exclusively ● Test are encouraging ● The last on should follow soon Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 61 Some known issues

● ZFS­Netapp issue ● Solved by using old VTOC (avoid EFI­Label) ● Shut down time out problem ● Solved by tuning zone­method (Sun learns, but slowly) ● AFS client cache & ZFS ● Solved by using zvol ● Export of wild­root zpools ( zpool-root != / ) ● Solved by introduce a further service (→ manifest example) ● No auto­takeover or auto­start right now ● Solved by human resources ● Which host serves a particular zone ● Patched zone­method to write host name into zones

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 62 Other dark corners

● ARP­cache time out issue on fail over

● Just flush the arp­table when starting/stoping a zone ● No inherent solid locking of Zpools

● Badly needed for robust automated startup and failover ● Software­Installation (e.g. Update) wants to have zones running – but is working w/o anyway

● No real problem up to now

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 63 What could come up next

● Auto / scheduled replication – Check whether customer would like it ● e.g. for web pages ● Maybe it could cool some hotspots – Present conception ● Spent a file server zone ● Initially replicate designated volumes to this zone – smaller volumes are preferred (short lock times) ● Run the following program within the zone – Check all hosted volumes for date(update)>date(release) ● Release those volumes – Sleep a few minutes – Repeat from start

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 64 abstract request

Virtualization lead to think a slightly different way ... and thus gain me to

ask again for unique vice partition IDs stored on the partition instead of have server IDs laying around on visited OS instances

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 65 Thank You

Thank You for Your kind attention

Mathias Feiler feiler@uni­hohenheim.de

Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 66 Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 67 Feiler@uni­hohenheim.de Why do I use Solaris for OpenAFS 68