Granular Recovery of Active Directory Objects

Andrew Zhelezko Veeam Technical Marketing Engineer Granular Recovery of Active Directory Objects

Contents Introduction ...... 3

Reanimating Active Directory tombstone objects ...... 4

Active Directory recycle bin ...... 5

Windows Server Backup ...... 7.

Active Directory application restore from a system state ...... 10

Active Directory snapshots ...... 14

Veeam Explorer for Microsoft Active Directory ...... 16

Virtual Domain Controller ...... 16

Physical Domain Controller ...... 18

Active Directory object restore ...... 19

Conclusion ...... 24

Appendix A: Resources used ...... 24

About the Author ...... 25

About Veeam Software ...... 25.

Introduction Microsoft Active Directory is a standard in corporate environments where policy-based management and easy scalability are required . It’s almost impossible to imagine how system administrators would be able to do their jobs effectively if LDAP protocol didn’t exist . Not only is Active Directory a great power, but it's also a great responsibility — and it requires spending lots of time with it in order to maximize its capabilities . For example, creating and removing Active Directory objects is one of the most frequent requests for system administrators . Beyond that, there are many situations when something goes wrong and an admin is required to recover an Active Directory object, a bunch of objects or its attributes after an accidental change or even an intentional deletion .

In this white paper, you will learn more about recovering Active Directory objects . I’ll show you a few different options for performing those tasks, including native Windows tools and external tools like Veeam® Explorer™ for Microsoft Active Directory . I’ll also compare these tools by performing the same operations with each of them . In addition, since it’s not possible to talk about recovery operations without mentioning backup, I’ll also discuss Domain Controller backup processes .

Because Active Directory has existed for several years, there are more tools for managing it than I’m going to cover . This is not the ultimate guide to cover all possible scenarios or a universal solution to solve any Active Directory-related problem . Moreover, because there might be multiple domain controllers (DCs), complicated architectures and different security policies in your environment, I won’t be taking into consideration every single case . Feel free to reach out to me after you’ve read the white paper and I’d be happy to discuss your personal Active Directory experience or other interesting cases .

Reanimating Active Directory tombstone objects Prior to Windows Server 2008 R2, a deleted Active Directory object was made immediately invisible to the administrator . Active Directory simply marked it as tombstoned, dropped most of its attributes and preserved it physically for a lifetime period (60 days for Windows Server 2000/2003 and 180 days for Windows 2003 SP1/2008), just to ensure that the information was successfully replicated across the system . Once a lifetime period was over, a special process called garbage collector physically removed the object from the database .

The Tombstone mechanism was never intended to be a temporary recycle bin, and objects weren’t ever supposed to be reanimated, even if was technically possible to do so by using a program like ldp . Below, is my attempt to find all tombstoned objects in my lab .

Figure 1. Searching deleted objects with LDP utility.

If you’re interested in the reanimation of Active Directory tombstone objects, I recommend the following guide: https://technet.microsoft.com/en-us/magazine/2007.09.tombstones.aspx

Pros:

• Universal method for Windows 2003+ domains

• DC doesn’t need to be down

Cons:

• Clunky user interface of LDP tool

• Object attributes are not preserved

• Works only for deleted, not changed, objects

• Recovery is limited by a lifetime period value

• No automation for hierarchy recovery Active Directory recycle bin Things started to change after Windows 2008 R2 when Microsoft implemented a long-awaited Active Directory recycle bin . The standard life cycle of an Active Directory object was extended and the logic of object deletion changed . With this feature enabled, the object started going to the deleted objects container right after deletion, where it’s kept for the lifetime of the deleted object . Most important, all of the object’s link-valued and non-link-valued attributes were able to be preserved by the system for the same lifetime period . This means you can now easily recover an object with those attributes during this period .

Once the lifetime is over, the system changes the object status to recycled, drops most of its attributes and the objects becomes logically equal to what used to be tombstoned in Windows Server 2003 and Windows Server 2008 . The only difference is the recycled object can’t be restored or reanimated, so it should be automatically removed by a garbage collector .

Figure 2. Active Directory Object life cycle with Active Directory recycle bin enabled

So far, the Active Directory recycle bin is not enabled by default on any Windows Server OS . To utilize this tool, you should prepare your environment, make sure that every DC in your forest is running Windows Server 2008 R2 and above, and set your forest functional level to Windows 2008 R2 or above .

NOTE: Enabling the Active Directory recycle bin requires you to upgrade the forest schema configuration and it can’t be undone later.

Before using Active Directory recycle bin, keep in mind that:

1 . Enabling the Active Directory recycle bin changes all current tombstoned objects into recycled objects, so you won’t be able to restore them once enabling is done .

2 . The process of restoring multiple dependent objects can be difficult, since it requires a strict order of restore, starting from the higher-placed objects .

3 . In Windows Server 2008 R2, every operation related to the Active Directory recycled bin should be done via PowerShell cmdlets, no GUI provided . Windows Server 2012 and above introduce Active Directory Administration Center (ADAC), where all recycle bin operations can be performed via GUI .

4 . The recycle bin doesn’t have anything in common with Active Directory backup, and it won’t help to restore a whole DC if it is damaged .

Figure 3. Enabling Active Directory recycle bin in Windows Server 2012 via ADAC

Additional information about Active Directory recycle bin architecture in Windows Server 2008 R2 and above can be found here:

• https://technet.microsoft.com/en-us/library/dd379542(v=ws.10).aspx

• http://blogs.technet.com/b/askds/archive/2009/08/27/the-ad-recycle-bin-understanding- implementing-best-practices-and-troubleshooting.aspx

Pros:

• A universal method for Windows 2008 (or newer) domains

• Object attributes are preserved for a lifetime period

• Doesn’t require reboot of DC

• GUI for Windows Server 2012+

Cons:

• Works for Windows Server 2008 R2+ domains

• Works only for deleted, not changed objects

• Recovery is limited by a lifetime period value

• No automation for hierarchy recovery Windows Server Backup The easiest way to protect the whole DC is to use native Windows Server Backup — a successor of the formerly used NTBackup utility . This feature comes with Windows Server 2008 and newer versions, and has the basic capabilities for backup and recovery of volumes, files and even system state . You can install it on your Domain Controller (DC) using the add roles and features wizard .

Figure 4: Windows Server Backup feature installation

Run wbadmin from the command line, or launch Windows Server Backup from the administrative tools to start the utility . You can make an individual backup, schedule a backup task or restore data from an existing backup point .

Figure 5: wbadmin GUI

Let’s make an Active Directory backup using this command line tool . First, run command line (cmd) in the elevated mode . Then, type wbadmin ? and get the list of supported commands . There is a special command for this case: wbadmin start systemstatebackup -backupTarget:, which is where you should change to any local volume or a path to a network share .

NOTE: Once running, it grabs not only Active Directory-related items, which are needed for the Active Directory restore, but a system state, including the SYSVOL and NTDS folders, and puts them into a specified location.

I’ll run the following script just to demonstrate how it works: wbadmin start systemstatebackup –backupTarget:d:

Figure 6: wbadmin, system state backup

Depending on the amount of data in Active Directory or the server size, the backup may take some time to complete . Then, you’ll be able to find backup files the correct place, which you’ve chosen . See this example of my backup below:

Figure 7: System state backup files by wbadmin utility

Active Directory application restore from a system state

Because we have a backup copy of the system state, we can either restore the whole DC — rebooting it into Directory Services Restore Mode (DSRM) to revert the database to the previous state — neglect all last changes or recover the Active Directory database and granularly restore desired objects . The second option is more convenient, simply because you wouldn’t want to lose all newly created data, so I’m about to use the same Windows Backup tool for this purpose . But first, it’s nice to check the backup copy for the items it contains . wbadmin get items –version:date-of-backup-file

Figure 8: Windows Backup tool, get items command

Interestingly, backup copy has an application Active Directory and ntds component, so I can restore the whole application itself . The following command does that for me and places all Active Directory files into a specified folder: wbadmin start recovery -version:07/01/2015-15:22 -itemtype:App -items:AD -recoveryTarget:C:\share\

Figure 9: Windows Backup tool, Active Directory recovery

I can now access files in the folder and see that those files include the Active Directory databasentds. ( dit) . To restore Active Directory objects from that database, I should mount it to the LDAP instance and using the dsamain utility . You should keep these parameters in mind:

• dbpath — path to ntds .dit file

• ldapPort — port where you’d like to put this LDAP instance dsamain -dbpath "C:\share\Active Directory\ntds\ntds.dit" -ldapPort 30000

Figure 10: DSAMAIN utility, mounting AD database

Once the temporary LDAP instance is up and running, you can finally access it . Open your Active Directory Users and Computers (ADUC) console (dsa.msc) or Active Directory Administrative Center (ADAC) on new Windows Servers, right-click on your domain and select change domain controller . Then, choose a new instance name (localhost) and port . You will see that the status of this DC goes to online, and you can then apply new settings to get access to the Active Directory saved copy .

Figure 11: ADUC, Change Domain Controller

You can now access old data by navigating via the ADUC/ADAC console . However, you might want to perform an actual restore operation, instead of just browsing through old data . Because there is no easy way to export, we’ll have to use another utility to complete this operation . My suggestion is to use the LDIFDE utility to export the desired data and then import it back to the production Active Directory .

Just like the previous two processes, these operations should be done in the elevated mode of a command line . The example below allows me to export the data of a deleted user, Maria, and move it to the .ldf file: ldifde -f C:\exportuserMaria.ldf -s hvdc.andrewlab.local:30000 -r "CN=Maria Levkina" –m

• f — To let the utility operate with the file; in this case, to create it in C drive

• s — To specify the LDAP instance

• r — To implement a filter, CN (common name)

• m — To forbid the utility from exporting the attributes that are owned by the system account . This will prevent failure when you import data

Figure 12: LDIFDE utility, export user operation

As a result of this operation, you have an .ldf file with user data exported to a custom place .

Figure 13: User data exported by ldifde

You might want to import the data back to your Active Directory right away, but if you have password complexity requirements enabled in your domain, you will need to take one more step .

Because LDIFDE can’t extract a user’s password data, the .ldf file contains password information, which will be perceived as a blank password during the importing attempt . At the same time, it is against your domain policy to have an enabled user with a password that doesn’t meet complexity requirements . This means that the importing attempt will fail .

The workaround is to open the .ldf file again and change theuserAccountControl value from 66048 (user is enabled) to 514 (user is disabled) . Now, you can import data back to Active Directory: ldifde –i -f C:\exportuserMaria.ldf

• i — To select import mode

• f — To operate with the specified file

Figure 14: ldifde, importing user account

Now, you have to reset the user’s password to set up a password with the required complexity, and then activate the user account . On the first login attempt to the domain, the user will be prompted to change a password .

Once you’re finished, don’t forget to unmount the temporary LDAP instance and switch back to the original Active Directory database .

Pros:

• Works for both deleted and changed objects

• Restoring to a different DC is possible

• Applicable for both DC backup and Active Directory database backup

Cons:

• Usage complexity

• Few object attributes are preserved

• Requires downtime for services

• No automation for hierarchy recovery

Active Directory snapshots Wait! Active Directory snapshots have nothing to do with VM snapshots . Active Directory snapshots are a special mechanism for Windows Server 2008 (or newer), which is supposed to make the life of a domain administrator easier .

NOTE: Domain and Forest functional levels should be at least at Windows Server 2008 to try this technology.

To manage Active Directory data with these snapshots, we need the ntdsutil utility, which is also present by default in Windows Server . These snapshots are created by using a VSS service and are nothing more than checkpoints of the whole system disk at a specific point in time .

Open command-line in elevated mode . Type ntdsutil to launch the utility . Execute activate instance ntds to activate the instance . Then, go to the snapshot-related functionality by typing snapshot .

Run list all to display all the snapshots that you have . To check if you already have any mounted snapshots, run list mounted .

Then, you can mount any existing snapshots by executing mount snapshot GUID .

(For more information about the Active Directory snapshots syntax, check out https://technet.microsoft.com/en-us/library/cc731620(WS.10).aspx) .

Once mounted, you’ll see the snapshot mounting point (system drive) and will be able to access snapshot data .

Note: Mounted snapshots are presented in a read-only state, so you won’t be able to change any data inside, including Active Directory entries.

Figure 15: ntdsutil utility

After a snapshot is mounted, you will be able to access its data via the system drive .

Figure 16: Mounted VSS snapshot

Now you can use dsamain and LDIDFE utilities again to access the data, mount Active Directory to the temporary LDAP instance, and export and import desired object(s) just as we did before . This method can be an alternative to using the Windows Server Backup tool because it works faster and is more flexible . However, a drawback of using Active Directory snapshots is the overhead disk I/O within the write operation to Active Directory . In addition, the longer a snapshot lives, the larger the delta file becomes, which may also affect performance . Finally, the issue of restoring special attributes owned by the system account and the passwords will not be solved .

Pros:

• All utilities are installed by default

• There is no DC reboot required

• Works for both deleted and changed objects

• You can view snapshots in LDP, ADUC, ADAC, ADSIEDIT or PowerShell

Cons:

• Command-line interface

• Complexity of managing three different utilities

• Only certain writeable attributes can be restored (without preserving some old values)

• The recovered attribute value is a new write, not a true recovery

• It shouldn’t be used for long-term purposes (shadow copy)

• No clear monitoring of disk usage

• Snapshots are not a full recovery solution

Veeam Explorer for Microsoft Active Directory Veeam® offers different products for handling backing up virtual and physical systems . In the end, you can gather them in the same infrastructure . A Veeam Backup & Replication™ backup repository can be the storage target for Veeam Endpoint™ Backup jobs and also for Veeam Backup & Replication itself . It ends up being pretty easy to manage all backup copies of all machines within one console . This is especially important if you have a mixed infrastructure with both physical and virtual DCs .

Virtual Domain Controller

NOTE: Be careful while working with virtual DCs. Keep in mind that there is a possible USN rollback issue for old Windows Server versions. Read more here: https://technet.microsoft.com/en-us/library/d2cae85b- 41ac-497f-8cd1-5fbaa6740ffe(v=ws.10)#usn_and_usn_rollback

For virtual DCs, you’ll need to have Veeam Backup & Replication installed and configured . If you need help with this requirement, please refer to the following video recorded by a Veeam system engineer: https://youtu.be/9PJL8wa4s8A

Now, let’s configure a backup job for a virtual DC . The process is rather easy: Just add a DC to the task, specify the retention policy for the backup chain and schedule the task . Don’t forget to enable application-aware processing to ensure transactional consistency of backup files, including the AD DS database .

Figure 17: Edit Backup Job: Guest processing

Now, you can run a backup job and check its status .

Figure 18: Performing incremental backup of DC

You can store a backup in the cloud with Veeam Cloud Connect, copy it to another datastore or to tape using Veeam Backup Copy jobs and much more . The backup is now safe and can be restored as soon as you need it .

Physical Domain Controller

For a physical DC, you should use Veeam Endpoint Backup Free, which is a new, free utility for protecting the remaining Windows-based endpoints in your infrastructure . Install the program directly on your DC, configure a backup task forsystem state, which will allow you to perform bare-metal restores when needed, and point it to store the backup files in the Veeam Backup & Replication repository .

Figure 19: Veeam Endpoint Backup Free: Backup job statistics

Then, navigate to Veeam Backup & Replication and find both physical and virtual DC among the backups-disk files .

Figure 20. Veeam Backup & Replication: Backups, disk

Active Directory object restore

Performing granular restore of Active Directory items is very easy with Veeam Explorer for Microsoft Active Directory . Start navigating to backups in the Veeam Backup & Replication GUI . Then, find your DC backup . Right-click on it and select restore application items -> Microsoft Active Directory objects .

Figure 21. Veeam Backup & Replication: Restore application items

NOTE: For Veeam Endpoint backups, you should perform a guest files restore of the ntds.dit database and then open the restored database with Veeam Explorer for Microsoft Active Directory.

Veeam Backup & Replication then quickly extracts the corresponding AD DS database from the DC image-level backup and adds the database to the Veeam Explorer for Microsoft Active Directory scope .

Figure 22: Veeam Explorer for Microsoft Active Directory: Mounting restore point

You’re able to navigate through the actual data of this database and see all the items included .

Figure 23: Veeam Explorer for Microsoft Active Directory: “restore to…” option

The great thing here is that you can make a comparison with a production environment and see only changed attributes .

Figure 24. Veeam Explorer for Microsoft Active Directory: “compare with production”

“Restore to” allows you to specify a couple of options before the actual restore . You can also restore to a different server or with another account . As an example, I’ll be restoring the account of the same domain user, Maria, which was recently deleted . First, I need to specify the server connection parameters and the administrative account .

Figure 25: Active Directory Restore Wizard options

Select the original container or any other for your convenience:

Figure 26: Veeam Explorer for Microsoft Active Directory, selection of a restore location

You will then be prompted to select password options . You can also restore an account with a predefined password, which will reduce the administrator’s load . Imagine dropping an entire organizational unit (OU) with hundreds of users during the night and then having to restore it . In the morning, all personnel will be prompted to change their passwords upon log in, and they will also start asking questions . Clearly, it’s a good idea to avoid such a situation if possible .

NOTE: If you’re restoring a password from an imported backup or from a VeeamZIP™ file (backup file done by Veeam Backup Free Edition), be sure to place the system registry hive (%systemroot%\System32\Config by default) in the same folder with the AD DS database referenced by Veeam Explorer.

Figure 27: Veeam Explorer for Microsoft Active Directory: Specify password restore options

You can enable the account right after restore, disable it or to use a state from a backup:

Figure 28: Veeam Explorer for Microsoft Active Directory: Specify account restore options

In Specify restore options, choose which object types to restore: changed, deleted or both . Then, select whether you want to restore entire objects or selected attributes and define what to do with multi-valued attributes (replace is selected by default) .

Figure 29: Veeam Explorer for Microsoft Active Directory: Specify restore options

After you select restore, Veeam Explorer proceeds with the operation, and you’ll soon get a result . The user is now restored to a production system and will be able to log in to the domain with an old password with no questions asked from the user’s side .

Figure 30: Veeam Explorer for Microsoft Active Directory: Restore summary

Don’t forget that Veeam Explorer for Microsoft Active Directory is just a small functionality of Veeam Backup & Replication and it’s already included in the free edition .

Active Directory backup and object restore is only one of the functions of this software . Cloud and tape targets for backup copies, incremental backup, flexible scheduling and integration with third-party hardware are also reasons to make Veeam a solid part of any environment .

Pros

• Simplicity, user-friendly UI

• Restores attributes/objects right to production

• All attributes can be compared with current ones and restored with preserved old values

• Passwords hash restore (users won’t notice anything)

• Any target (cloud, tape or external NAS) for storing backup copies – easy disk management

• Working for Windows Server 2003 and newer

• Recovery delegation (from Veeam Enterprise Manager)

Cons

• The cost of a license for Veeam Backup & Replication

• No option to restore Group Policy Objects (GPOs)

NOTE: Veeam Backup & Replication v9 will be released at Q1 of 2016 and include an updated version of Veeam Explorer for Microsoft Active Directory. New features will allow you to restore GPOs, Active Directory- integrated DNS records and Configuration Partition objects. Conclusion There are many options for Active Directory recovery, but the most important is to stick with the method that is the most suitable for you and that you know by heart . Plan it out, don’t hesitate to emulate a disaster recovery strike and test your actions and corporate policies . May the Active Directory power be on your side! Appendix A: Resources used TechNet: Active Directory Recycle bin understanding and implementing: http://blogs.technet.com/b/askds/archive/2009/08/27/the-ad-recycle-bin-understanding- implementing-best-practices-and-troubleshooting.aspx

TechNet: Ultimate tombstoned objects and LDP .exe guide: https://technet.microsoft.com/en-us/magazine/2007.09.tombstones.aspx

Active Directory Deep Dive, video course: http://go.veeam.com/learn-active-directory-deep-dive-expert-video-tutorials

About the Author Andrew Zhelezko is a Veeam Technical Marketing Engineer and a certified IT Professional with years of experience in the Virtualization world . He gained a strong understanding of Veeam products by working initially in Veeam technical support .

This practical experience has helped him speak the same language as Veeam community members . His goal is to help others realize the beauty and power of virtualization . Talk to Andrew at SpiceWorks . About Veeam Software Veeam® recognizes the new challenges companies across the globe face in enabling the Always- On Business™, a business that must operate 24/7/365 . To address this, Veeam has pioneered a new market of Availability for the Always-On Enterpriser™ by helping organizations meet recovery time and point objectives (RTPO™) of less than 15 minutes for all applications and data, through a fundamentally new kind of solution that delivers high-speed recovery, data loss avoidance, verified protection, leveraged data and complete visibility . Veeam Availability Suite™, which includes Veeam Backup & Replication™, leverages virtualization, storage, and cloud technologies that enable the modern data center to help organizations save time, mitigate risks, and dramatically reduce capital and operational costs .

Founded in 2006, Veeam currently has 34,500 ProPartners and more than 168,000 customers worldwide . Veeam's global headquarters are located in Baar, Switzerland, and the company has offices throughout the world . To learn more, visit http://www.veeam.com .

NEW Veeam® Availability Suite ™ v9

RTPO™ <15 minutes for ALL applications and data

Learn more and preview the upcoming v9 release

vee.am/v9