System databases, Backup procedure and restore statergies

Following are the system databases and their recommended backup procedures

·         master Database

Records all the system-level information for an instance of SQL Server. Symantec keys and Certificates information. Back up master as often as necessary to protect the data sufficiently for your business needs. Best practices recommendation are for regular backup schedule, which you can supplement with an additional backup after a substantial update.

·        msdb Database

Is used by SQL Server Agent for scheduling alerts and jobs. Back up msdb whenever it is updated.

·        model Database

Is used as the template for all databases created on the instance of SQL Server. Modifications made to the model database, such as database size, collation, recovery model, and other database options, are applied to any databases created afterward. Back up model only when necessary for your business needs; for example, immediately after customizing its database options. Best practice:  We recommend that you create only full database backups of model, as required.

·        Resource Database

Is a read-only database that contains system objects that are included with SQL Server. System objects are physically persisted in the Resource database, but they logically appear in the sys schema of every database. The Resource database resides in the mssqlsystemresource.mdf file, which contains only code. Therefore, SQL Server cannot back up the Resource database.

·        tempdb Database

Is a workspace for holding temporary objects or intermediate result sets. You cannot back up the tempdb system database.

·        Configure Distribution

A database that exists only if the server is configured as a replication Distributor. This database stores metadata and history data for all types of replication, and transactions for transactional replication.

Database restore:

System databases

System databases can be restored only from backups that are created on the version of SQL Server that the server instance is currently running. For example, to restore a system database on a server instance that is running on SQL Server 2008 SP1, you must use a database backup that was created after the server instance was upgraded to SQL Server 2008 SP1.

To restore any database, the instance of SQL Server must be running. Startup of an instance of SQL Server requires that the master database is accessible and at least partly usable. If master becomes unusable, you can return the database to a usable state in either of the following ways:

·         Restore master from a current database backup.

·         Rebuild master completely

Performing a Complete Database Restore (Simple Recovery Model)

In a complete database restore, the goal is to restore the whole database. The whole database is offline for the duration of the restore. Before any part of the database can come online, all data is recovered to a consistent point in which all parts of the database are at the same point in time and no uncommitted transactions exist. Under the simple recovery model, the database cannot be restored to a specific point in time within a specific backup.

1.       A full database restore under the simple recovery model involves only one or two RESTORE statements, depending on whether a differential database backup has to be restored.

If you are using only a full database backup, just restore the most recent backup, as shown in the following illustration.

If you are also using a differential database backup, restore the most recent full database backup without recovering the database, and then restore the most recent differential database backup and recover the database. The following illustration shows this process.

When you are completely restoring a database, one restore sequence should be used. The following example shows the critical options in a restore sequence for the complete database restore scenario. A restore sequence consists of one or more restore operations that move data through one or more of the phases of restore. Syntax and details that are not relevant to this purpose are omitted.

Performing a Complete Database Restore (Full Recovery Model)

Under the full recovery model, the database can be restored to a specific point in time. The point in time can be the most recently available backup, a specific date and time, or a marked transaction.

Typically, recovering a database to the point of failure involves the following basic steps:

1.       Back up the active transaction log (known as the tail of the log). This creates a tail-log backup. If the active transaction log is unavailable, all transactions in that part of the log are lost.

2.       Restore the most recent full database backup without recovering the database (RESTORE DATABASE database_name FROM backup_device WITH NORECOVERY).

3.       If differential backups exist, restore the most recent one without recovering the database (RESTORE DATABASE database_name FROM differential_backup_device WITH NORECOVERY).

4.       Starting with the first transaction log backup that was created after the backup you just restored, restore the logs in sequence with NORECOVERY.

5.       Recover the database (RESTORE DATABASE database_name WITH RECOVERY). Alternatively, this step can be combined with restoring the last log backup.

6.       A complete database restore can usually be recovered to a point of time or marked transaction within a log backup. However, under the bulk-logged recovery model, if the log backup contains bulk-logged changes, point-in-time recovery is not possible. For more information, see Restoring a Database to a Point Within a Backup.

7.       The following illustration shows this process. After a failure occurs (1), a tail-log backup is created (2). Next, the database is restored to the point of the failure. This involves restoring a database backup, a subsequent differential backup, and every log backup taken after the differential backup, including the tail-log backup.

1.       When you are completely restoring a database, a single restore sequence should be used. The following example shows the critical options in a restore sequence for the complete database restore scenario in which the database is restored to the point of failure. A restore sequence consists of one or more restore operations that move data through one or more of the phases of restore. Syntax and details that are not relevant to this purpose are omitted.

The database is restored and rolled forward. A database differential is used to reduce roll-forward time. This restore sequence is intended to eliminate work loss; the last backup that is restored is a tail-log backup.

All this data is reference to different sources i.e Microsoft

Reading SQL Server Database Transaction Log

This is very exciting for me to query and look into Transaction log and see what is happening behind the scene. I will explain how to read your database transaction log file and how transactions are written for your database if you perform any database activity. There is an undocumented function called "fn_dblog" which enables you to read data from your transaction log which contains very informative data about things that are happening in your database.

The function (fn_dblog) requires a beginning LSN and ending LSN for a transaction. NULL is the default for this function and this will return all log records from the transaction log file.

Create a Database

To show how this works, we will create a database and a table to play with this function. Run the below SQL code to create a database and table.

--Create DB.

USE [master];

GO

CREATE DATABASE ReadLog;

GO

-- Create tables.

USE ReadLog;

GO

CREATE TABLE [Abc] (

    [Sr.No] INT IDENTITY,

    [Date] DATETIME DEFAULT GETDATE (),

    [Value] CHAR (25) DEFAULT 'Hoora');

We have created a database named "ReadLog" and a table 'Abc' with three columns. Now you can check all information and processes which have been used by SQL Server to create the database and table. We will run the below code to check the log file for this newly created database to check what processes and steps SQL Server took to create the database and table.

USE ReadLog;

GO

select COUNT(*) from fn_dblog(null,null)

  

We can see there are 476 rows that have been generated for just creating a dummy database and a blank table. Look at the below code to see the data in the transaction log file as shown in above attached picture

USE ReadLog;

GO

select [Current LSN],

       [Operation],

       [Transaction Name],

       [Transaction ID],

       [Transaction SID],

       [SPID],

       [Begin Time]

FROM   fn_dblog(null,null)

You can see in the above screenshot that the transaction name column shows the database name, similarly it will show the create table for the table creation code. Transaction ID is the same for all parts of a transaction. The value for transaction name will be filled only when the particular transaction starts with "LOP_BEGIN_XACT" in the Operation column. "LOP_BEGIN_XACT" means begin transaction. The operation column will let us know which operation is performing like an insert, update, delete, shrink, lock, page allocation etc...  It is pretty easy to understand the operation based on these key words to see what operation is being performed by SQL Server.

Looking at Insert Log

Now we will run a few DML scripts to check how data insertion, updating or deletion is logged in the database log file. During this operation you can also track how a page is allocated or de-allocated.

USE ReadLog

go

INSERT INTO Abc DEFAULT VALUES ;

GO 100

GO

UPDATE Abc

SET Value=Manama'

WHERE [Sr.No]<5

GO

DELETE Abc 

WHERE [Sr.No]>90

Go

Let's check our database log file again. As we saw from above, there is a lot of info logged in the transaction log file, so I will filter the data.

USE ReadLog

go

SELECT

 [Current LSN],

 [Transaction ID],

 [Operation],

  [Transaction Name],

 [CONTEXT],

 [AllocUnitName],

 [Page ID],

 [Slot ID],

 [Begin Time],

 [End Time],

 [Number of Locks],

 [Lock Information]

FROM sys.fn_dblog(NULL,NULL)

WHERE Operation IN 

   ('LOP_INSERT_ROWS','LOP_MODIFY_ROW',

    'LOP_DELETE_ROWS','LOP_BEGIN_XACT','LOP_COMMIT_XACT')  

Your output will look something like the above screenshot after running the above script. From context it is easy to understand that table inserted as HEAP with Start and End Time. Page information, table info etc.

Similarly, it will show you this same kind of information for UPDATE and DELETE statements. You can have a lots of information from Log file like Page Split, Partition, Pages allocation etc.

Transaction Log behavior on Backup

Now I will run a backup and see the transaction log file again. Run a backup on this database and then again check the transaction log file.

Go USE ReadLog;

GO

Select COUNT(*)

FROM   fn_dblog(null,null)

BACKUP DATABASE [ReadLog] TO  DISK = N'c:\Program Files\Microsoft SQL Server\MSSQL10_50.SQLEXPRESS\MSSQL\Backup\ReadLog.bak' WITH NOFORMAT, NOINIT,  

NAME = N'ReadLog-Full Database Backup', SKIP, NOREWIND, NOUNLOAD,  STATS = 10

GO

Select COUNT(*)

FROM   fn_dblog(null,null)

As we can see, the number of rows has been drastically reduced after doing a backup and it has been reduced to 9 rows from 151. This means the inactive part of the log which tracked the transactions has been dumped to a backup file and the original entries from the log file have been flushed. Now you can shrink the log file if necessary.

How we can see Log data other than this function

DBCC Log(): command to see log information, but this command will not give you detail information. 
Trace flag 2537: look at all logs and not just the active log.

Recover data from Log file.

Using this fucntion fn_DBLog() you can restore T-Log backup upto required LSN so that you can retrive you data up to that point. 

There are some interesting options to recover your data from log and log backups. I will write blog on these after testing. I gather all this data from different blogs as follows

Trace flag 2536: You can see inactive portion of the log if not truncated or backed up.

fn_dump_dblog: allows you to dump and search log records from a log backup file, without having to restore the database

Related Blogs from different users.

using-fn_dblog-fn_dump_dblog-and-restoring-with-stopbeforemark-to-an-lsn
recover-deleted-data-in-sql-server