Disk formatting is the initial part of the process for preparing a hard disk or other storage medium for its first use. The disk formatting includes setting up an empty file system. A disk formatting may set up multiple file systems by formatting partitions for each file system (Dell XPS M1210 Battery) http://www.hdd-shop.co.uk .
Disk formatting is also part of a process involving rebuilding an entire disk from scratch. A variety of utilities and programs exist for this task; pictured to the right is the iconic FORMAT.COM of DOS .
History
Modern HDDs, such as SAS and SATA drives, appear at their interfaces as a contiguous set of fixed size logical blocks; typically 512 bytes long but the industry is in the process of changing to 4,096 byte logical blocks (Dell Studio XPS 1640 Battery) .
The earliest disk drives also had fixed block sizes (e.g. IBM 350 disk storage unit block size was 100 6 bit characters) but starting with the 1301 IBM marketed subsystems that featured variable block sizes - a particular track could have blocks of different sizes. The disk subsystems on the IBM System/360 expanded this concept in the form of Count Key Data (CKD) and later Extended Count Key Data (ECKD) (Dell Studio XPS 1340 Battery) ;
however the use of variable block size in HDDs fell out of use in the 1990s; one of the last HDDs to support variable block size was the IBM 3390 Model 9, announced May 1993
Floppy disks generally only used fixed block sizes but these sizes were a function of the host controller so that a particular type of media (e.g., 5¼-inch DSDD) would have different block sizes depending upon the host's controller (Dell Vostro 1710 Battery) .
Optical disks generally only use fixed block sizes.
Two levels of formatting
Formatting a disk for use by an operating system and its applications and files involves two quite different types of formatting (Sony VGP-BPS13 battery) .
First, a low-level (closer to the hardware) formatting program will mark the surface of the disk with sector numbers and other information to be used later, in normal operations, by the disk controller. This is intended to be the permanent foundation of the disk, and is often completed at the factory (Sony VGP-BPS13/B battery) .
High-level formatting occurs during operating system installation, or when adding a new disk. This inscribes the file system format. Disk and distributed file system will specify an optional boot block, and various volume and directory information for the operating system (Sony VGP-BPS13/S battery) .
Low-level formatting of floppy disks
The low-level format of floppy disks (and early hard disks) is performed by the disk drive's controller.
The process is most easily described with a standard 1.44 MB floppy disk in mind. Low-level formatting of the floppy disk, normally writes 18 sectors of 512 bytes to each of 160 tracks (80 on each side) of the floppy disk, providing 1,474,560 bytes of storage on the disk (Sony VGP-BPS13A/B battery) .
Physical sectors are actually larger than 512 bytes, as in addition to the 512 byte data field they include a sector identifier field, CRC bytes (in some cases error correction bytes) and gaps between the fields. These additional bytes are not normally included in the quoted figure for overall storage capacity of the disk (Sony VGP-BPS13B/B battery) .
Different low-level formats can be used on the same media; for example, large records can be used to cut down on inter-record gap size.
Several freeware, shareware and free software programs (e.g. GParted, FDFORMAT, NFORMAT and 2M) allowed considerably more control over formatting, allowing the formatting of high-density 3.5" disks with a capacity up to 2 MB (Sony VGP-BPS13B/B battery) .
Techniques used include:
- head/track sector skew (moving the sector numbering forward at side change and track stepping to reduce mechanical delay),
- interleaving sectors (to minimize sector gap and thereby allowing the number of sectors per track to be increased) (Sony VGP-BPL9 battery) ,
- increasing the number of sectors per track (while a normal 1.44 MB format uses 18 sectors per track, it is possible to increase this to a maximum of 21), and
- increasing the number of tracks (most drives could tolerate extension to 82 tracks – though some could handle more, others could jam) (Sony VGP-BPL11 battery) .
Linux supports a variety of sector sizes, and DOS and Windows support a large-record-size DMF-formatted floppy format.
Low-level formatting (LLF) of hard disks
User instigated low-level formatting (LLF) of hard disk drives was common for minicomputer and personal computer systems until the 1990s (Sony VGP-BPL15 battery) .
IBM and other mainframe system vendors typically supplied their hard disk drives (or media in the case of removable media HDDs) with a low-level format. Typically this involved subdividing each track on the disk into one or more blocks which would contain the user data and associated control information (Dell Inspiron E1505 battery) .
Different computers used different block sizes and IBM notably used variable block sizes but the popularity of the IBM PC caused the industry to adopt a standard of 512 user data bytes per block by the middle 1980s.
Depending upon the system, low-level formatting was generally done by an operating system system utility (Dell Latitude E6400 battery) .
IBM compatible PCs used the BIOS which is involved using the MS-DOS debug program to transfer control to a routine hidden at different addresses in different BIOSs . Low-level format function can also be called as "erase" or "wipe" in different tools. For best results it's highly recommended to use tools created by hard disk's manufacturer (HP Pavilion dv6000 Battery) .
Transition away from LLF
Starting in the late 1980s, driven by the volume of IBM compatible PCs, HDDs became routinely available pre-formatted with a compatible low-level format. At the same time, the industry moved from historical (dumb) bit serial interfaces to modern (intelligent) bit serial interfaces and Word serial interfaces wherein the low level format was performed at the factory (Sony Vaio VGN-FZ31S battery) .
Today, an end-user, in most cases, should never perform a low-level formatting of an IDE or ATA hard drive, and in fact it is often not possible to do so on modern hard drives outside of the factory (Sony VGN-FZ31S battery) .
Disk reinitialization
While it is impossible to perform an LLF on most modern hard drives (since the mid-1990s) outside the factory, the term "low-level format" is still being used (erroneously) for what should be called thereinitialization of a hard drive to its factory configuration (and even these terms may be misunderstood) (Hp pavilion dv6000 battery) .
Reinitialization should include identifying (and sparing out if possible) any sectors which cannot be written to and read back from the drive, correctly. The term has, however, been used by some to refer to only a portion of that process, in which every sector of the drive is written to; usually by writing a zero byte to every addressable location on the disk, sometimes called zero-filling (SONY VGN-FZ38M Battery) .
The present ambiguity in the term "low-level format" seems to be due to both inconsistent documentation on web sites and the belief by many users that any process below a "high-level (file system) format" must be called a low-level format. Instead of correcting this mistaken idea (by clearly stating such a process cannot be performed on specific drives) (SONY VGN-FZ31z Battery) ,
various drive manufacturers have actually described reinitialization software as LLF utilities on their web sites. Since users generally have no way to determine the difference between a true LLF and reinitialization (they simply observe running the software results in a hard disk that must be partitioned and "high-level formatted") (Sony VGN-FZ31Z Battery) ,
both the misinformed user and mixed signals from various drive manufacturers have perpetuated this error. Note: Whatever possible misuse of such terms may exist (search hard drive manufacturers' web sites for all these terms), many sites do make such reinitialization utilities available (possibly as bootable floppy diskette or CD image files) (SONY VGN-FZ31E Battery) ,
to both overwrite every byte and check for damaged sectors on the hard disk.
One popular method for performing only the "zero-fill" operation on a hard disk is by writing zero-bytes to the drive using the Unix dd utility (available under Linux as well) with the "/dev/zero" stream as the input file (if=) and the drive itself (either the whole disk, or a specific partition) as the output file (of=) (SONY VGN-FZ31J Battery) .
Another method for SCSI disks may use the sg_format command to issue a low level "SCSI FORMAT UNIT" command.
High-level formatting
High-level formatting is the process of setting up an empty file system on the disk and installing a boot sector. This alone takes little time, and is sometimes referred to as a "quick format" (SONY VGN-FZ31M Battery) .
In addition, the entire disk may optionally be scanned for defects, which takes considerably longer, up to several hours on larger hard disks.
In the case of floppy disks, both high- and low-level formatting are customarily done in one pass by the software. In recent years, most floppies have shipped pre-formatted from the factory as DOSFAT12 floppies (SONY VGN-FZ31B Battery) .
It is possible to format them again to other formats, if necessary.
Host Protected Area
The host protected area, sometimes referred to as hidden protected area , is an area of a hard drive that is high level formatted so that the area is not normally visible to its operating system (OS) (SONY VGP-BPS13 Battery) .
Reformatting
"Reformatting" refers to a high-level format performed on a functioning disk drive to free the contents of it medium. "Reformatting" is unique to each operating system because what actually is done to existing data varies by OS. The most important aspect of the process is that it frees disk space for use by other data (Dell Inspiron 1320 Battery) .
To actually "erase" everything requires overwriting each block of data on the medium; something that is not done by many high-level formatting utilities.
Reformatting often carries the implication that the operating system and all other software will be reinstalled after the format is complete (Dell Inspiron 1320n Battery) .
Rather than fixing an installation suffering from malfunction or security compromise, it is sometimes judged easier to simply reformat everything and start from scratch. Various colloquialism exist for this process, such as "wipe and reload", "nuke and pave", "reimage", etc (Dell Inspiron 1464 Battery) .
Formatting
In DOS, OS/2 and Windows
Under MS-DOS, PC-DOS, OS/2 and Microsoft Windows, disk formatting can be performed by the format command (Dell Inspiron 1564 Battery) .
The format program usually asks for confirmation beforehand to prevent accidental removal of data, but some versions of DOS have an undocumented /AUTOTEST option; if used, the usual confirmation is skipped and the format begins right away. The WM/ FormatC macro virus uses this command to format the C: drive as soon as a document is opened (Dell Inspiron 1764 Battery) .
There is also the undocumented /U parameter that performs an unconditional format which under most circumstances overwrites the entire partition,preventing the recovery of data through software. Note however that the /U switch only works reliably with floppy diskettes (technically because unless /Q is used, floppies are always low-level formatted in addition to high-level formatted) (Dell Studio 1450 Battery) .
Under certain circumstances with hard drive partitions, however, the /U switch merely prevents the creation ofunformat information in the partition to be formatted while otherwise leaving the partition's contents entirely intact (still on disk but marked deleted) (Dell Studio 1457 Battery) .
In such cases, the user's data remains ripe for recovery with specialist tools such as EnCase or disk editors. Reliance upon /U for secure overwriting of hard drive partitions is therefore inadvisable, and purpose-built tools such as DBAN should be considered instead (Dell Latitude D610 Battery) .
Under OS/2, if you use the /L parameter, which specifies a long format, then format will overwrite the entire partition or logical drive.
In Unix and similar operating systems
Formatting disks on these systems is traditionally done using the mkfs command (Toshiba NB100 Battery) .
On Linux (and potentially other systems as well) mkfs is typically a wrapper around filesystem-specific commands which have the name mkfs.fsname, where fsname is the name of the filesystem with which to format the disk (Toshiba Satellite M65 battery) .
Some Unix-like operating systems have higher-level formatting tools, usually for the purpose of making disk formatting easier and/or allowing the user to partition the disk with the same tool. Examples include GNU Parted (and it's various GUI frontends such as GParted and the KDE Partition Manager) and the Disk Utility application on Mac OS X (Toshiba Satellite M60 battery) .
Some filesystems which are not supported by certain implementations of mkfs have their own manipulation tools; for example Ntfsprogs provides a format utility for the NTFS filesystem.
Recovery of data from a formatted disk
As with regular deletion, data on a disk is not fully destroyed during a high-level format (Dell Latitude D830 Battery) .
Instead, the area on the disk containing the data is merely marked as available (in whatever file system structure the format uses), and retains the old data until it is overwritten. If the formatting is done with a different file system than previously existed in the partition, some data may be overwritten that wouldn't be if the same file system had been used (Dell Studio 1735 Battery) .
However, under some file systems (e.g., NTFS, but not FAT), the file indexes (such as $MFTs under NTFS, "inodes" under ext2/3, etc.) may not be written to the same exact locations. And if the partition size is increased, even FAT file systems will overwrite more data at the beginning of that new partition (Dell Latitude D620 Battery) .
From the perspective of preventing the recovery of sensitive data through recovery tools, the data must either be completely overwritten (every sector) with random data before the format, or the format program itself must perform this overwriting, as the DOS FORMAT command did with floppy diskettes, filling every data sector with the byte value F6 in hex (Dell Inspiron Mini 10 Battery) .
Data recovery
Data recovery is the process of salvaging data from damaged, failed, corrupted, or inaccessible secondary storage media when it cannot be accessed normally. Often the data are being salvaged from storage media such as hard disk drives, storage tapes, CDs, DVDs, RAID, and other electronics (Sony VGN-FW11S Battery) .
Recovery may be required due to physical damage to the storage device or logical damage to the file system that prevents it from being mounted by the host operating system.
The most common "data recovery" scenario involves an operating system (OS) failure (typically on a single-disk, single-partition, single-OS system), in which case the goal is simply to copy all wanted files to another disk (Sony VGN-FW11M Battery) .
This can be easily accomplished with a Live CD, most of which provide a means to mount the system drive and backup disks or removable media, and to move the files from the system disk to the backup media with a file manager or optical disc authoring software (Dell Studio 1555 battery) .
Such cases can often be mitigated by disk partitioning and consistently storing valuable data files (or copies of them) on a different partition from the replaceable OS system files.
Another scenario involves a disk-level failure, such as a compromised file system or disk partition or a hard disk failure (Dell Latitude E5400 Battery) .
In any of these cases, the data cannot be easily read. Depending on the situation, solutions involve repairing the file system, partition table or master boot record, or hard disk recovery techniques ranging from software-based recovery of corrupted data to hardware replacement on a physically damaged disk (Dell Latitude E4200 Battery) .
If hard disk recovery is necessary, the disk itself has typically failed permanently, and the focus is rather on a one-time recovery, salvaging whatever data can be read.
In a third scenario, files have been "deleted" from a storage medium (Dell Vostro A840 Battery) .
Typically, deleted files are not erased immediately; instead, references to them in the directory structure are removed, and the space they occupy is made available for later overwriting. In the meantime, the original file may be restored. Although there is some confusion over the term, "data recovery" may also be used in the context of forensic applications or espionage (Dell Inspiron 300M Battery) .
Recovering data after physical damage
A wide variety of failures can cause physical damage to storage media. CD-ROMs can have their metallic substrate or dye layer scratched off; hard disks can suffer any of several mechanical failures, such as head crashes and failed motors; tapes can simply break (Dell Studio 1737 battery) .
Physical damage always causes at least some data loss, and in many cases the logical structures of the file system are damaged as well. Any logical damage must be dealt with before files can be salvaged from the failed media.
Most physical damage cannot be repaired by end users (Dell Inspiron E1505 battery) .
For example, opening a hard disk in a normal environment can allow airborne dust to settle on the platter and become caught between the platter and the read/write head, causing new head crashes that further damage the platter and thus compromise the recovery process (Dell Latitude E6400 battery) .
Furthermore, end users generally do not have the hardware or technical expertise required to make these repairs. Consequently, costly data recovery companies are often employed to salvage important data.
Recovery techniques
Recovering data from physically-damaged hardware can involve multiple techniques (Dell RM791 battery) .
Some damage can be repaired by replacing parts in the hard disk. This alone may make the disk usable, but there may still be logical damage. A specialized disk-imaging procedure is used to recover every readable bit from the surface. Once this image is acquired and saved on a reliable medium, the image can be safely analysed for logical damage and will possibly allow for much of the original file system to be reconstructed (Dell XPS M1530 battery) .
Hardware repair
Examples of physical recovery procedures are: removing a damaged PCB (printed circuit board) and replacing it with a matching PCB from a healthy drive, performing a live PCB swap (in which the System Area of the HDD is damaged on the target drive which is then instead read from the donor drive (Dell XPS M2010 battery) ,
the PCB then disconnected while still under power and transferred to the target drive), read/write head assembly with matching parts from a healthy drive, removing the hard disk platters from the original damaged drive and installing them into a healthy drive, and often a combination of all of these procedures (Dell Vostro 1000 battery) .
Some data recovery companies have procedures that are highly technical in nature and are not recommended for an untrained individual. Each of them will void the manufacturer's warranty. For companies who will not void a warranty, see companies such as Kroll Ontrack, SalvageData, and DriveSavers (Acer Aspire One battery) .
Recovering from logical (non-hardware) damage
Overwritten data
See also: Data erasure
When data have been physically overwritten on a hard disk it is generally assumed that the previous data are no longer possible to recover (Toshiba Satellite P10 Battery) .
In 1996,Peter Gutmann, a computer scientist , presented a paper that suggested overwritten data could be recovered through the use of Scanning transmission electron microscopy. In 2001, he presented another paper on a similar topic. Substantial criticism has followed, primarily dealing with the lack of any concrete examples of significant amounts of overwritten data being recovered (SONY VGN-FZ210CE Battery) .
To guard against this type of data recovery, he and Colin Plumb designed the Gutmann method, which is used by several disk scrubbing software packages.
Although Gutmann's theory may be correct, there's no practical evidence that overwritten data can be recovered (Dell Precision M70 Battery) .
Moreover, there are good reasons to think that it cannot.
Corrupt filesystems
In some cases, data on a hard drive can be unreadable due to damage to the filesystem (Toshiba Satellite L305 Battery) .
In the majority of these cases, at least a portion of the original data can be recovered by repairing the damaged filesystem using specialized data recovery software. This type of data recovery can be performed by knowledgeable end-users as it requires no special physical equipment (Toshiba Satellite T4900 Battery) .
However, more serious cases can still require expert intervention.
Online Data Recovery
"Online" or "Remote" data recovery is yet another method to restore the lost or deleted data (Toshiba PA3399U-2BRS battery) .
It is same as performing the regular software based recoveries except that this kind of recovery is performed over the Internet without physically having the drive or computer in possession. The recovery technician sitting somewhere else gains access to user's computer and complete the recovery job online (Toshiba Satellite A200 Battery) .
In this scenario, the user doesn't have to travel or send the media to anywhere physically.
Although online data recovery is convenient and useful in many cases, it still carries some points making it less popular than the classic data recovery methods (Toshiba Satellite 1200 Battery) .
First of all, it requires a stable broadband Internet connection for it to be performed correctly, which many third world countries still lack. Also, it cannot be performed in case of physical damage to media and for such cases, the traditional in-lab recovery has to take place (Toshiba Satellite M300 Battery) .
No comments:
Post a Comment