makes x be the root of T. If T is not empty, then the first phase inserts x as a child of an existing node. Because we assume that the value of x. f
depends only on information in the other attributes of x itself and the
information in x’s children, and because x’s children are both the sentinel T. nil, it takes only O(1) time to compute the value of x. f.
Having computed x. f, the change propagates up the tree. Thus, the total time for the first phase of insertion is O(lg n). During the second phase, the only structural changes to the tree come from rotations. Since only
two nodes change in a rotation, but a change to an attribute might need
to propagate up to the root, the total time for updating the f attributes is
O(lg n) per rotation. Since the number of rotations during insertion is at most two, the total time for insertion is O(lg n).
Like insertion, deletion has two phases, as Section 13.4 discusses. In the first phase, changes to the tree occur when a node is deleted, and at
most two other nodes could move within the tree. Propagating the
updates to f caused by these changes costs at most O(lg n), since the changes modify the tree locally along a simple path from the lowest
changed node to the root. Fixing up the red-black tree during the
second phase requires at most three rotations, and each rotation
requires at most O(lg n) time to propagate the updates to f. Thus, like insertion, the total time for deletion is O(lg n).
▪
In many cases, such as maintaining the size attributes in order-
statistic trees, the cost of updating after a rotation is O(1), rather than
the O(lg n) derived in the proof of Theorem 17.1. Exercise 17.2-3 gives an example.
On the other hand, when an update after a rotation requires a traversal all the way up to the root, it is important that insertion into
and deletion from a red-black tree require a constant number of
rotations. The chapter notes for Chapter 13 list other schemes for balancing search trees that do not bound the number of rotations per
insertion or deletion by a constant. If each operation might require Θ(lg
n) rotations and each rotation traverses a path up to the root, then a single operation could require Θ(lg2 n) time, rather than the O(lg n) time bound given by Theorem 17.1.
Exercises
17.2-1
Show, by adding pointers to the nodes, how to support each of the
dynamic-set queries MINIMUM, MAXIMUM, SUCCESSOR, and
PREDECESSOR in O(1) worst-case time on an augmented order-
statistic tree. The asymptotic performance of other operations on order-
statistic trees should not be affected.
17.2-2
Can you maintain the black-heights of nodes in a red-black tree as
attributes in the nodes of the tree without affecting the asymptotic
performance of any of the red-black tree operations? Show how, or
argue why not. How about maintaining the depths of nodes?
17.2-3
Let ⊗ be an associative binary operator, and let a be an attribute
maintained in each node of a red-black tree. Suppose that you want to
include in each node x an additional attribute f such that x. f = x 1. a ⊗
x 2. a ⊗ … ⊗ xm. a, where x 1, x 2, … , xm is the inorder listing of nodes in the subtree rooted at x. Show how to update the f attributes in O(1) time after a rotation. Modify your argument slightly to apply it to the
size attributes in order-statistic trees.
This section shows how to augment red-black trees to support
operations on dynamic sets of intervals. In this section, we’ll assume
that intervals are closed. Extending the results to open and half-open
intervals is conceptually straightforward. (See page 1157 for definitions
of closed, open, and half-open intervals.)
Intervals are convenient for representing events that each occupy a
continuous period of time. For example, you could query a database of
time intervals to find out which events occurred during a given interval.
The data structure in this section provides an efficient means for
maintaining such an interval database.
A simple way to represent an interval [ t 1, t 2] is as an object i with attributes i. low = t 1 (the low endpoint) and i. high = t 2 (the high endpoint). We say that intervals i and i′ overlap if i ∩ i′ ≠ ∅ , that is, if i. low ≤ i′. high and i′. low ≤ i. high.
Figure 17.3 The interval trichotomy for two closed intervals i and i′. (a) If i and i′ overlap, there are four situations, and in each, i. low ≤ i′. high and i′. low ≤ i. high. (b) The intervals do not overlap, and i. high < i′. low. (c) The intervals do not overlap, and i′. high < i. low.
As Figure 17.3 shows, any two intervals i and i′ satisfy the interval trichotomy, that is, exactly one of the following three properties holds:
a. i and i′ overlap,
b. i is to the left of i′ (i.e., i. high < i′. low), c. i is to the right of i′ (i.e., i′. high < i. low).
An interval tree is a red-black tree that maintains a dynamic set of
elements, with each element x containing an interval x. int. Interval trees
support the following operations:
INTERVAL-INSERT( T, x) adds the element x, whose int attribute is assumed to contain an interval, to the interval tree T.
INTERVAL-DELETE( T, x) removes the element x from the interval tree T.
INTERVAL-SEARCH( T, i) returns a pointer to an element x in the interval tree T such that x. int overlaps interval i, or a pointer to the sentinel T. nil if no such element belongs to the set.
Figure 17.4 shows how an interval tree represents a set of intervals. The four-step method from Section 17.2 will guide our design of an interval tree and the operations that run on it.
Step 1: Underlying data structure
A red-black tree serves as the underlying data structure. Each node x
contains an interval x. int. The key of x is the low endpoint, x. int. low, of the interval. Thus, an inorder tree walk of the data structure lists the
intervals in sorted order by low endpoint.
Figure 17.4 An interval tree. (a) A set of 10 intervals, shown sorted bottom to top by left endpoint. (b) The interval tree that represents them. Each node x contains an interval, shown above the dashed line, and the maximum value of any interval endpoint in the subtree rooted at x, shown below the dashed line. An inorder tree walk of the tree lists the nodes in sorted order by left endpoint.
Step 2: Additional information
In addition to the intervals themselves, each node x contains a value x. max, which is the maximum value of any interval endpoint stored in
the subtree rooted at x.
Step 3: Maintaining the information
We must verify that insertion and deletion take O(lg n) time on an interval tree of n nodes. It is simple enough to determine x. max in O(1)
time, given interval x. int and the max values of node x’s children: x. max = max { x. int. high, x. left. max, x. right. max}.
Thus, by Theorem 17.1, insertion and deletion run in O(lg n) time. In fact, you can use either Exercise 17.2-3 or 17.3-1 to show how to update
all the max attributes that change after a rotation in just O(1) time.
Step 4: Developing new operations
The only new operation is INTERVAL-SEARCH( T, i), which finds a
node in tree T whose interval overlaps interval i. If there is no interval in the tree that overlaps i, the procedure returns a pointer to the sentinel
T. nil.
INTERVAL-SEARCH( T, i)
1 x = T. root
2 while x ≠ T. nil and i does not overlap x. int 3
if x. left ≠ T. nil and x. left. max ≥ i. low 4
x = x. left // overlap in left subtree or no overlap in right subtree
5
else x
=// no overlap in left subtree
x. right
6 return x
The search for an interval that overlaps i starts at the root of the tree
and proceeds downward. It terminates when either it finds an
overlapping interval or it reaches the sentinel T. nil. Since each iteration of the basic loop takes O(1) time, and since the height of an n-node red-black tree is O(lg n), the INTERVAL-SEARCH procedure takes O(lg n) time.
Before we see why INTERVAL-SEARCH is correct, let’s examine
how it works on the interval tree in Figure 17.4. Let’s look for an interval that overlaps the interval i = [22, 25]. Begin with x as the root, which contains [16, 21] and does not overlap i. Since x. left. max = 23 is greater than i. low = 22, the loop continues with x as the left child of the root—the node containing [8, 9], which also does not overlap i. This
time, x. left. max = 10 is less than i. low = 22, and so the loop continues with the right child of x as the new x. Because the interval [15, 23]
stored in this node overlaps i, the procedure returns this node.
Now let’s try an unsuccessful search, for an interval that overlaps i =
[11, 14] in the interval tree of Figure 17.4. Again, begin with x as the root. Since the root’s interval [16, 21] does not overlap i, and since x. left. max = 23 is greater than i. low = 11, go left to the node containing
[8, 9]. Interval [8, 9] does not overlap i, and x. left. max = 10 is less than i. low = 11, and so the search goes right. (No interval in the left subtree overlaps i.) Interval [15, 23] does not overlap i, and its left child is T. nil, so again the search goes right, the loop terminates, and INTERVAL-SEARCH returns the sentinel T. nil.
To see why INTERVAL-SEARCH is correct, we must understand
why it suffices to examine a single path from the root. The basic idea is
that at any node x, if x. int does not overlap i, the search always proceeds in a safe direction: the search will definitely find an overlapping interval
if the tree contains one. The following theorem states this property more
precisely.
Theorem 17.2
Any execution of INTERVAL-SEARCH( T, i) either returns a node
whose interval overlaps i, or it returns T. nil and the tree T contains no node whose interval overlaps i.
Proof The while loop of lines 2–5 terminates when either x = T. nil or i overlaps x. int. In the latter case, it is certainly correct to return x.
Therefore, we focus on the former case, in which the while loop
terminates because x = T. nil, which is the node that INTERVAL-SEARCH returns.
We’ll prove that if the procedure returns T. nil, then it did not miss
any intervals in T that overlap i. The idea is to show that whether the search goes left in line 4 or right in line 5, it always heads toward a node
containing an interval overlapping i, if any such interval exists. In particular, we’ll prove that
1. If the search goes left in line 4, then the left subtree of node x
contains an interval that overlaps i or the right subtree of x
contains no interval that overlaps i. Therefore, even if x’s left subtree contains no interval that overlaps i but the search goes
left, it does not make a mistake, because x’s right subtree does
not contain an interval overlapping i, either.
2. If the search goes right in line 5, then the left subtree of x
contains no interval that overlaps i. Thus, if the search goes
right, it does not make a mistake.
For both cases, we rely on the interval trichotomy. Let’s start with
the case where the search goes right, whose proof is simpler. By the tests
in line 3, we know that x. left = T. nil or x. left. max < i. low. If x. left =
T. nil, then x’s left subtree contains no interval that overlaps i, since it contains no intervals at all. Now suppose that x. left ≠ T. nil, so that we must have x. left. max < i. low. Consider any interval i′ in x’s left subtree.
Because x. left. max is the maximum endpoint in x’s left subtree, we have i′. high ≤ x. left. max. Thus, as Figure 17.5(a) shows, i′. high ≤ x. left. max
< i. low.
By the interval trichotomy, therefore, intervals i and i′ do not overlap, and so x’s left subtree contains no interval that overlaps i.
Figure 17.5 Intervals in the proof of Theorem 17.2. The value of x. left. max is shown in each case as a dashed line. (a) The search goes right. No interval i′ in x’s left subtree can overlap i. (b) The search goes left. The left subtree of x contains an interval that overlaps i (situation not shown), or x’s left subtree contains an interval i′ such that i′. high = x. left. max. Since i does not overlap i′, neither does it overlap any interval i″ in x’s right subtree, since i′. low ≤ i″. low.
Now we examine the case in which the search goes left. If the left
subtree of node x contains an interval that overlaps i, we’re done, so let’s
assume that no node in x’s left subtree overlaps i. We need to show that in this case, no node in x’s right subtree overlaps i, so that going left will not miss any overlaps in x’s right subtree. By the tests in line 3, the left
subtree of x is not empty and x. left. max ≥ i. low. By the definition of the max attribute, x’s left subtree contains some interval i′ such that i′. high = x. left. max
≥ i. low,
as illustrated in Figure 17.5(b). Since i′ is in x’s left subtree, it does not overlap i, and since i′. high ≥ i. low, the interval trichotomy tells us that i. high < i′. low. Now we bring in the property that interval trees are keyed on the low endpoints of intervals. Because i′ is in x’s left subtree, we have i′. low ≤ x. int. low. Now consider any interval i″ in x’s right subtree, so that x. int. low ≤ i″. low. Putting inequalities together, we get i. high < i′. low
≤ x. int. low
≤ i″. low.
Because i. high < i″. low, the interval trichotomy tells us that i and i″ do not overlap. Since we chose i″ as any interval in x’s right subtree, no node in x’s right subtree overlaps i.
▪
Thus, the INTERVAL-SEARCH procedure works correctly.
Exercises
17.3-1
Write pseudocode for LEFT-ROTATE that operates on nodes in an
interval tree and updates all the max attributes that change in O(1) time.
17.3-2
Describe an efficient algorithm that, given an interval i, returns an interval overlapping i that has the minimum low endpoint, or T. nil if no such interval exists.
Given an interval tree T and an interval i, describe how to list all intervals in T that overlap i in O(min { n, k lg n}) time, where k is the number of intervals in the output list. ( Hint: One simple method makes
several queries, modifying the tree between queries. A slightly more
complicated method does not modify the tree.)
17.3-4
Suggest modifications to the interval-tree procedures to support the new
operation INTERVAL-SEARCH-EXACTLY( T, i), where T is an
interval tree and i is an interval. The operation should return a pointer
to a node x in T such that x. int. low = i. low and x. int. high = i. high, or T. nil if T contains no such node. All operations, including INTERVAL-SEARCH-EXACTLY, should run in O(lg n) time on an n-node interval tree.
17.3-5
Show how to maintain a dynamic set Q of numbers that supports the
operation MIN-GAP, which gives the absolute value of the difference of
the two closest numbers in Q. For example, if we have Q = {1, 5, 9, 15, 18, 22}, then MIN-GAP( Q) returns 3, since 15 and 18 are the two
closest numbers in Q. Make the operations INSERT, DELETE,
SEARCH, and MIN-GAP as efficient as possible, and analyze their
running times.
★ 17.3-6
VLSI databases commonly represent an integrated circuit as a list of
rectangles. Assume that each rectangle is rectilinearly oriented (sides
parallel to the x- and y-axes), so that each rectangle is represented by four values: its minimum and maximum x- and y-coordinates. Give an
O( n lg n)-time algorithm to decide whether a set of n rectangles so represented contains two rectangles that overlap. Your algorithm need
not report all intersecting pairs, but it must report that an overlap exists
if one rectangle entirely covers another, even if the boundary lines do
not intersect. ( Hint: Move a “sweep” line across the set of rectangles.)
17-1 Point of maximum overlap
You wish to keep track of a point of maximum overlap in a set of
intervals—a point with the largest number of intervals in the set that
overlap it.
a. Show that there is always a point of maximum overlap that is an
endpoint of one of the intervals.
b. Design a data structure that efficiently supports the operations
INTERVAL-INSERT, INTERVAL-DELETE, and FIND-POM,
which returns a point of maximum overlap. ( Hint: Keep a red-black
tree of all the endpoints. Associate a value of +1 with each left
endpoint, and associate a value of −1 with each right endpoint.
Augment each node of the tree with some extra information to
maintain the point of maximum overlap.)
17-2 Josephus permutation
We define the Josephus problem as follows. A group of n people form a
circle, and we are given a positive integer m ≤ n. Beginning with a designated first person, proceed around the circle, removing every m th
person. After each person is removed, counting continues around the
circle that remains. This process continues until nobody remains in the
circle. The order in which the people are removed from the circle defines
the ( n, m)- Josephus permutation of the integers 1, 2, … , n. For example, the (7, 3)-Josephus permutation is 〈3, 6, 2, 7, 5, 1, 4〉.
a. Suppose that m is a constant. Describe an O( n)-time algorithm that, given an integer n, outputs the ( n, m)-Josephus permutation.
b. Suppose that m is not necessarily a constant. Describe an O( n lg n)-
time algorithm that, given integers n and m, outputs the ( n, m)-
Josephus permutation.
Chapter notes
In their book, Preparata and Shamos [364] describe several of the interval trees that appear in the literature, citing work by H.
Edelsbrunner (1980) and E. M. McCreight (1981). The book details an
interval tree that, given a static database of n intervals, allows us to enumerate all k intervals that overlap a given query interval in O( k + lg n) time.
B-trees are balanced search trees designed to work well on disk drives or
other direct-access secondary storage devices. B-trees are similar to red-
black trees (Chapter 13), but they are better at minimizing the number of operations that access disks. (We often say just “disk” instead of
“disk drive.”) Many database systems use B-trees, or variants of B-trees,
to store information.
B-trees differ from red-black trees in that B-tree nodes may have
many children, from a few to thousands. That is, the “branching factor”
of a B-tree can be quite large, although it usually depends on
characteristics of the disk drive used. B-trees are similar to red-black
trees in that every n-node B-tree has height O(lg n), so that B-trees can implement many dynamic-set operations in O(lg n) time. But a B-tree has a larger branching factor than a red-black tree, so the base of the
logarithm that expresses its height is larger, and hence its height can be
considerably lower.
B-trees generalize binary search trees in a natural manner. Figure
18.1 shows a simple B-tree. If an internal B-tree node x contains x. n
keys, then x has x. n + 1 children. The keys in node x serve as dividing points separating the range of keys handled by x into x. n + 1 subranges, each handled by one child of x. A search for a key in a B-tree makes an
( x. n + 1)-way decision based on comparisons with the x. n keys stored at node x. An internal node contains pointers to its children, but a leaf node does not.
Section 18.1 gives a precise definition of B-trees and proves that the height of a B-tree grows only logarithmically with the number of nodes
it contains. Section 18.2 describes how to search for a key and insert a key into a B-tree, and Section 18.3 discusses deletion. Before proceeding, however, we need to ask why we evaluate data structures
designed to work on a disk drive differently from data structures
designed to work in main random-access memory.
Figure 18.1 A B-tree whose keys are the consonants of English. An internal node x containing x. n keys has x. n + 1 children. All leaves are at the same depth in the tree. The blue nodes are examined in a search for the letter R.
Data structures on secondary storage
Computer systems take advantage of various technologies that provide
memory capacity. The main memory of a computer system normally
consists of silicon memory chips. This technology is typically more than
an order of magnitude more expensive per bit stored than magnetic
storage technology, such as tapes or disk drives. Most computer systems
also have secondary storage based on solid-state drives (SSDs) or
magnetic disk drives. The amount of such secondary storage often
exceeds the amount of primary memory by one to two orders of
magnitude. SSDs have faster access times than magnetic disk drives,
which are mechanical devices. In recent years, SSD capacities have
increased while their prices have decreased. Magnetic disk drives
typically have much higher capacities than SSDs, and they remain a
more cost-effective means for storing massive amounts of information.
Disk drives that store several terabytes1 can be found for under $100.
Figure 18.2 shows a typical disk drive. The drive consists of one or more platters, which rotate at a constant speed around a common
spindle. A magnetizable material covers the surface of each platter. The
drive reads and writes each platter by a head at the end of an arm. The arms can move their heads toward or away from the spindle. The
surface that passes underneath a given head when it is stationary is
called a track.
Although disk drives are cheaper and have higher capacity than
main memory, they are much, much slower because they have moving
mechanical parts. The mechanical motion has two components: platter
rotation and arm movement. As of this writing, commodity disk drives
rotate at speeds of 5400–15,000 revolutions per minute (RPM). Typical
speeds are 15,000 RPM in server-grade drives, 7200 RPM in drives for
desktops, and 5400 RPM in drives for laptops. Although 7200 RPM
may seem fast, one rotation takes 8.33 milliseconds, which is over 5
orders of magnitude longer than the 50 nanosecond access times (more
or less) commonly found for main memory. In other words, if a
computer waits a full rotation for a particular item to come under the
read/write head, it could access main memory more than 100,000 times
during that span. The average wait is only half a rotation, but still, the
difference in access times for main memory compared with disk drives is
enormous. Moving the arms also takes some time. As of this writing,
average access times for commodity disk drives are around 4
milliseconds.
Figure 18.2 A typical magnetic disk drive. It consists of one or more platters covered with a magnetizable material (two platters are shown here) that rotate around a spindle. Each platter is read and written with a head, shown in red, at the end of an arm. Arms rotate around a common pivot axis. A track, drawn in blue, is the surface that passes beneath the read/write head when the head is stationary.
In order to amortize the time spent waiting for mechanical
movements, also known as latency, disk drives access not just one item
but several at a time. Information is divided into a number of equal-
sized blocks of bits that appear consecutively within tracks, and each disk read or write is of one or more entire blocks. 2 Typical disk drives have block sizes running from 512 to 4096 bytes. Once the read/write
head is positioned correctly and the platter has rotated to the beginning
of the desired block, reading or writing a magnetic disk drive is entirely
electronic (aside from the rotation of the platter), and the disk drive can
quickly read or write large amounts of data.
Often, accessing a block of information and reading it from a disk
drive takes longer than processing all the information read. For this
reason, in this chapter we’ll look separately at the two principal
components of the running time:
the number of disk accesses, and
the CPU (computing) time.
We measure the number of disk accesses in terms of the number of
blocks of information that need to be read from or written to the disk
drive. Although disk-access time is not constant—it depends on the distance between the current track and the desired track and also on the
initial rotational position of the platters—the number of blocks read or
written provides a good first-order approximation of the total time
spent accessing the disk drive.
In a typical B-tree application, the amount of data handled is so
large that all the data do not fit into main memory at once. The B-tree
algorithms copy selected blocks from disk into main memory as needed
and write back onto disk the blocks that have changed. B-tree
algorithms keep only a constant number of blocks in main memory at
any time, and thus the size of main memory does not limit the size of B-
trees that can be handled.
B-tree procedures need to be able to read information from disk into
main memory and write information from main memory to disk.
Consider some object x. If x is currently in the computer’s main memory, then the code can refer to the attributes of x as usual: x. key, for example. If x resides on disk, however, then the procedure must
perform the operation DISK-READ( x) to read the block containing
object x into main memory before it can refer to x’s attributes. (Assume that if x is already in main memory, then DISK-READ( x) requires no
disk accesses: it is a “no-op.”) Similarly, procedures call DISK-
WRITE( x) to save any changes that have been made to the attributes of
object x by writing to disk the block containing x. Thus, the typical pattern for working with an object is as follows:
x = a pointer to some object
DISK-READ( x)
operations that access and/or modify the attributes of x
DISK-WRITE( x) // omitted if no attributes of x were changed
other operations that access but do not modify attributes of x
The system can keep only a limited number of blocks in main memory
at any one time. Our B-tree algorithms assume that the system
automatically flushes from main memory blocks that are no longer in
use.
Since in most systems the running time of a B-tree algorithm
depends primarily on the number of DISK-READ and DISK-WRITE
operations it performs, we typically want each of these operations to
read or write as much information as possible. Thus, a B-tree node is
usually as large as a whole disk block, and this size limits the number of
children a B-tree node can have.
Figure 18.3 A B-tree of height 2 containing over one billion keys. Shown inside each node x is x. n, the number of keys in x. Each internal node and leaf contains 1000 keys. This B-tree has 1001 nodes at depth 1 and over one million leaves at depth 2.
Large B-trees stored on disk drives often have branching factors
between 50 and 2000, depending on the size of a key relative to the size
of a block. A large branching factor dramatically reduces both the
height of the tree and the number of disk accesses required to find any
key. Figure 18.3 shows a B-tree with a branching factor of 1001 and height 2 that can store over one billion keys. Nevertheless, if the root
node is kept permanently in main memory, at most two disk accesses
suffice to find any key in this tree.
To keep things simple, let’s assume, as we have for binary search trees
and red-black trees, that any satellite information associated with a key
resides in the same node as the key. In practice, you might actually store
with each key just a pointer to another disk block containing the
satellite information for that key. The pseudocode in this chapter
implicitly assumes that the satellite information associated with a key, or
the pointer to such satellite information, travels with the key whenever the key is moved from node to node. A common variant on a B-tree,
known as a B+- tree, stores all the satellite information in the leaves and stores only keys and child pointers in the internal nodes, thus
maximizing the branching factor of the internal nodes.
A B-tree T is a rooted tree with root T. root having the following properties:
1. Every node x has the following attributes:
a. x. n, the number of keys currently stored in node x,
b. the x. n keys themselves, x. key 1, x. key 2, … , x. keyx. n, stored in monotonically increasing order, so that x. key 1 ≤ x. key 2 ≤ ⋯ ≤
x. keyx. n,
c. x. leaf, a boolean value that is TRUE if x is a leaf and FALSE
if x is an internal node.
2. Each internal node x also contains x. n + 1 pointers x. c 1, x. c 2, …
, x. cx. n+1 to its children. Leaf nodes have no children, and so their ci attributes are undefined.
3. The keys x. keyi separate the ranges of keys stored in each subtree: if ki is any key stored in the subtree with root x. ci, then k 1 ≤ x. key 1 ≤ k 2 ≤ x. key 2 ≤ ⋯ ≤ x. keyx. n ≤ kx. n+1.
4. All leaves have the same depth, which is the tree’s height h.
5. Nodes have lower and upper bounds on the number of keys they
can contain, expressed in terms of a fixed integer t ≥ 2 called the
minimum degree of the B-tree:
a. Every node other than the root must have at least t − 1 keys.
Every internal node other than the root thus has at least t
children. If the tree is nonempty, the root must have at least
one key.
b. Every node may contain at most 2 t − 1 keys. Therefore, an
internal node may have at most 2 t children. We say that a node
is full if it contains exactly 2 t − 1 keys. 3
The simplest B-tree occurs when t = 2. Every internal node then has
either 2, 3, or 4 children, and it is a 2-3-4 tree. In practice, however, much larger values of t yield B-trees with smaller height.
The height of a B-tree
The number of disk accesses required for most operations on a B-tree is
proportional to the height of the B-tree. The following theorem bounds
the worst-case height of a B-tree.
Figure 18.4 A B-tree of height 3 containing a minimum possible number of keys. Shown inside each node x is x. n.
Theorem 18.1
If n ≥ 1, then for any n-key B-tree T of height h and minimum degree t ≥
2,
Proof By definition, the root of a nonempty B-tree T contains at least one key, and all other nodes contain at least t − 1 keys. Let h be the height of T. Then T contains at least 2 nodes at depth 1, at least 2 t
nodes at depth 2, at least 2 t 2 nodes at depth 3, and so on, until at depth
h, it has at least 2 th−1 nodes. Figure 18.4 illustrates such a tree for h =
3. The number n of keys therefore satisfies the inequality
so that th ≤ ( n + 1)/2. Taking base- t logarithms of both sides proves the theorem.
▪
You can see the power of B-trees as compared with red-black trees.
Although the height of the tree grows as O(log n) in both cases (recall that t is a constant), for B-trees the base of the logarithm can be many
times larger. Thus, B-trees save a factor of about lg t over red-black trees
in the number of nodes examined for most tree operations. Because
examining an arbitrary node in a tree usually entails accessing the disk,
B-trees avoid a substantial number of disk accesses.
Exercises
18.1-1
Why isn’t a minimum degree of t = 1 allowed?
18.1-2
For what values of t is the tree of Figure 18.1 a legal B-tree?
18.1-3
Show all legal B-trees of minimum degree 2 that store the keys 1, 2, 3, 4,
5.
18.1-4
As a function of the minimum degree t, what is the maximum number
of keys that can be stored in a B-tree of height h?
Describe the data structure that results if each black node in a red-black
tree absorbs its red children, incorporating their children with its own.
18.2 Basic operations on B-trees
This section presents the details of the operations B-TREE-SEARCH,
B-TREE-CREATE, and B-TREE-INSERT. These procedures observe
two conventions:
The root of the B-tree is always in main memory, so that no
procedure ever needs to perform a DISK-READ on the root. If
any changes to the root node occur, however, then DISK-WRITE
must be called on the root.
Any nodes that are passed as parameters must already have had a
DISK-READ operation performed on them.
The procedures are all “one-pass” algorithms that proceed downward
from the root of the tree, without having to back up.
Searching a B-tree
Searching a B-tree is much like searching a binary search tree, except
that instead of making a binary, or “two-way,” branching decision at
each node, the search makes a multiway branching decision according
to the number of the node’s children. More precisely, at each internal
node x, the search makes an ( x. n + 1)-way branching decision.
The procedure B-TREE-SEARCH generalizes the TREE-SEARCH
procedure defined for binary search trees on page 316. It takes as input
a pointer to the root node x of a subtree and a key k to be searched for in that subtree. The top-level call is thus of the form B-TREE-SEARCH( T. root, k). If k is in the B-tree, then B-TREE-SEARCH
returns the ordered pair ( y, i) consisting of a node y and an index i such that y. keyi = k. Otherwise, the procedure returns NIL.
B-TREE-SEARCH( x, k)
2 while i ≤ x. n and k > x. keyi
3
i = i + 1
4 if i ≤ x. n and k == x. keyi
5
return ( x, i)
6 elseif x. leaf
7
returnNIL
8 else DISK-READ( x. ci)
9
return B-TREE-SEARCH( x. ci, k)
Using a linear-search procedure, lines 1–3 of B-TREE-SEARCH find
the smallest index i such that k ≤ x. keyi, or else they set i to x. n + 1.
Lines 4–5 check to see whether the search has discovered the key,
returning if it has. Otherwise, if x is a leaf, then line 7 terminates the search unsuccessfully, and if x is an internal node, lines 8–9 recurse to
search the appropriate subtree of x, after performing the necessary
DISK-READ on that child. Figure 18.1 illustrates the operation of BTREE-SEARCH. The blue nodes are those examined during a search
for the key R.
As in the TREE-SEARCH procedure for binary search trees, the
nodes encountered during the recursion form a simple path downward
from the root of the tree. The B-TREE-SEARCH procedure therefore
accesses O( h) = O(log t n) disk blocks, where h is the height of the B-tree and n is the number of keys in the B-tree. Since x. n < 2 t, the while loop of lines 2–3 takes O( t) time within each node, and the total CPU time is O( th) = O( t log tn).
Creating an empty B-tree
To build a B-tree T, first use the B-TREE-CREATE procedure on the
next page to create an empty root node and then call the B-TREE-
INSERT procedure on page 508 to add new keys. Both of these
procedures use an auxiliary procedure ALLOCATE-NODE, whose
pseudocode we omit and which allocates one disk block to be used as a
new node in O(1) time. A node created by ALLOCATE-NODE requires
no DISK-READ, since there is as yet no useful information stored on
the disk for that node. B-TREE-CREATE requires O(1) disk operations
and O(1) CPU time.
B-TREE-CREATE( T)
1 x = ALLOCATE-NODE()
2 x. leaf = TRUE
3 x. n = 0
4 DISK-WRITE( x)
5 T. root = x
Inserting a key into a B-tree
Inserting a key into a B-tree is significantly more complicated than
inserting a key into a binary search tree. As with binary search trees,
you search for the leaf position at which to insert the new key. With a B-
tree, however, you cannot simply create a new leaf node and insert it, as
the resulting tree would fail to be a valid B-tree. Instead, you insert the
new key into an existing leaf node. Since you cannot insert a key into a
leaf node that is full, you need an operation that splits a full node y (having 2 t − 1 keys) around its median key y. keyt into two nodes having only t − 1 keys each. The median key moves up into y’s parent to identify the dividing point between the two new trees. But if y’s parent is
also full, you must split it before you can insert the new key, and thus
you could end up splitting full nodes all the way up the tree.
To avoid having to go back up the tree, just split every full node you
encounter as you go down the tree. In this way, whenever you need to
split a full node, you are assured that its parent is not full. Inserting a
key into a B-tree then requires only a single pass down the tree from the
root to a leaf.
Splitting a node in a B-tree
The procedure B-TREE-SPLIT-CHILD on the facing page takes as
input a nonfull internal node x (assumed to reside in main memory) and
an index i such that x. ci (also assumed to reside in main memory) is a full child of x. The procedure splits this child in two and adjusts x so that it has an additional child. To split a full root, you first need to make
the root a child of a new empty root node, so that you can use B-TREE-
SPLIT-CHILD. The tree thus grows in height by 1: splitting is the only
means by which the tree grows taller.
B-TREE-SPLIT-CHILD( x, i)
1 y = x. ci
// full node to split
2 z = ALLOCATE-NODE()
// z will take half of y
3 z. leaf = y. leaf
4 z. n = t − 1
5 for j = 1 to t − 1
// z gets y’s greatest keys …
6
z. keyj = y. keyj+ t
7 if not y. leaf
8
for j = 1 to t
// … and its corresponding children
9
z. cj = y. cj+ t
10 y. n = t − 1
// y keeps t − 1 keys
11 for j = x. n + 1 downto i + 1
// shift x’s children to the right …
12
x. cj+1 = x. cj
13 x. ci+1 = z
// … to make room for z as a child
14 for j = x. n downto i
// shift the corresponding keys in x
15
x. keyj+1 = x. keyj
16 x. keyi = y. keyt
// insert y’s median key
17 x. n = x. n + 1
// x has gained a child
18 DISK-WRITE( y)
19 DISK-WRITE( z)
20 DISK-WRITE( x)
Figure 18.5 illustrates how a node splits. B-TREE-SPLIT-CHILD
splits the full node y = x. ci about its median key ( S in the figure), which moves up into y’s parent node x. Those keys in y that are greater than
the median key move into a new node z, which becomes a new child of
x. B-TREE-SPLIT-CHILD works by straightforward cutting and
pasting. Node x is the parent of the node y being split, which is x’s i th child (set in line 1). Node y originally has 2 t children and 2 t − 1 keys, but splitting reduces y to t children and t − 1 keys. The t largest children and t − 1 keys of node y move over to node z, which becomes a new child of x, positioned just after y in x’s table of children. The median key of y moves up to become the key in node x that separates the pointers to nodes y and z.
Lines 2–9 create node z and give it the largest t − 1 keys and, if y and z are internal nodes, the corresponding t children of y. Line 10 adjusts the key count for y. Then, lines 11–17 shift keys and child pointers in x
to the right in order to make room for x’s new child, insert z as a new child of x, move the median key from y up to x in order to separate y from z, and adjust x’s key count. Lines 18–20 write out all modified disk blocks. The CPU time used by B-TREE-SPLIT-CHILD is Θ( t), due to
the for loops in lines 5–6 and 8–9. (The for loops in lines 11–12 and 14–
15 also run for O( t) iterations.) The procedure performs O(1) disk operations.
Figure 18.5 Splitting a node with t = 4. Node y = x. ci splits into two nodes, y and z, and the median key S of y moves up into y’s parent.
Inserting a key into a B-tree in a single pass down the tree
Inserting a key k into a B-tree T of height h requires just a single pass down the tree and O( h) disk accesses. The CPU time required is O( th) =
O( t log t n). The B-TREE-INSERT procedure uses B-TREE-SPLIT-CHILD to guarantee that the recursion never descends to a full node. If
the root is full, B-TREE-INSERT splits it by calling the procedure B-
TREE-SPLIT-ROOT on the facing page.
B-TREE-INSERT( T, k)
1 r = T. root
2 if r. n == 2 t − 1
3
s = B-TREE-SPLIT-ROOT( T)
4
B-TREE-INSERT-NONFULL( s, k)
5 else B-TREE-INSERT-NONFULL( r, k)
B-TREE-INSERT works as follows. If the root is full, then line 3
calls B-TREE-SPLIT-ROOT in line 3 to split it. A new node s (with
two children) becomes the root and is returned by B-TREE-SPLIT-
ROOT. Splitting the root, illustrated in Figure 18.6, is the only way to increase the height of a B-tree. Unlike a binary search tree, a B-tree
increases in height at the top instead of at the bottom. Regardless of
whether the root split, B-TREE-INSERT finishes by calling B-TREE-
INSERT-NONFULL to insert key k into the tree rooted at the nonfull
root node, which is either the new root (the call in line 4) or the original
root (the call in line 5).
Figure 18.6 Splitting the root with t = 4. Root node r splits in two, and a new root node s is created. The new root contains the median key of r and has the two halves of r as children. The B-tree grows in height by one when the root is split. A B-tree’s height increases only when the root splits.
B-TREE-SPLIT-ROOT( T)
1 s = ALLOCATE-NODE()
2 s. leaf = FALSE
3 s. n = 0
4 s. c 1 = T. root
5 T. root = s
6 B-TREE-SPLIT-CHILD( s, 1)
7 return s
The auxiliary procedure B-TREE-INSERT-NONFULL on page 511
inserts key k into node x, which is assumed to be nonfull when the procedure is called. B-TREEINSERT-NONFULL recurses as
necessary down the tree, at all times guaranteeing that the node to
which it recurses is not full by calling B-TREE-SPLIT-CHILD as
necessary. The operation of B-TREE-INSERT and the recursive
operation of B-TREE-INSERT-NONFULL guarantee that this
assumption is true.
Figure 18.7 illustrates the various cases of how B-TREE-INSERT-NONFULL inserts a key into a B-tree. Lines 3–8 handle the case in
which x is a leaf node by inserting key k into x, shifting to the right all
keys in x that are greater than k. If x is not a leaf node, then k should go into the appropriate leaf node in the subtree rooted at internal node x.
Lines 9–11 determine the child x. ci to which the recursion descends.
Line 13 detects whether the recursion would descend to a full child, in
which case line 14 calls B-TREE-SPLIT-CHILD to split that child into
two nonfull children, and lines 15–16 determine which of the two
children is the correct one to descend to. (Note that DISK-READ( x. ci)
is not needed after line 16 increments i, since the recursion descends in
this case to a child that was just created by B-TREE-SPLIT-CHILD.)
The net effect of lines 13–16 is thus to guarantee that the procedure
never recurses to a full node. Line 17 then recurses to insert k into the
appropriate subtree.
Figure 18.7 Inserting keys into a B-tree. The minimum degree t for this B-tree is 3, so that a node can hold at most 5 keys. Blue nodes are modified by the insertion process. (a) The initial tree for this example. (b) The result of inserting B into the initial tree. This case is a simple insertion into a leaf node. (c) The result of inserting Q into the previous tree. The node RST U V splits into two nodes containing RS and U V, the key T moves up to the root, and Q is inserted in the leftmost of the two halves (the RS node). (d) The result of inserting L into the previous tree. The root splits right away, since it is full, and the B-tree grows in height by one. Then L is inserted into the leaf containing JK. (e) The result of inserting F into the previous tree. The node ABCDE splits before F is inserted into the rightmost of the two halves (the DE node).
B-TREE-INSERT-NONFULL( x, k)
2 if x. leaf
// inserting into a leaf?
3
while i ≥ 1 and k < x. keyi // shift keys in x to make room for k 4
x. keyi+1 = x. keyi
5
i = i − 1
6
x. keyi+1 = k
// insert key k in x
7
x. n = x. n + 1
// now x has 1 more key
8
DISK-WRITE( x)
9 else while i ≥ 1 and k < x. keyi // find the child where k belongs 10
i = i − 1
11
i = i + 1
12
DISK-READ( x. ci)
13
if x. ci. n == 2 t − 1
// split the child if it’s full
14
B-TREE-SPLIT-CHILD( x, i)
15
if k > x. keyi
// does k go into x. ci or x. ci+1?
16
i = i + 1
17
B-TREE-INSERT-NONFULL( x. ci, k)
For a B-tree of height h, B-TREE-INSERT performs O( h) disk accesses, since only O(1) DISK-READ and DISK-WRITE operations
occur at each level of the tree. The total CPU time used is O( t) in each level of the tree, or O( th) = O( t log t n) overall. Since B-TREE-INSERT-NONFULL is tail-recursive, you can instead implement it with a while
loop, thereby demonstrating that the number of blocks that need to be
in main memory at any time is O(1).
Exercises
18.2-1
Show the results of inserting the keys
F, S, Q, K, C, L, H, T, V, W, M, R, N, P, A, B, X, Y, D, Z, E
in order into an empty B-tree with minimum degree 2. Draw only the
configurations of the tree just before some node must split, and also
18.2-2
Explain under what circumstances, if any, redundant DISK-READ or
DISK-WRITE operations occur during the course of executing a call to
B-TREE-INSERT. (A redundant DISK-READ is a DISK-READ for a
block that is already in memory. A redundant DISK-WRITE writes to
disk a block of information that is identical to what is already stored
there.)
18.2-3
Professor Bunyan asserts that the B-TREE-INSERT procedure always
results in a B-tree with the minimum possible height. Show that the
professor is mistaken by proving that with t = 2 and the set of keys {1,
2, … , 15}, there is no insertion sequence that results in a B-tree with the
minimum possible height.
★ 18.2-4
If you insert the keys {1, 2, … , n} into an empty B-tree with minimum
degree 2, how many nodes does the final B-tree have?
18.2-5
Since leaf nodes require no pointers to children, they could conceivably
use a different (larger) t value than internal nodes for the same disk block size. Show how to modify the procedures for creating and
inserting into a B-tree to handle this variation.
18.2-6
Suppose that you implement B-TREE-SEARCH to use binary search
rather than linear search within each node. Show that this change makes
the required CPU time O(lg n), independent of how t might be chosen as a function of n.
18.2-7
Suppose that disk hardware allows you to choose the size of a disk
block arbitrarily, but that the time it takes to read the disk block is
a+ bt, where a and b are specified constants and t is the minimum degree
for a B-tree using blocks of the selected size. Describe how to choose t so as to minimize (approximately) the B-tree search time. Suggest an
optimal value of t for the case in which a = 5 milliseconds and b = 10
microseconds.
18.3 Deleting a key from a B-tree
Deletion from a B-tree is analogous to insertion but a little more
complicated, because you can delete a key from any node—not just a
leaf—and when you delete a key from an internal node, you must
rearrange the node’s children. As in insertion, you must guard against
deletion producing a tree whose structure violates the B-tree properties.
Just as a node should not get too big due to insertion, a node must not
get too small during deletion (except that the root is allowed to have
fewer than the minimum number t − 1 of keys). And just as a simple
insertion algorithm might have to back up if a node on the path to
where the key is to be inserted is full, a simple approach to deletion
might have to back up if a node (other than the root) along the path to
where the key is to be deleted has the minimum number of keys.
The procedure B-TREE-DELETE deletes the key k from the subtree
rooted at x. Unlike the procedures TREE-DELETE on page 325 and
RB-DELETE on page 348, which are given the node to delete—
presumably as the result of a prior search—B-TREE-DELETE
combines the search for key k with the deletion process. Why do we
combine search and deletion in B-TREE-DELETE? Just as B-TREE-
INSERT prevents any node from becoming overfull (having more than
2 t − 1 keys) while making a single pass down the tree, B-TREE-
DELETE prevents any node from becoming underfull (having fewer
than t − 1 keys) while also making a single pass down the tree, searching
for and ultimately deleting the key.
To prevent any node from becoming underfull, the design of B-
TREE-DELETE guarantees that whenever it calls itself recursively on a
node x, the number of keys in x is at least the minimum degree t at the time of the call. (Although the root may have fewer than t keys and a
recursive call may be made from the root, no recursive call is made on
the root.) This condition requires one more key than the minimum
required by the usual B-tree conditions, and so a key might have to be
moved from x into one of its child nodes (still leaving x with at least the minimum t − 1 keys) before a recursive call is made on that child, thus
allowing deletion to occur in one downward pass without having to
traverse back up the tree.
We describe how the procedure B-TREE-DELETE( T, k) deletes a
key k from a B-tree T instead of presenting detailed pseudocode. We examine three cases, illustrated in Figure 18.8. The cases are for when the search arrives at a leaf, at an internal node containing key k, and at
an internal node not containing key k. As mentioned above, in all three
cases node x has at least t keys (with the possible exception of when x is the root). Cases 2 and 3—when x is an internal node—guarantee this
property as the recursion descends through the B-tree.
Figure 18.8 Deleting keys from a B-tree. The minimum degree for this B-tree is t = 3, so that, other than the root, every node must have at least 2 keys. Blue nodes are those that are modified by the deletion process. (a) The B-tree of Figure 18.7(e). (b) Deletion of F, which is case 1: simple deletion from a leaf when all nodes visited during the search (other than the root) have at least t = 3 keys. (c) Deletion of M, which is case 2a: the predecessor L of M moves up to take M’s position. (d) Deletion of G, which is case 2c: push G down to make node DEGJK and then delete G from this leaf (case 1). (e) Deletion of D, which is case 3b: since the recursion cannot descend to node CL because it has only 2 keys, push P down and merge it with CL and TX to form CLP TX. Then delete D from a leaf (case 1). (e0) After (e), delete the empty root. The tree shrinks in height by 1. (f) Deletion of B, which is case 3a: C moves to fill B’s position and E
moves to fill C’s position.
Case 1: The search arrives at a leaf node x. If x contains key k, then delete k from x. If x does not contain key k, then k was not in the Btree and nothing else needs to be done.
Case 2: The search arrives at an internal node x that contains key k. Let k
= x. keyi. One of the following three cases applies, depending on the
number of keys in x. ci (the child of x that precedes k) and x. ci+1 (the child of x that follows k).
Case 2a: x. ci has at least t keys. Find the predecessor k′ of k in the subtree rooted at x. ci. Recursively delete k′ from x. ci, and replace k by k′ in x. (Key k′ can be found and deleted in a single downward pass.)
Case 2b: x. ci has t − 1 keys and x. ci+1 has at least t keys. This case is symmetric to case 2a. Find the successor k′ of k in the subtree rooted
at x. ci+1. Recursively delete k′ from x. ci+1, and replace k by k′ in x.
(Again, finding and deleting k′ can be done in a single downward
pass.)
Case 2c: Both x. ci and x. ci+1 have t − 1 keys. Merge k and all of x. ci+1
into x. ci, so that x loses both k and the pointer to x. ci+1, and x. ci now contains 2 t − 1 keys. Then free x. ci+1 and recursively delete k from x. ci.
Case 3: The search arrives at an internal node x that does not contain key
k. Continue searching down the tree while ensuring that each node
visited has at least t keys. To do so, determine the root x. ci of the appropriate subtree that must contain k, if k is in the tree at all. If x. ci has only t − 1 keys, execute case 3a or 3b as necessary to guarantee
descending to a node containing at least t keys. Then finish by
recursing on the appropriate child of x.
Case 3a: x. ci has only t − 1 keys but has an immediate sibling with at least t keys. Give x. ci an extra key by moving a key from x down into x. ci, moving a key from x. ci’s immediate left or right sibling up into x, and moving the appropriate child pointer from the sibling into x. ci.
Case 3b: x. ci and each of x. ci’s immediate siblings have t − 1 keys. (It is possible for x. ci to have either one or two siblings.) Merge x. ci with one sibling, which involves moving a key from x down into the new
merged node to become the median key for that node.
In cases 2c and 3b, if node x is the root, it could end up having no
keys. When this situation occurs, then x is deleted, and x’s only child x. c 1 becomes the new root of the tree. This action decreases the height of the tree by one and preserves the property that the root of the tree
contains at least one key (unless the tree is empty).
Since most of the keys in a B-tree are in the leaves, deletion
operations often end up deleting keys from leaves. The B-TREE-
DELETE procedure then acts in one downward pass through the tree,
without having to back up. When deleting a key in an internal node x,
however, the procedure might make a downward pass through the tree
to find the key’s predecessor or successor and then return to node x to
replace the key with its predecessor or successor (cases 2a and 2b).
Returning to node x does not require a traversal through all the levels
between x and the node containing the predecessor or successor,
however, since the procedure can just keep a pointer to x and the key
position within x and put the predecessor or successor key directly
there.
Although this procedure seems complicated, it involves only O( h) disk operations for a B-tree of height h, since only O(1) calls to DISK-READ and DISK-WRITE are made between recursive invocations of
the procedure. The CPU time required is O( th) = O( t log tn).
Exercises
18.3-1
Show the results of deleting C, P, and V, in order, from the tree of
18.3-2
Write pseudocode for B-TREE-DELETE.
Problems
18-1 Stacks on secondary storage
Consider implementing a stack in a computer that has a relatively small
amount of fast primary memory and a relatively large amount of slower
disk storage. The operations PUSH and POP work on single-word
values. The stack can grow to be much larger than can fit in memory,
and thus most of it must be stored on disk.
A simple, but inefficient, stack implementation keeps the entire stack
on disk. Maintain in memory a stack pointer, which is the disk address
of the top element on the stack. Indexing block numbers and word
offsets within blocks from 0, if the pointer has value p, the top element is the ( p mod m)th word on block ⌊ p/ m⌊ of the disk, where m is the number of words per block.
To implement the PUSH operation, increment the stack pointer,
read the appropriate block into memory from disk, copy the element to
be pushed to the appropriate word on the block, and write the block
back to disk. A POP operation is similar. Read in the appropriate block
from disk, save the top of the stack, decrement the stack pointer, and
return the saved value. You need not write back the block, since it was
not modified, and the word in the block that contained the popped
value is ignored.
As in the analyses of B-tree operations, two costs matter: the total
number of disk accesses and the total CPU time. A disk access also
incurs a cost in CPU time. In particular, any disk access to a block of m
words incurs charges of one disk access and Θ( m) CPU time.
a. Asymptotically, what is the worst-case number of disk accesses for n
stack operations using this simple implementation? What is the CPU
time for n stack operations? Express your answer in terms of m and n for this and subsequent parts.
Now consider a stack implementation in which you keep one block of
the stack in memory. (You also maintain a small amount of memory to
record which block is currently in memory.) You can perform a stack
operation only if the relevant disk block resides in memory. If necessary,
you can write the block currently in memory to the disk and read the
new block from the disk into memory. If the relevant disk block is
already in memory, then no disk accesses are required.
b. What is the worst-case number of disk accesses required for n PUSH
operations? What is the CPU time?
c. What is the worst-case number of disk accesses required for n stack operations? What is the CPU time?
Suppose that you now implement the stack by keeping two blocks in
memory (in addition to a small number of words for bookkeeping).
d. Describe how to manage the stack blocks so that the amortized
number of disk accesses for any stack operation is O(1/ m) and the
amortized CPU time for any stack operation is O(1).
18-2 Joining and splitting 2-3-4 trees
The join operation takes two dynamic sets S′ and S″ and an element x such that x′. key < x. key < x″. key for any x′ ∈ S′ and x″ ∈ S″. It returns a set S = S′ ∪ { x} ∪ S″. The split operation is like an “inverse” join: given a dynamic set S and an element x ∈ S, it creates a set S′ that consists of all elements in S − { x} whose keys are less than x. key and another set S″ that consists of all elements in S − { x} whose keys are greater than x. key. This problem investigates how to implement these operations on 2-3-4 trees (B-trees with t = 2). Assume for convenience
that elements consist only of keys and that all key values are distinct.
a. Show how to maintain, for every node x of a 2-3-4 tree, the height of the subtree rooted at x as an attribute x. height. Make sure that your implementation does not affect the asymptotic running times of
searching, insertion, and deletion.
b. Show how to implement the join operation. Given two 2-3-4 trees T′
and T″ and a key k, the join operation should run in O(1 + | h′ − h″|) time, where h′ and h″ are the heights of T′ and T″, respectively.
c. Consider the simple path p from the root of a 2-3-4 tree T to a given key k, the set S′ of keys in T that are less than k, and the set S″ of keys in T that are greater than k. Show that p breaks S′ into a set of trees and a set of keys
such that
for i
= 1, 2, … , m and any keys
and
. What is the relationship
between the heights of
and ? Describe how p breaks S″ into sets
of trees and keys.
d. Show how to implement the split operation on T. Use the join
operation to assemble the keys in S′ into a single 2-3-4 tree T′ and the keys in S″ into a single 2-3-4 tree T″. The running time of the split
operation should be O(lg n), where n is the number of keys in T. ( Hint: The costs for joining should telescope.)
Knuth [261], Aho, Hopcroft, and Ullman [5], and Sedgewick and Wayne [402] give further discussions of balanced-tree schemes and Btrees. Comer [99] provides a comprehensive survey of B-trees. Guibas and Sedgewick [202] discuss the relationships among various kinds of balanced-tree schemes, including red-black trees and 2-3-4 trees.
In 1970, J. E. Hopcroft invented 2-3 trees, a precursor to B-trees and
2-3-4 trees, in which every internal node has either two or three children.
Bayer and McCreight [39] introduced B-trees in 1972 with no explanation of their choice of name.
Bender, Demaine, and Farach-Colton [47] studied how to make Btrees perform well in the presence of memory-hierarchy effects. Their
cache-oblivious algorithms work efficiently without explicitly knowing
the data transfer sizes within the memory hierarchy.
1 When specifying disk capacities, one terabyte is one trillion bytes, rather than 240 bytes.
2 SSDs also exhibit greater latency than main memory and access data in blocks.
3 Another common variant on a B-tree, known as a B*- tree, requires each internal node to be at least 2/3 full, rather than at least half full, as a B-tree requires.
19 Data Structures for Disjoint Sets
Some applications involve grouping n distinct elements into a collection
of disjoint sets—sets with no elements in common. These applications
often need to perform two operations in particular: finding the unique
set that contains a given element and uniting two sets. This chapter
explores methods for maintaining a data structure that supports these
operations.
Section 19.1 describes the operations supported by a disjoint-set data structure and presents a simple application. Section 19.2 looks at a simple linked-list implementation for disjoint sets. Section 19.3 presents a more efficient representation using rooted trees. The running time
using the tree representation is theoretically superlinear, but for all
practical purposes it is linear. Section 19.4 defines and discusses a very quickly growing function and its very slowly growing inverse, which
appears in the running time of operations on the tree-based
implementation, and then, by a complex amortized analysis, proves an
upper bound on the running time that is just barely superlinear.
A disjoint-set data structure maintains a collection S = { S 1, S 2, … , Sk}
of disjoint dynamic sets. To identify each set, choose a representative, which is some member of the set. In some applications, it doesn’t matter
which member is used as the representative; it matters only that if you
ask for the representative of a dynamic set twice without modifying the
set between the requests, you get the same answer both times. Other applications may require a prespecified rule for choosing the
representative, such as choosing the smallest member in the set (for a set
whose elements can be ordered).
As in the other dynamic-set implementations we have studied, each
element of a set is represented by an object. Letting x denote an object,
we’ll see how to support the following operations:
MAKE-SET( x), where x does not already belong to some other set, creates a new set whose only member (and thus representative) is x.
UNION( x, y) unites two disjoint, dynamic sets that contain x and y, say Sx and Sy, into a new set that is the union of these two sets. The representative of the resulting set is any member of Sx ∪ Sy, although many implementations of UNION specifically choose the
representative of either Sx or Sy as the new representative. Since the
sets in the collection must at all times be disjoint, the UNION
operation destroys sets Sx and Sy, removing them from the collection
S. In practice, implementations often absorb the elements of one of
the sets into the other set.
FIND-SET( x) returns a pointer to the representative of the unique set
containing x.
Throughout this chapter, we’ll analyze the running times of disjoint-
set data structures in terms of two parameters: n, the number of MAKE-
SET operations, and m, the total number of MAKE-SET, UNION, and
FIND-SET operations. Because the total number of operations m
includes the n MAKE-SET operations, m ≥ n. The first n operations are always MAKE-SET operations, so that after the first n operations, the
collection consists of n singleton sets. Since the sets are disjoint at all times, each UNION operation reduces the number of sets by 1. After n −
1 UNION operations, therefore, only one set remains, and so at most n −
1 UNION operations can occur.
An application of disjoint-set data structures
One of the many applications of disjoint-set data structures arises in
determining the connected components of an undirected graph (see
Section B.4). Figure 19.1(a), for example, shows a graph with four connected components.
The procedure CONNECTED-COMPONENTS on the following
page uses the disjoint-set operations to compute the connected
components of a graph. Once the CONNECTED-COMPONENTS
procedure has preprocessed the graph, the procedure SAME-
COMPONENT answers queries about whether two vertices belong to
the same connected component. In pseudocode, we denote the set of
vertices of a graph G by G. V and the set of edges by G. E.
The procedure CONNECTED-COMPONENTS initially places each
vertex v in its own set. Then, for each edge ( u, v), it unites the sets containing u and v. By Exercise 19.1-2, after all the edges are processed, two vertices belong to the same connected component if and only if the
objects corresponding to the vertices belong to the same set. Thus
CONNECTED-COMPONENTS computes sets in such a way that the
procedure SAME-COMPONENT can determine whether two vertices
are in the same connected component. Figure 19.1(b) illustrates how CONNECTED-COMPONENTS computes the disjoint sets.
Figure 19.1 (a) A graph with four connected components: { a, b, c, d}, { e, f, g}, { h, i}, and { j }. (b) The collection of disjoint sets after processing each edge.
CONNECTED-COMPONENTS( G)
2
MAKE-SET( v)
3 for each edge ( u, v) ∈ G. E
4
if FIND-SET( u) ≠ FIND-SET( v)
5
UNION( u, v)
SAME-COMPONENT( u, v)
1 if FIND-SET( u) == FIND-SET( v)
2
return TRUE
3 else returnFALSE
In an actual implementation of this connected-components
algorithm, the representations of the graph and the disjoint-set data
structure would need to reference each other. That is, an object
representing a vertex would contain a pointer to the corresponding
disjoint-set object, and vice versa. Since these programming details
depend on the implementation language, we do not address them further
here.
When the edges of the graph are static—not changing over time—
depth-first search can compute the connected components faster (see
Exercise 20.3-12 on page 572). Sometimes, however, the edges are added
dynamically, with the connected components updated as each edge is
added. In this case, the implementation given here can be more efficient
than running a new depth-first search for each new edge.
Exercises
19.1-1
The CONNECTED-COMPONENTS procedure is run on the
undirected graph G = ( V, E), where V = { a, b, c, d, e, f, g, h, i, j, k}, and the edges of E are processed in the order ( d, i), ( f, k), ( g, i), ( b, g), ( a, h), ( i, j), ( d, k), ( b, j), ( d, f), ( g, j), ( a, e). List the vertices in each connected component after each iteration of lines 3–5.
19.1-2
Show that after all edges are processed by CONNECTED-
COMPONENTS, two vertices belong to the same connected component
if and only if they belong to the same set.
19.1-3
During the execution of CONNECTED-COMPONENTS on an
undirected graph G = ( V, E) with k connected components, how many times is FIND-SET called? How many times is UNION called? Express
your answers in terms of | V |, | E|, and k.
19.2 Linked-list representation of disjoint sets
Figure 19.2(a) shows a simple way to implement a disjoint-set data structure: each set is represented by its own linked list. The object for
each set has attributes head, pointing to the first object in the list, and
tail, pointing to the last object. Each object in the list contains a set member, a pointer to the next object in the list, and a pointer back to the
set object. Within each linked list, the objects may appear in any order.
The representative is the set member in the first object in the list.
With this linked-list representation, both MAKE-SET and FIND-
SET require only O(1) time. To carry out MAKE-SET( x), create a new
linked list whose only object is x. For FIND-SET( x), just follow the pointer from x back to its set object and then return the member in the
object that head points to. For example, in Figure 19.2(a), the call FIND-SET( g) returns f.
Figure 19.2 (a) Linked-list representations of two sets. Set S 1 contains members d, f, and g, with representative f, and set S 2 contains members b, c, e, and h, with representative c. Each object in the list contains a set member, a pointer to the next object in the list, and a pointer back to the set object. Each set object has pointers head and tail to the first and last objects, respectively. (b) The result of UNION( g, e), which appends the linked list containing e to the linked list containing g.
The representative of the resulting set is f. The set object for e’s list, S 2, is destroyed.
A simple implementation of union
The simplest implementation of the UNION operation using the linked-
list set representation takes significantly more time than MAKE-SET or
FIND-SET. As Figure 19.2(b) shows, the operation UNION( x, y) appends y’s list onto the end of x’s list. The representative of x’s list becomes the representative of the resulting set. To quickly find where to
append y’s list, use the tail pointer for x’s list. Because all members of y’s list join x’s list, the UNION operation destroys the set object for y’s list.
The UNION operation is where this implementation pays the price for
FIND-SET taking constant time: UNION must also update the pointer
to the set object for each object originally on y’s list, which takes time
linear in the length of y’s list. In Figure 19.2, for example, the operation UNION( g, e) causes pointers to be updated in the objects for b, c, e, and h.
In fact, we can construct a sequence of m operations on n objects that
requires Θ( n 2) time. Starting with objects x 1, x 2, … , xn, execute the sequence of n MAKE-SET operations followed by n − 1 UNION
operations shown in Figure 19.3, so that m = 2 n−1. The n MAKE-SET
operations take Θ( n) time. Because the i th UNION operation updates i objects, the total number of objects updated by all n−1 UNION
operations forms an arithmetic series:
Figure 19.3 A sequence of 2 n − 1 operations on n objects that takes Θ( n 2) time, or Θ( n) time per operation on average, using the linked-list set representation and the simple implementation of UNION.
The total number of operations is 2 n−1, and so each operation on
average requires Θ( n) time. That is, the amortized time of an operation is
Θ( n).
A weighted-union heuristic
In the worst case, the above implementation of UNION requires an
average of Θ( n) time per call, because it might be appending a longer list
onto a shorter list, and the procedure must update the pointer to the set
object for each member of the longer list. Suppose instead that each list
also includes the length of the list (which can be maintained
straightforwardly with constant overhead) and that the UNION
procedure always appends the shorter list onto the longer, breaking ties arbitrarily. With this simple weighted-union heuristic, a single UNION
operation can still take Ω( n) time if both sets have Ω( n) members. As the following theorem shows, however, a sequence of m MAKE-SET,
UNION, and FIND-SET operations, n of which are MAKE-SET
operations, takes O( m + n lg n) time.
Theorem 19.1
Using the linked-list representation of disjoint sets and the weighted-
union heuristic, a sequence of m MAKE-SET, UNION, and FIND-SET
operations, n of which are MAKE-SET operations, takes O( m + n lg n) time.
Proof Because each UNION operation unites two disjoint sets, at most
n − 1 UNION operations occur over all. We now bound the total time
taken by these UNION operations. We start by determining, for each
object, an upper bound on the number of times the object’s pointer back
to its set object is updated. Consider a particular object x. Each time x’s pointer is updated, x must have started in the smaller set. The first time
x’s pointer is updated, therefore, the resulting set must have at least 2
members. Similarly, the next time x’s pointer is updated, the resulting set
must have had at least 4 members. Continuing on, for any k ≤ n, after x’s pointer has been updated ⌈lg k⌉ times, the resulting set must have at least
k members. Since the largest set has at most n members, each object’s pointer is updated at most ⌈lg n⌉ times over all the UNION operations.
Thus the total time spent updating object pointers over all UNION
operations is O( n lg n). We must also account for updating the tail pointers and the list lengths, which take only Θ(1) time per UNION
operation. The total time spent in all UNION operations is thus O( n lg
n). The time for the entire sequence of m operations follows. Each MAKE-SET and FIND-SET operation takes O(1) time, and there are
O( m) of them. The total time for the entire sequence is thus O( m + n lg n).
▪
19.2-1
Write pseudocode for MAKE-SET, FIND-SET, and UNION using the
linked-list representation and the weighted-union heuristic. Make sure to
specify the attributes that you assume for set objects and list objects.
19.2-2
Show the data structure that results and the answers returned by the
FIND-SET operations in the following program. Use the linked-list
representation with the weighted-union heuristic. Assume that if the sets
containing xi and xj have the same size, then the operation UNION( xi, xj) appends xj’s list onto xi’s list.
1for i = 1 to 16
2
MAKE-SET( xi)
3for i = 1 to 15 by 2
4
UNION( xi, xi+1)
5for i = 1 to 13 by 4
6
UNION( xi, xi+2)
7UNION( x 1, x 5)
8UNION( x 11, x 13)
9UNION( x 1, x 10)
10FIND-SET( x 2)
11FIND-SET( x 9)
19.2-3
Adapt the aggregate proof of Theorem 19.1 to obtain amortized time
bounds of O(1) for MAKE-SET and FIND-SET and O(lg n) for UNION using the linked-list representation and the weighted-union
heuristic.
19.2-4
Give a tight asymptotic bound on the running time of the sequence of
operations in Figure 19.3 assuming the linked-list representation and the weighted-union heuristic.
Professor Gompers suspects that it might be possible to keep just one
pointer in each set object, rather than two ( head and tail), while keeping the number of pointers in each list element at two. Show that the
professor’s suspicion is well founded by describing how to represent each
set by a linked list such that each operation has the same running time as
the operations described in this section. Describe also how the
operations work. Your scheme should allow for the weighted-union
heuristic, with the same effect as described in this section. ( Hint: Use the
tail of a linked list as its set’s representative.)
19.2-6
Suggest a simple change to the UNION procedure for the linked-list
representation that removes the need to keep the tail pointer to the last
object in each list. Regardless of whether the weighted-union heuristic is
used, your change should not change the asymptotic running time of the
UNION procedure. ( Hint: Rather than appending one list to another,
splice them together.)
A faster implementation of disjoint sets represents sets by rooted trees,
with each node containing one member and each tree representing one
set. In a disjoint-set forest, illustrated in Figure 19.4(a), each member points only to its parent. The root of each tree contains the
representative and is its own parent. As we’ll see, although the
straightforward algorithms that use this representation are no faster than
ones that use the linked-list representation, two heuristics—“union by
rank” and “path compression”—yield an asymptotically optimal
disjoint-set data structure.
The three disjoint-set operations have simple implementations. A
MAKE-SET operation simply creates a tree with just one node. A
FIND-SET operation follows parent pointers until it reaches the root of
the tree. The nodes visited on this simple path toward the root constitute
the find path. A UNION operation, shown in Figure 19.4(b), simply causes the root of one tree to point to the root of the other.
Figure 19.4 A disjoint-set forest. (a) Trees representing the two sets of Figure 19.2. The tree on the left represents the set { b, c, e, h}, with c as the representative, and the tree on the right represents the set { d, f, g}, with f as the representative. (b) The result of UNION ( e, g).
Heuristics to improve the running time
So far, disjoint-set forests have not improved on the linked-list
implementation. A sequence of n − 1 UNION operations could create a
tree that is just a linear chain of n nodes. By using two heuristics, however, we can achieve a running time that is almost linear in the total
number m of operations.
The first heuristic, union by rank, is similar to the weighted-union
heuristic we used with the linked-list representation. The common-sense
approach is to make the root of the tree with fewer nodes point to the
root of the tree with more nodes. Rather than explicitly keeping track of
the size of the subtree rooted at each node, however, we’ll adopt an
approach that eases the analysis. For each node, maintain a rank, which
is an upper bound on the height of the node. Union by rank makes the
root with smaller rank point to the root with larger rank during a
UNION operation.
The second heuristic, path compression, is also quite simple and
highly effective. As shown in Figure 19.5, FIND-SET operations use it to make each node on the find path point directly to the root. Path
compression does not change any ranks.
Pseudocode for disjoint-set forests
The union-by-rank heuristic requires its implementation to keep track of
ranks. With each node x, maintain the integer value x. rank, which is an upper bound on the height of x (the number of edges in the longest simple path from a descendant leaf to x). When MAKE-SET creates a
singleton set, the single node in the corresponding tree has an initial rank
of 0. Each FIND-SET operation leaves all ranks unchanged. The
UNION operation has two cases, depending on whether the roots of the
trees have equal rank. If the roots have unequal ranks, make the root
with higher rank the parent of the root with lower rank, but don’t
change the ranks themselves. If the roots have equal ranks, arbitrarily
choose one of the roots as the parent and increment its rank.
Figure 19.5 Path compression during the operation FIND-SET. Arrows and self-loops at roots are omitted. (a) A tree representing a set prior to executing FIND-SET( a). Triangles represent subtrees whose roots are the nodes shown. Each node has a pointer to its parent. (b) The same set after executing FIND-SET( a). Each node on the find path now points directly to the root.
Let’s put this method into pseudocode, appearing on the next page.
The parent of node x is denoted by x. p. The LINK procedure, a subroutine called by UNION, takes pointers to two roots as inputs. The
FIND-SET procedure with path compression, implemented recursively,
turns out to be quite simple.
The FIND-SET procedure is a two-pass method: as it recurses, it makes one pass up the find path to find the root, and as the recursion
unwinds, it makes a second pass back down the find path to update each
node to point directly to the root. Each call of FIND-SET( x) returns x. p in line 3. If x is the root, then FIND-SET skips line 2 and just returns
x. p, which is x. In this case the recursion bottoms out. Otherwise, line 2
executes, and the recursive call with parameter x. p returns a pointer to the root. Line 2 updates node x to point directly to the root, and line 3
returns this pointer.
MAKE-SET( x)
1 x. p = x
2 x. rank = 0
UNION( x, y)
1 LINK(FIND-SET( x), FIND-SET( y))
LINK( x, y)
1 if x. rank > y. rank
2
y. p = x
3 else x. p = y
4
if x. rank == y. rank
5
y. rank = y. rank + 1
FIND-SET( x)
1 if x ≠ x. p
// not the root?
2
x. p = FIND-SET( x. p)
// the root becomes the parent
3 return x. p
// return the root
Effect of the heuristics on the running time
Separately, either union by rank or path compression improves the
running time of the operations on disjoint-set forests, and combining the
two heuristics yields an even greater improvement. Alone, union by rank
yields a running time of O( m lg n) for a sequence of m operations, n of which are MAKE-SET (see Exercise 19.4-4), and this bound is tight (see
Exercise 19.3-3). Although we won’t prove it here, for a sequence of n MAKE-SET operations (and hence at most n − 1 UNION operations)
and f FIND-SET operations, the worst-case running time using only the
path-compression heuristic is Θ( n + f · (1 + log2+ f/ nn)).
Combining union by rank and path compression gives a worst-case
running time of O( m α( n)), where α( n) is a very slowly growing function, defined in Section 19.4. In any conceivable application of a disjoint-set data structure, α( n) ≤ 4, and thus, its running time is as good as linear in m for all practical purposes. Mathematically speaking, however, it is
superlinear. Section 19.4 proves this O( mα( n)) upper bound.
Exercises
19.3-1
Redo Exercise 19.2-2 using a disjoint-set forest with union by rank and
path compression. Show the resulting forest with each node including its
xi and rank.
19.3-2
Write a nonrecursive version of FIND-SET with path compression.
19.3-3
Give a sequence of m MAKE-SET, UNION, and FIND-SET
operations, n of which are MAKE-SET operations, that takes Ω( m lg n) time when using only union by rank and not path compression.
19.3-4
Consider the operation PRINT-SET( x), which is given a node x and prints all the members of x’s set, in any order. Show how to add just a
single attribute to each node in a disjoint-set forest so that PRINT-
SET( x) takes time linear in the number of members of x’s set and the asymptotic running times of the other operations are unchanged.
Assume that you can print each member of the set in O(1) time.
★ 19.3-5
Show that any sequence of m MAKE-SET, FIND-SET, and LINK
operations, where all the LINK operations appear before any of the
FIND-SET operations, takes only O( m) time when using both path compression and union by rank. You may assume that the arguments to
LINK are roots within the disjoint-set forest. What happens in the same
situation when using only path compression and not union by rank?
★ 19.4 Analysis of union by rank with path compression
As noted in Section 19.3, the combined union-by-rank and path-compression heuristic runs in O( m α( n)) time for m disjoint-set operations on n elements. In this section, we’ll explore the function α to see just how slowly it grows. Then we’ll analyze the running time using
the potential method of amortized analysis.
A very quickly growing function and its very slowly growing inverse
For integers j, k ≥ 0, we define the function Ak( j) as
where the expression
uses the functional-iteration notation
defined in equation (3.30) on page 68. Specifically, equation (3.30) gives
and
for i ≥ 1. We call the parameter
k the level of the function A.
The function Ak( j) strictly increases with both j and k. To see just how quickly this function grows, we first obtain closed-form expressions
for A 1( j) and A 2( j).
Lemma 19.2
For any integer j ≥ 1, we have A 1( j) = 2 j + 1.
Proof We first use induction on i to show that
. For the
base case,
. For the inductive step, assume that
. Then
.
Finally, we note that
.
▪
Lemma 19.3
For any integer j ≥ 1, we have A 2 ( j) = 2 j+1( j + 1) − 1.
Proof We first use induction on i to show that
.
For the base case, we have
. For the inductive
step,
assume
that
.
Then
. Finally, we note that
.
Now we can see how quickly Ak( j) grows by simply examining Ak(1) for levels k = 0, 1, 2, 3, 4. From the definition of A 0( j) and the above lemmas, we have A 0(1) = 1 + 1 = 2, A 1(1) = 2 · 1 + 1 = 3, and A 2(1) =
21+1 · (1 + 1) − 1 = 7. We also have
A 3(1) =
= A 2( A 2(1))
= A 2(7)
= 28 · 8 − 1
= 211 − 1
= 2047
and
A 4(1) =
= A 3( A 3(1))
= A 3(2047)
=
≫ A 2(2047)
= 22048 · 2048 − 1
= 22059 − 1
> 22056
= (24)514
= 16514
≫ 1080,
which is the estimated number of atoms in the observable universe. (The
symbol “≫” denotes the “much-greater-than” relation.)
We define the inverse of the function Ak( n), for integer n ≥ 0, by In words, α( n) is the lowest level k for which Ak(1) is at least n. From the above values of Ak(1), we see that
It is only for values of n so large that the term “astronomical”
understates them (greater than A 4(1), a huge number) that α( n) > 4, and so α( n) ≤ 4 for all practical purposes.
Properties of ranks
In the remainder of this section, we prove an O( mα( n)) bound on the running time of the disjoint-set operations with union by rank and path
compression. In order to prove this bound, we first prove some simple
properties of ranks.
Lemma 19.4
For all nodes x, we have x. rank ≤ x. p. rank, with strict inequality if x ≠ x. p ( x is not a root). The value of x. rank is initially 0, increases through time until x ≠ x. p, and from then on, x. rank does not change. The value of x. p. rank monotonically increases over time.
Proof The proof is a straightforward induction on the number of
operations, using the implementations of MAKE-SET, UNION, and
FIND-SET that appear on page 530, and is left as Exercise 19.4-1.
▪
Corollary 19.5
On the simple path from any node going up toward a root, node ranks
strictly increase.
▪
Lemma 19.6
Every node has rank at most n − 1.
Proof Each node’s rank starts at 0, and it increases only upon LINK
operations. Because there are at most n − 1 UNION operations, there
are also at most n − 1 LINK operations. Because each LINK operation
either leaves all ranks alone or increases some node’s rank by 1, all ranks
are at most n − 1.
▪
Lemma 19.6 provides a weak bound on ranks. In fact, every node has
rank at most ⌊lg n⌊ (see Exercise 19.4-2). The looser bound of Lemma
19.6 suffices for our purposes, however.
Proving the time bound
In order to prove the O( mα( n)) time bound, we’ll use the potential method of amortized analysis from Section 16.3. In performing the amortized analysis, it will be convenient to assume that we invoke the
LINK operation rather than the UNION operation. That is, since the
parameters of the LINK procedure are pointers to two roots, we act as
though we perform the appropriate FIND-SET operations separately.
The following lemma shows that even if we count the extra FIND-SET
operations induced by UNION calls, the asymptotic running time
remains unchanged.
Lemma 19.7
Suppose that we convert a sequence S′ of m′ MAKE-SET, UNION, and
FIND-SET operations into a sequence S of m MAKE-SET, LINK, and
FIND-SET operations by turning each UNION into two FIND-SET
operations followed by one LINK. Then, if sequence S runs in O( mα( n)) time, sequence S′ runs in O( m′ α( n)) time.
Proof Since each UNION operation in sequence S′ is converted into three operations in S, we have m′ ≤ m ≤ 3 m′, so that m = Θ( m′), Thus, an O( m α( n)) time bound for the converted sequence S implies an O( m′ α( n)) time bound for the original sequence S′.
▪
From now on, we assume that the initial sequence of m′ MAKE-SET,
UNION, and FIND-SET operations has been converted to a sequence
