Question: Memory usage. Compare the memory usage of BinarySearchST with that of SequentialSearchST for N key-value pairs, under the assumptions described in Section 1.4. Do not count the memory for the keys and values themselves, but do count references to them. For BinarySearchST, assume that array resizing is used, so that the array is between 25 percent and 100 percent full.

Answer:
3.1.21 - Memory usage

* BinarySearchST
   object overhead -> 16 bytes
   Key[] reference (keys) -> 8 bytes
   Value[] reference (values) -> 8 bytes
   int value (size) -> 4 bytes
   padding -> 4 bytes
   Key[]
      object overhead -> 16 bytes
      int value (length) -> 4 bytes
      padding -> 4 bytes
      N Key references -> between 8N and 32N bytes (the resizing array may be 25% to 100% full)
   Value[]
      object overhead -> 16 bytes
      int value (length) -> 4 bytes
      padding -> 4 bytes
      N Value references -> between 8N and 32N bytes (the resizing array may be 25% to 100% full)
Amount of memory needed: 16 + 8 + 8 + 4 + 4 + 16 + 4 + 4 + (8N to 32N) + 16 + 4 + 4 + (8N to 32N) = (16N to 64N) + 88 bytes

* SequentialSearchST
   object overhead -> 16 bytes
   Node reference (first) -> 8 bytes
   Node
      object overhead -> 16 bytes
      extra overhead for reference to the enclosing instance -> 8 bytes
      Key reference (key) -> 8 bytes
      Value reference (value) -> 8 bytes
      Node reference (next) -> 8 bytes
            (N Node references -> 48N bytes)
   int value (size) -> 4 bytes
   padding -> 4 bytes
Amount of memory needed: 16 + 8 + (16 + 8 + 8 + 8 + 8)N + 4 + 4 = 48N + 32 bytes


Question: Analysis of binary search. Prove that the maximum number of compares used for a binary search in a table of size N is precisely the number of bits in the binary representation of N, because the operation of shifting 1 bit to the right converts the binary representation of N into the binary representation of floor(N/2).

Answer:
3.1.23 - Analysis of binary search

As the book and exercise 3.1.20 have proven, the maximum number of compares used for a binary search in a table of size N is lg N + 1.

A number N has exactly lg N + 1 bits. This is because shifting 1 bit to the right reduces the number by half (rounded down).

For example:

N   Bit representation   Number of bits   lg N + 1
1                    1                1          1
2                   10                2          2
4                  100                3          3
5                  101                3          3
9                 1001                4          4

Therefore, the maximum number of compares used for a binary search in a table of size N is precisely the number of bits in the binary representation of N.


Question: Small tables. Suppose that a BinarySearchST client has S search operations and N distinct keys. Give the order of growth of S such that the cost of building the table is the same as the cost of all the searches.

Answer:
3.1.27 - Small tables

Building the binary search symbol table requires N calls to put().
Every put() operation makes a call to rank() and does a search, an operation with order of growth O(lg N).
Assuming that we can choose the order of the keys to insert, we can create the table in a sorted order, starting with the smallest element and ending with the highest element.
By doing this, every element will be inserted at the end of the keys[] and values[] array, making the put() operation use O(lg N) for the rank() operation and O(1) for the insert (there will be no need to move keys and values to the right since the new element is the rightmost element). N inserts will have an order of growth of O(N lg N).

A search operation has the order of growth O(lg N).
Therefore, the order of growth of S should be O(N), with S search operations having an order of growth O(N lg N), making the cost of building the table the same as the cost of all searches.

If the items are inserted in random order, the put operation has an order of growth O(N): O(lg N) for the rank operation and O(N) for inserting an element. N inserts will have an order of growth of O(N^2).
In this case, the order of growth of S should be O(N^2 / lg N).

Thanks to dragon-dreamer (https://github.com/dragon-dreamer) for correcting the orders of growth of S.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/120


Question: Add to BST a method height() that computes the height of the tree. Develop two implementations: a recursive method (which takes linear time and space proportional to the height), and a method like size() that adds a field to each node in the tree (and takes linear space and constant time per query).

Answer:
3.1.6

The put() method will be called once for every word.
The get() method will be called once for every word, except on the first time a word in being inserted in the symbol table.

Calls to put() = W
Calls to get() = W - D

Thanks to faame (https://github.com/faame) for suggesting a better answer.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/221


Question: Add to BST a recursive method avgCompares() that computes the average number of compares required by a random search hit in a given BST (the internal path length of the tree divided by its size, plus one). Develop two implementations: a recursive method (which takes linear time and space proportional to the height), and a method like size() that adds a field to each node in the tree (and takes linear space and constant time per query).

Answer:
3.1.7

Number of distinct keys among 10 random nonnegative integers less than 1000: 10
Number of distinct keys among 100 random nonnegative integers less than 1000: 97
Number of distinct keys among 1000 random nonnegative integers less than 1000: 635
Number of distinct keys among 10000 random nonnegative integers less than 1000: 999
Number of distinct keys among 100000 random nonnegative integers less than 1000: 1000
Number of distinct keys among 1000000 random nonnegative integers less than 1000: 1000


Question: Write a static method optCompares() that takes an integer argument N and computes the number of compares required by a random search hit in an optimal (perfectly balanced) BST, where all the null links are on the same level if the number of links is a power of 2 or on one of two levels otherwise.

Answer:
3.1.8

Most frequently used word of ten letters or more in Tale of Two Cities: Monseigneur Frequency: 47


Question: Draw all the different BST shapes that can result when N keys are inserted into an initially empty tree, for N = 2, 3, 4, 5, and 6.

Answer:
3.1.9

Length cutoff Last word inserted  Words processed
            1            known."           135937
            8          faltering           135905
           10      disfigurement           135890


Question: Write a test client TestBST.java for use in testing the implementations of min(), max(), floor(), ceiling(), select(), rank(), delete(), deleteMin(), deleteMax(), and keys() that are given in the text. Start with the standard indexing client given on page 370. Add code to take additional command-line arguments, as appropriate.

Answer:
3.1.10

key  value  first
 E     0      E0                                         0 compares
 A     1      A1  E0                                     1 compare
 S     2      S2  A1  E0                                 2 compares
 Y     3      Y3  S2  A1  E0                             3 compares
 Q     4      Q4  Y3  S2  A1  E0                         4 compares
 U     5      U5  Q4  Y3  S2  A1  E0                     5 compares
 E     6      U5  Q4  Y3  S2  A1  E6                     6 compares
 S     7      U5  Q4  Y3  S7  A1  E6                     4 compares
 T     8      T8  U5  Q4  Y3  S7  A1  E6                 6 compares
 I     9      I9  T8  U5  Q4  Y3  S7  A1  E6             7 compares
 O    10      O10  I9  T8  U5  Q4  Y3  S7  A1  E6        8 compares
 N    11      N11  O10  I9  T8  U5  Q4  Y3  S7  A1  E6   9 compares

Total: 55 compares

Thanks to ebber22 (https://github.com/ebber22) for finding an issue with the way the nodes were being added.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/102


Question: How many binary tree shapes of N nodes are there with height N? How many different ways are there to insert N distinct keys into an initially empty BST that result in a tree of height N? (See Exercise 3.2.2.)

Answer:
3.1.11

			keys[]				            vals[]
key  value    0  1  2  3  4  5  6  7  8  9    N   0  1  2  3  4  5  6  7  8  9
 E     0      E                               1   0                               0 compares
 A     1      A  E                            2   1  0                            2 compares
 S     2      A  E  S                         3   1  0  2                         2 compares
 Y     3      A  E  S  Y                      4   1  0  2  3                      2 compares
 Q     4      A  E  Q  S  Y                   5   1  0  4  2  3                   3 compares
 U     5      A  E  Q  S  U  Y                6   1  0  4  2  5  3                4 compares
 E     6      A  E  Q  S  U  Y                6   1  6  4  2  5  3                4 compares
 S     7      A  E  Q  S  U  Y                6   1  6  4  7  5  3                4 compares
 T     8      A  E  Q  S  T  U  Y             6   1  6  4  7  8  5  3             4 compares
 I     9      A  E  I  Q  S  T  U  Y          6   1  6  9  4  7  8  5  3          4 compares
 O    10      A  E  I  O  Q  S  T  U  Y       6   1  6  9  10 4  7  8  5  3       4 compares
 N    11      A  E  I  N  O  Q  S  T  U  Y    6   1  6  9  11 10 4  7  8  5  3    5 compares
              A  E  I  N  O  Q  S  T  U  Y        1  6  9  11 10 4  7  8  5  3

Total: 38 compares


Question: Give nonrecursive implementations of get() and put() for BST.

Answer:
3.1.13

For an application that does 10^3 put() operations and 10^6 get() operations I would use a binary search symbol table implementation.
The application does a lot more get() than put() operations and a binary search symbol table implementation has a O(log(n)) runtime complexity for the get() operation, which is better than the O(n) runtime complexity of the sequential search symbol table implementation.


Question: Give nonrecursive implementations of min(), max(), floor(), ceiling(), rank(), and select().

Answer:
3.1.14

For an application that does 10^6 put() operations and 10^3 get() operations I would use a sequential search symbol table implementation.

The worst-case runtime cost of the put() operation for a binary search symbol table is 2N while for a sequential search symbol table it is N.
For the get() operation, the worst-case runtime cost of a binary search symbol table is lg(n) while for the sequential search symbol table it is N.

Comparing the total number of operations in the worst-case:
Sequential search symbol table: 10^6 + 10^3 ~ 10^6 operations
Binary search symbol table: 2 * 10^6 + lg(10^3) ~ 2 * 10^6 operations

The average-case runtime cost of both implementations in this case is similar because most of the operations are put() and both the sequential search symbol table and the binary search symbol table implementations have an average-case cost of N for it.

So for this application the sequential search symbol table performs better than the binary search symbol table in the worst-case and both perform similarly in the average-case, making the sequential search symbol table the best choice.

Thanks to dragon-dreamer (https://github.com/dragon-dreamer) for showing that the sequential search symbol table is the best option in this question.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/119


Question:
Give the sequences of nodes examined when the methods in BST are used to compute each of the following quantities for the tree drawn at right.
a. floor("Q")
b. select(5)
c. ceiling("Q")
d. rank("J")
e. size("D", "T")
f. keys("D", "T")

Answer:
3.1.15

     Searches Percentage of total time spent on insertions
         1000                                        0.00%
      1000000                                        6.10%
   1000000000                                       96.26%

The average insertion cost is N, so the total insertion cost for N keys is N^2.
The average search cost is lg N, so the total search cost for M keys is M * lg N.
According to the question, M = 1000 * N.

The total cost is then:
total cost = N^2 + M * lg N = N^2 + 1000N * lg N

The insertion percentage is:
P = (N^2) / (N^2 + 1000N * lg N)
N^2 is the higher-order element in the equation, so as N increases, the insertion percentage approaches 100%.

Thanks to faame (https://github.com/faame) for improving this answer.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/222


Question: Draw the sequence of BSTs that results when you delete the keys from the tree of Exercise 3.2.1, one by one, in alphabetical order.

Answer:
3.1.18

The rank() method in BinarySearchST always starts the search in the middle of the array if it has an odd number of elements.
If the array has an even number of elements, the rank() method starts the search on the left of the two middle elements.
After comparing the middle element with the search key, if it is smaller, the search continues on the right side of the array.
If it is bigger, the search continues on the left side of the array.
If they have the same value, the current element's index is the rank we are looking for.

This guarantees that if an element exists in the symbol table its rank will be found in the rank() method.

When the element does not exist the value of the "low" variable will have passed the value of the "high" variable, pointing to the correct rank of where the key should be. This only happens when both the element on the left of the final rank has been checked and the element on the current (final) rank has been checked. After these checks, low will be pointing to the correct rank location.

Example:

Symbol Table: 0 1 2 3 5 6
Rank of key 4 (non-existent)

1- The initial search range is [0..5]. The rank() method checks the left of the middle elements on index 2 -> value 2
2- 2 is less than 4, so the new range to search is [3..5]. The rank() method checks the middle element on index 4 -> value 5
3- 5 is more than 4, so the new range to search is [3..3]. The rank() method checks the only element left (index 3) -> value 3
4- 3 is less than 4, so the new range to search is [4..3]. Now the "low" variable is bigger than the "high" variable.
5- The rank() method returns the value of the "low" variable, 4. This is the correct rank for key 4.


Question: Prove that the running time of the two-argument keys() in a BST with N nodes is at most proportional to the tree height plus the number of keys in the range.

Answer:
3.1.20

Proposition B. Binary search in an ordered array with N keys uses no more than lg N + 1 compares for a search (successful or unsuccessful).

Proof: Let C(N) be the number of compares to search for a key in a symbol table of size N.
We have C(0) = 0, C(1) = 1, and for N > 0 we can write a recurrence relationship that directly mirrors the recursive method:
C(N) <= C(FLOOR(N / 2)) + 1

Whether the search goes to the left or to the right, the size of the subarray is no more than FLOOR(N / 2), and we can use one compare to check for equality and to choose whether to go left or right.

Let's first prove by induction that C(N) is monotonic: C(N) <= C(N + 1) for all N > 0.

It is trivial to prove that:
C(1) = 1
C(2) = 1 or 2

Thus we have:
C(1) <= C(2)

Assume for all N=[0, K]
C(N + 1) >= C(N)

Thus we have:
C(K + 1) >= C(K)

At the beginning of the binary search, low has value 0 and high has value N. Mid is computed as:
mid = low + (high - low) / 2 = high / 2

Let's use L to represent the size of the left half and R to represent the size of the right half.
The mid element has size 1.
When N = K + 1 the total size is L + 1 + R.

When N = K + 2:
mid = low + (high - low) / 2 = (high + 1) / 2 = (N + 1) / 2

When N is incremented by 1 (from K + 1 to K + 2), the mid point either remains in the same place or shifts to the right by 1.
1- If mid remains in the same place, then L remains the same, and R is incremented by 1.
Since (R + 1) <= K + 1, we have:
C(K + 2) = C(L) + 1 + C(R + 1) >= C(L) + 1 + C(R) = C(K + 1)
2- If mid shifts to the right by 1, then L is incremented by 1 and R remains the same.
Since (L + 1) <= K + 1, we have:
C(K + 2) = C(L + 1) + 1 + C(R) >= C(L) + 1 + C(R) = C(K + 1)

Therefore, given C(K + 1) >= C(K) it is also true that C(K + 2) >= C(K + 1), which proves that C(N) is monotonic.

For a general N, we have that:

C(N) <= C(N / 2) + 1 (one comparison to check equality or decide which way of the subarray to go)
C(N / 2) <= C(N / 4) + 1

Putting the value of C(N / 2) in the first equation:
C(N) <= C(N / 4) + 1 + 1

And adding all values to the first equation until C = 1:
C(N) <= C(N / 8) + 1 + 1 + 1
C(N) <= C(N / 16) + 1 + 1 + 1 + 1
C(N) <= C(N / 2^k) + 1 + 1 + 1 + 1 + ... + 1

until we get to
C(N) <= 1 + 1 + 1 + 1 + 1 + ... + 1 (even if N is not divisible by 2 there is still a compare operation)
C(N) <= k + 1

In this case, 2^k = N
k <= lg N + 1

We can also prove it using the Master theorem:

Binary search recurrence relation:
T(N) = T(N/2) + O(1)

Master theorem:
T(N) = aT(N/b) + f(N)

Here, a = 1, b = 2 and f(n) is O(1) (constant)

c = log(a base b) = log(1 base 2) = 0

We can see that this is the case 2 of the Master theorem by taking k = 0 in this equation:
O(n^c * (log n)^k)
O(n^0 * (log n)^0) = O(1) = f(n) -> This means we are in case 2 of the Master theorem

From the Case 2 of the Master Theorem we know that:
T(n) = O(n^(log a base b) * (log n)^(k + 1))
T(n) = O(n^0 * log(n)^1) = O(log n)

With binary search, we achieve a logarithmic-time search guarantee.

Reference: https://en.wikipedia.org/wiki/Master_theorem

Thanks to faame (https://github.com/faame) for adding the section proving that C(N) is monotonic.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/223


Question: Add a BST method randomKey() that returns a random key from the symbol table in time proportional to the tree height, in the worst case.

Answer:
3.1.21 - Memory usage

* BinarySearchST
   object overhead -> 16 bytes
   Key[] reference (keys) -> 8 bytes
   Value[] reference (values) -> 8 bytes
   int value (size) -> 4 bytes
   padding -> 4 bytes
   Key[]
      object overhead -> 16 bytes
      int value (length) -> 4 bytes
      padding -> 4 bytes
      N Key references -> between 8N and 32N bytes (the resizing array may be 25% to 100% full)
   Value[]
      object overhead -> 16 bytes
      int value (length) -> 4 bytes
      padding -> 4 bytes
      N Value references -> between 8N and 32N bytes (the resizing array may be 25% to 100% full)
Amount of memory needed: 16 + 8 + 8 + 4 + 4 + 16 + 4 + 4 + (8N to 32N) + 16 + 4 + 4 + (8N to 32N) = (16N to 64N) + 88 bytes

* SequentialSearchST
   object overhead -> 16 bytes
   Node reference (first) -> 8 bytes
   Node
      object overhead -> 16 bytes
      extra overhead for reference to the enclosing instance -> 8 bytes
      Key reference (key) -> 8 bytes
      Value reference (value) -> 8 bytes
      Node reference (next) -> 8 bytes
            (N Node references -> 48N bytes)
   int value (size) -> 4 bytes
   padding -> 4 bytes
Amount of memory needed: 16 + 8 + (16 + 8 + 8 + 8 + 8)N + 4 + 4 = 48N + 32 bytes


Question: Is delete() commutative? (Does deleting x, then y give the same result as deleting y, then x?)

Answer:
3.1.23 - Analysis of binary search

As the book and exercise 3.1.20 have proven, the maximum number of compares used for a binary search in a table of size N is lg N + 1.

A number N has exactly lg N + 1 bits. This is because shifting 1 bit to the right reduces the number by half (rounded down).

For example:

N   Bit representation   Number of bits   lg N + 1
1                    1                1          1
2                   10                2          2
4                  100                3          3
5                  101                3          3
9                 1001                4          4

Therefore, the maximum number of compares used for a binary search in a table of size N is precisely the number of bits in the binary representation of N.


Question: Memory usage. Compare the memory usage of BST with the memory usage of BinarySearchST and SequentialSearchST for N key-value pairs, under the assumptions described in Section 1.4 (see Exercise 3.1.21). Do not count the memory for the keys and values themselves, but do count references to them. Then draw a diagram that depicts the precise memory usage of a BST with String keys and Integer values (such as the ones built by FrequencyCounter), and then estimate the memory usage (in bytes) for the BST built when FrequencyCounter uses BST for Tale of Two Cities.

Answer:
3.1.27 - Small tables

Building the binary search symbol table requires N calls to put().
Every put() operation makes a call to rank() and does a search, an operation with order of growth O(lg N).
Assuming that we can choose the order of the keys to insert, we can create the table in a sorted order, starting with the smallest element and ending with the highest element.
By doing this, every element will be inserted at the end of the keys[] and values[] array, making the put() operation use O(lg N) for the rank() operation and O(1) for the insert (there will be no need to move keys and values to the right since the new element is the rightmost element). N inserts will have an order of growth of O(N lg N).

A search operation has the order of growth O(lg N).
Therefore, the order of growth of S should be O(N), with S search operations having an order of growth O(N lg N), making the cost of building the table the same as the cost of all searches.

If the items are inserted in random order, the put operation has an order of growth O(N): O(lg N) for the rank operation and O(N) for inserting an element. N inserts will have an order of growth of O(N^2).
In this case, the order of growth of S should be O(N^2 / lg N).

Thanks to dragon-dreamer (https://github.com/dragon-dreamer) for correcting the orders of growth of S.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/120


Question: Equal key check. Write a method hasNoDuplicates() that takes a Node as argument and returns true if there are no equal keys in the binary tree rooted at the argument node, false otherwise. Assume that the test of the previous exercise has passed.

Answer:
3.1.31 - Performance driver

Number of Experiments |  Array Size |  AVG Time Taken
                    1        100000             12.32
                    2        100000             12.13
                    3        100000             12.15
                    4        100000             14.73
                    5        100000             19.76


Question: Certification. Write a method isBST() that takes a Node as argument and returns true if the argument node is the root of a binary search tree, false otherwise. Hint: This task is also more difficult than it might seem, because the order in which you call the methods in the previous three exercises is important.

Answer:
3.1.32 - Exercise driver

               Input Type |  Array Size |  Running Time
                  Ordered        100000             6.8
                  Ordered        200000            37.7
                  Ordered        400000           142.9
          Reverse Ordered        100000            24.1
          Reverse Ordered        200000           102.2
          Reverse Ordered        400000           469.6
                Same Keys        100000             0.0
                Same Keys        200000             0.0
                Same Keys        400000             0.0
  Keys with only 2 Values        100000             0.0
  Keys with only 2 Values        200000             0.0
  Keys with only 2 Values        400000             0.0


Question: Select/rank check. Write a method that checks, for all i from 0 to size()-1, whether i is equal to rank(select(i)) and, for all keys in the BST, whether key is equal to select(rank(key)).

Answer:
3.1.33 - Driver for self-organizing search

Array Size |  Self-Organizing Search Time |  Default Binary Search Time
      1000                           0.01                          0.01
     10000                           0.05                          0.01
    100000                           4.46                          0.10
   1000000                         576.02                          0.87


Question:
Threading. Your goal is to support an extended API ThreadedST that supports the following additional operations in constant time:
Key next(Key key) key that follows key (null if key is the maximum)
Key prev(Key key) key that precedes key (null if key is the minimum)
To do so, add fields pred and succ to Node that contain links to the predecessor and successor nodes, and modify put(), deleteMin(), deleteMax(), and delete() to maintain these fields.

Answer:
3.1.34 - Zipf's law

Array Size |      Move-to-Front Time |   Optimal Arrangement Time
      1000                      0.04                         0.01
     10000                      0.42                         0.18
    100000                     33.46                         6.18
   1000000                   4069.86                       481.80


Question: Refined analysis. Refine the mathematical model to better explain the experimental results in the table given in the text. Specifically, show that the average number of compares for a successful search in a tree built from random keys approaches the limit 2 ln N + 2γ - 3 ≈ 1.39 lg N – 1.85 as N increases, where γ ≈ .57721... is Euler’s constant. Hint: Referring to the quicksort analysis in Section 2.3, use the fact that the integral of 1/x approaches ln N + γ.

Answer:
3.1.35 - Performance validation I

The running time of FrequencyCounter is close to quadratic when it uses SequentialSearchST for its symbol table.

Number of Words |  Running Time |    Ratio |  Complexity (N^X)
           1000            0.01       1.00                0.00
           2000            0.01       1.60                0.68
           4000            0.03       3.50                1.81
           8000            0.11       3.86                1.95
          16000            0.37       3.44                1.78
          32000            1.34       3.60                1.85
          64000            3.56       2.66                1.41
         128000           12.67       3.56                1.83


Question: Iterator. Is it possible to write a nonrecursive version of keys() that uses space proportional to the tree height (independent of the number of keys in the range)?

Answer:
3.1.36 - Performance validation II

The performance of BinarySearchST and SequentialSearchST for FrequencyCounter is even better than predicted by analysis because of the details of the FrequencyCounter application.
FrequencyCounter does not add all the words in the symbol table (only the words with a length higher than a specified minLength parameter). Also, there are many repeated words in the Tale of Two Cities book, which was used as benchmark. Out of 135,938 words, only 561 are distinct.
With more search hits than search misses, a sequential search does not have to check all the N words in the symbol table (its worst case) as often. The same applies to a binary search, that does not have to do lg N compares as often than if there were many search misses.
These details of the application make the performance of the symbol tables better than predicted.


Question: Level-order traversal. Write a method printLevel() that takes a Node as argument and prints the keys in the subtree rooted at that node in level order (in order of their distance from the root, with nodes on each level in order from left to right). Hint: Use a Queue.

Answer:
3.1.37 - Put/get ratio

For a symbol table with 10-bit int values the amount of time that BinarySearchST spent on put() was lower than the amount of time it spent on get().
However, for all the other scenarios (counting the frequency of 20 and 30-bit values and the frequency of the words in the book Tale of Two Cities) the amount of time that BinarySearchST spends on put() operations is much higher than the amount of time it spends on get().

             Input |   Running Time Put |   Running Time Get |     Ratio
 10-bit int values                 0.11                 0.29        0.39
 20-bit int values               216.62                 0.78      279.51
 30-bit int values              1303.54                 0.81     1613.29
Tale of Two Cities                 0.33                 0.03       12.37


Question: Find the probability that each of the 2-3 trees in Exercise 3.3.5 is the result of the insertion of N random distinct keys into an initially empty tree.

Answer:
3.1.6

The put() method will be called once for every word.
The get() method will be called once for every word, except on the first time a word in being inserted in the symbol table.

Calls to put() = W
Calls to get() = W - D

Thanks to faame (https://github.com/faame) for suggesting a better answer.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/221


Question: Draw diagrams like the one at the top of page 428 for the other five cases in the diagram at the bottom of that page.

Answer:
3.1.7

Number of distinct keys among 10 random nonnegative integers less than 1000: 10
Number of distinct keys among 100 random nonnegative integers less than 1000: 97
Number of distinct keys among 1000 random nonnegative integers less than 1000: 635
Number of distinct keys among 10000 random nonnegative integers less than 1000: 999
Number of distinct keys among 100000 random nonnegative integers less than 1000: 1000
Number of distinct keys among 1000000 random nonnegative integers less than 1000: 1000


Question: Show all possible ways that one might represent a 4-node with three 2-nodes bound together with red links (not necessarily left-leaning).

Answer:
3.1.8

Most frequently used word of ten letters or more in Tale of Two Cities: Monseigneur Frequency: 47


Question: Which of the following are red-black BSTs?

Answer:
3.1.9

Length cutoff Last word inserted  Words processed
            1            known."           135937
            8          faltering           135905
           10      disfigurement           135890


Question: Draw the red-black BST that results when you insert items with the keys E A S Y Q U T I O N in that order into an initially empty tree.

Answer:
3.1.10

key  value  first
 E     0      E0                                         0 compares
 A     1      A1  E0                                     1 compare
 S     2      S2  A1  E0                                 2 compares
 Y     3      Y3  S2  A1  E0                             3 compares
 Q     4      Q4  Y3  S2  A1  E0                         4 compares
 U     5      U5  Q4  Y3  S2  A1  E0                     5 compares
 E     6      U5  Q4  Y3  S2  A1  E6                     6 compares
 S     7      U5  Q4  Y3  S7  A1  E6                     4 compares
 T     8      T8  U5  Q4  Y3  S7  A1  E6                 6 compares
 I     9      I9  T8  U5  Q4  Y3  S7  A1  E6             7 compares
 O    10      O10  I9  T8  U5  Q4  Y3  S7  A1  E6        8 compares
 N    11      N11  O10  I9  T8  U5  Q4  Y3  S7  A1  E6   9 compares

Total: 55 compares

Thanks to ebber22 (https://github.com/ebber22) for finding an issue with the way the nodes were being added.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/102


Question: Draw the red-black BST that results when you insert items with the keys Y L P M X H C R A E S in that order into an initially empty tree.

Answer:
3.1.11

			keys[]				            vals[]
key  value    0  1  2  3  4  5  6  7  8  9    N   0  1  2  3  4  5  6  7  8  9
 E     0      E                               1   0                               0 compares
 A     1      A  E                            2   1  0                            2 compares
 S     2      A  E  S                         3   1  0  2                         2 compares
 Y     3      A  E  S  Y                      4   1  0  2  3                      2 compares
 Q     4      A  E  Q  S  Y                   5   1  0  4  2  3                   3 compares
 U     5      A  E  Q  S  U  Y                6   1  0  4  2  5  3                4 compares
 E     6      A  E  Q  S  U  Y                6   1  6  4  2  5  3                4 compares
 S     7      A  E  Q  S  U  Y                6   1  6  4  7  5  3                4 compares
 T     8      A  E  Q  S  T  U  Y             6   1  6  4  7  8  5  3             4 compares
 I     9      A  E  I  Q  S  T  U  Y          6   1  6  9  4  7  8  5  3          4 compares
 O    10      A  E  I  O  Q  S  T  U  Y       6   1  6  9  10 4  7  8  5  3       4 compares
 N    11      A  E  I  N  O  Q  S  T  U  Y    6   1  6  9  11 10 4  7  8  5  3    5 compares
              A  E  I  N  O  Q  S  T  U  Y        1  6  9  11 10 4  7  8  5  3

Total: 38 compares


Question: True or false: If you insert keys in increasing order into a red-black BST, the tree height is monotonically increasing.

Answer:
3.1.13

For an application that does 10^3 put() operations and 10^6 get() operations I would use a binary search symbol table implementation.
The application does a lot more get() than put() operations and a binary search symbol table implementation has a O(log(n)) runtime complexity for the get() operation, which is better than the O(n) runtime complexity of the sequential search symbol table implementation.


Question: Draw the red-black BST that results when you insert letters A through K in order into an initially empty tree, then describe what happens in general when trees are built by insertion of keys in ascending order (see also the figure in the text).

Answer:
3.1.14

For an application that does 10^6 put() operations and 10^3 get() operations I would use a sequential search symbol table implementation.

The worst-case runtime cost of the put() operation for a binary search symbol table is 2N while for a sequential search symbol table it is N.
For the get() operation, the worst-case runtime cost of a binary search symbol table is lg(n) while for the sequential search symbol table it is N.

Comparing the total number of operations in the worst-case:
Sequential search symbol table: 10^6 + 10^3 ~ 10^6 operations
Binary search symbol table: 2 * 10^6 + lg(10^3) ~ 2 * 10^6 operations

The average-case runtime cost of both implementations in this case is similar because most of the operations are put() and both the sequential search symbol table and the binary search symbol table implementations have an average-case cost of N for it.

So for this application the sequential search symbol table performs better than the binary search symbol table in the worst-case and both perform similarly in the average-case, making the sequential search symbol table the best choice.

Thanks to dragon-dreamer (https://github.com/dragon-dreamer) for showing that the sequential search symbol table is the best option in this question.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/119


Question: Answer the previous two questions for the case when the keys are inserted in descending order.

Answer:
3.1.15

     Searches Percentage of total time spent on insertions
         1000                                        0.00%
      1000000                                        6.10%
   1000000000                                       96.26%

The average insertion cost is N, so the total insertion cost for N keys is N^2.
The average search cost is lg N, so the total search cost for M keys is M * lg N.
According to the question, M = 1000 * N.

The total cost is then:
total cost = N^2 + M * lg N = N^2 + 1000N * lg N

The insertion percentage is:
P = (N^2) / (N^2 + 1000N * lg N)
N^2 is the higher-order element in the equation, so as N increases, the insertion percentage approaches 100%.

Thanks to faame (https://github.com/faame) for improving this answer.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/222


Question: Draw all the structurally different red-black BSTs with N keys, for N from 2 up to 10 (see Exercise 3.3.5).

Answer:
3.1.18

The rank() method in BinarySearchST always starts the search in the middle of the array if it has an odd number of elements.
If the array has an even number of elements, the rank() method starts the search on the left of the two middle elements.
After comparing the middle element with the search key, if it is smaller, the search continues on the right side of the array.
If it is bigger, the search continues on the left side of the array.
If they have the same value, the current element's index is the rank we are looking for.

This guarantees that if an element exists in the symbol table its rank will be found in the rank() method.

When the element does not exist the value of the "low" variable will have passed the value of the "high" variable, pointing to the correct rank of where the key should be. This only happens when both the element on the left of the final rank has been checked and the element on the current (final) rank has been checked. After these checks, low will be pointing to the correct rank location.

Example:

Symbol Table: 0 1 2 3 5 6
Rank of key 4 (non-existent)

1- The initial search range is [0..5]. The rank() method checks the left of the middle elements on index 2 -> value 2
2- 2 is less than 4, so the new range to search is [3..5]. The rank() method checks the middle element on index 4 -> value 5
3- 5 is more than 4, so the new range to search is [3..3]. The rank() method checks the only element left (index 3) -> value 3
4- 3 is less than 4, so the new range to search is [4..3]. Now the "low" variable is bigger than the "high" variable.
5- The rank() method returns the value of the "low" variable, 4. This is the correct rank for key 4.


Question: Compute the internal path length in a perfectly balanced BST of N nodes, when N is a power of 2 minus 1.

Answer:
3.1.20

Proposition B. Binary search in an ordered array with N keys uses no more than lg N + 1 compares for a search (successful or unsuccessful).

Proof: Let C(N) be the number of compares to search for a key in a symbol table of size N.
We have C(0) = 0, C(1) = 1, and for N > 0 we can write a recurrence relationship that directly mirrors the recursive method:
C(N) <= C(FLOOR(N / 2)) + 1

Whether the search goes to the left or to the right, the size of the subarray is no more than FLOOR(N / 2), and we can use one compare to check for equality and to choose whether to go left or right.

Let's first prove by induction that C(N) is monotonic: C(N) <= C(N + 1) for all N > 0.

It is trivial to prove that:
C(1) = 1
C(2) = 1 or 2

Thus we have:
C(1) <= C(2)

Assume for all N=[0, K]
C(N + 1) >= C(N)

Thus we have:
C(K + 1) >= C(K)

At the beginning of the binary search, low has value 0 and high has value N. Mid is computed as:
mid = low + (high - low) / 2 = high / 2

Let's use L to represent the size of the left half and R to represent the size of the right half.
The mid element has size 1.
When N = K + 1 the total size is L + 1 + R.

When N = K + 2:
mid = low + (high - low) / 2 = (high + 1) / 2 = (N + 1) / 2

When N is incremented by 1 (from K + 1 to K + 2), the mid point either remains in the same place or shifts to the right by 1.
1- If mid remains in the same place, then L remains the same, and R is incremented by 1.
Since (R + 1) <= K + 1, we have:
C(K + 2) = C(L) + 1 + C(R + 1) >= C(L) + 1 + C(R) = C(K + 1)
2- If mid shifts to the right by 1, then L is incremented by 1 and R remains the same.
Since (L + 1) <= K + 1, we have:
C(K + 2) = C(L + 1) + 1 + C(R) >= C(L) + 1 + C(R) = C(K + 1)

Therefore, given C(K + 1) >= C(K) it is also true that C(K + 2) >= C(K + 1), which proves that C(N) is monotonic.

For a general N, we have that:

C(N) <= C(N / 2) + 1 (one comparison to check equality or decide which way of the subarray to go)
C(N / 2) <= C(N / 4) + 1

Putting the value of C(N / 2) in the first equation:
C(N) <= C(N / 4) + 1 + 1

And adding all values to the first equation until C = 1:
C(N) <= C(N / 8) + 1 + 1 + 1
C(N) <= C(N / 16) + 1 + 1 + 1 + 1
C(N) <= C(N / 2^k) + 1 + 1 + 1 + 1 + ... + 1

until we get to
C(N) <= 1 + 1 + 1 + 1 + 1 + ... + 1 (even if N is not divisible by 2 there is still a compare operation)
C(N) <= k + 1

In this case, 2^k = N
k <= lg N + 1

We can also prove it using the Master theorem:

Binary search recurrence relation:
T(N) = T(N/2) + O(1)

Master theorem:
T(N) = aT(N/b) + f(N)

Here, a = 1, b = 2 and f(n) is O(1) (constant)

c = log(a base b) = log(1 base 2) = 0

We can see that this is the case 2 of the Master theorem by taking k = 0 in this equation:
O(n^c * (log n)^k)
O(n^0 * (log n)^0) = O(1) = f(n) -> This means we are in case 2 of the Master theorem

From the Case 2 of the Master Theorem we know that:
T(n) = O(n^(log a base b) * (log n)^(k + 1))
T(n) = O(n^0 * log(n)^1) = O(log n)

With binary search, we achieve a logarithmic-time search guarantee.

Reference: https://en.wikipedia.org/wiki/Master_theorem

Thanks to faame (https://github.com/faame) for adding the section proving that C(N) is monotonic.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/223


Question: Create a test client TestRB.java, based on your solution to Exercise 3.2.10.

Answer:
3.1.21 - Memory usage

* BinarySearchST
   object overhead -> 16 bytes
   Key[] reference (keys) -> 8 bytes
   Value[] reference (values) -> 8 bytes
   int value (size) -> 4 bytes
   padding -> 4 bytes
   Key[]
      object overhead -> 16 bytes
      int value (length) -> 4 bytes
      padding -> 4 bytes
      N Key references -> between 8N and 32N bytes (the resizing array may be 25% to 100% full)
   Value[]
      object overhead -> 16 bytes
      int value (length) -> 4 bytes
      padding -> 4 bytes
      N Value references -> between 8N and 32N bytes (the resizing array may be 25% to 100% full)
Amount of memory needed: 16 + 8 + 8 + 4 + 4 + 16 + 4 + 4 + (8N to 32N) + 16 + 4 + 4 + (8N to 32N) = (16N to 64N) + 88 bytes

* SequentialSearchST
   object overhead -> 16 bytes
   Node reference (first) -> 8 bytes
   Node
      object overhead -> 16 bytes
      extra overhead for reference to the enclosing instance -> 8 bytes
      Key reference (key) -> 8 bytes
      Value reference (value) -> 8 bytes
      Node reference (next) -> 8 bytes
            (N Node references -> 48N bytes)
   int value (size) -> 4 bytes
   padding -> 4 bytes
Amount of memory needed: 16 + 8 + (16 + 8 + 8 + 8 + 8)N + 4 + 4 = 48N + 32 bytes


Question: 2-3 trees without balance restriction. Develop an implementation of the basic symbol-table API that uses 2-3 trees that are not necessarily balanced as the underlying data structure. Allow 3-nodes to lean either way. Hook the new node onto the bottom with a black link when inserting into a 3-node at the bottom. Run experiments to develop a hypothesis estimating the average path length in a tree built from N random insertions.

Answer:
3.1.23 - Analysis of binary search

As the book and exercise 3.1.20 have proven, the maximum number of compares used for a binary search in a table of size N is lg N + 1.

A number N has exactly lg N + 1 bits. This is because shifting 1 bit to the right reduces the number by half (rounded down).

For example:

N   Bit representation   Number of bits   lg N + 1
1                    1                1          1
2                   10                2          2
4                  100                3          3
5                  101                3          3
9                 1001                4          4

Therefore, the maximum number of compares used for a binary search in a table of size N is precisely the number of bits in the binary representation of N.


Question: Allow right-leaning red links. Develop a modified version of your solution to Exercise 3.3.25 that allows right-leaning red links in the tree.

Answer:
3.1.27 - Small tables

Building the binary search symbol table requires N calls to put().
Every put() operation makes a call to rank() and does a search, an operation with order of growth O(lg N).
Assuming that we can choose the order of the keys to insert, we can create the table in a sorted order, starting with the smallest element and ending with the highest element.
By doing this, every element will be inserted at the end of the keys[] and values[] array, making the put() operation use O(lg N) for the rank() operation and O(1) for the insert (there will be no need to move keys and values to the right since the new element is the rightmost element). N inserts will have an order of growth of O(N lg N).

A search operation has the order of growth O(lg N).
Therefore, the order of growth of S should be O(N), with S search operations having an order of growth O(N lg N), making the cost of building the table the same as the cost of all searches.

If the items are inserted in random order, the put operation has an order of growth O(N): O(lg N) for the rank operation and O(N) for inserting an element. N inserts will have an order of growth of O(N^2).
In this case, the order of growth of S should be O(N^2 / lg N).

Thanks to dragon-dreamer (https://github.com/dragon-dreamer) for correcting the orders of growth of S.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/120


Question: Tree drawing. Add a method draw() to RedBlackBST that draws red-black BST figures in the style of the text (see Exercise 3.2.38).

Answer:
3.1.31 - Performance driver

Number of Experiments |  Array Size |  AVG Time Taken
                    1        100000             12.32
                    2        100000             12.13
                    3        100000             12.15
                    4        100000             14.73
                    5        100000             19.76


Question: AVL trees. An AVL tree is a BST where the height of every node and that of its sibling differ by at most 1. (The oldest balanced tree algorithms are based on using rotations to maintain height balance in AVL trees.) Show that coloring red links that go from nodes of even height to nodes of odd height in an AVL tree gives a (perfectly balanced) 2-3-4 tree, where red links are not necessarily left-leaning. Extra credit: Develop an implementation of the symbol-table API that uses this as the underlying data structure. One approach is to keep a height field in each node, using rotations after the recursive calls to adjust the height as necessary; another is to use the red-black representation and use methods like moveRedLeft() and moveRedRight() in Exercise 3.3.39 and Exercise 3.3.40.

Answer:
3.1.32 - Exercise driver

               Input Type |  Array Size |  Running Time
                  Ordered        100000             6.8
                  Ordered        200000            37.7
                  Ordered        400000           142.9
          Reverse Ordered        100000            24.1
          Reverse Ordered        200000           102.2
          Reverse Ordered        400000           469.6
                Same Keys        100000             0.0
                Same Keys        200000             0.0
                Same Keys        400000             0.0
  Keys with only 2 Values        100000             0.0
  Keys with only 2 Values        200000             0.0
  Keys with only 2 Values        400000             0.0


Question: Certification. Add to RedBlackBST a method is23() to check that no node is connected to two red links and that there are no right-leaning red links and a method isBalanced() to check that all paths from the root to a null link have the same number of black links. Combine these methods with code from isBST() in Exercise 3.2.32 to create a method isRedBlackBST() that checks that the tree is a red-black BST.

Answer:
3.1.33 - Driver for self-organizing search

Array Size |  Self-Organizing Search Time |  Default Binary Search Time
      1000                           0.01                          0.01
     10000                           0.05                          0.01
    100000                           4.46                          0.10
   1000000                         576.02                          0.87


Question: All 2-3 trees. Write code to generate all structurally different 2-3 trees of height 2, 3, and 4. There are 2, 7, and 122 such trees, respectively. (Hint: Use a symbol table.)

Answer:
3.1.34 - Zipf's law

Array Size |      Move-to-Front Time |   Optimal Arrangement Time
      1000                      0.04                         0.01
     10000                      0.42                         0.18
    100000                     33.46                         6.18
   1000000                   4069.86                       481.80


Question: 2-3 trees. Write a program TwoThreeST.java that uses two node types to implement 2-3 search trees directly.

Answer:
3.1.35 - Performance validation I

The running time of FrequencyCounter is close to quadratic when it uses SequentialSearchST for its symbol table.

Number of Words |  Running Time |    Ratio |  Complexity (N^X)
           1000            0.01       1.00                0.00
           2000            0.01       1.60                0.68
           4000            0.03       3.50                1.81
           8000            0.11       3.86                1.95
          16000            0.37       3.44                1.78
          32000            1.34       3.60                1.85
          64000            3.56       2.66                1.41
         128000           12.67       3.56                1.83


Question: 2-3-4-5-6-7-8 trees. Describe algorithms for search and insertion in balanced 2-3-4-5-6-7-8 search trees.

Answer:
3.1.36 - Performance validation II

The performance of BinarySearchST and SequentialSearchST for FrequencyCounter is even better than predicted by analysis because of the details of the FrequencyCounter application.
FrequencyCounter does not add all the words in the symbol table (only the words with a length higher than a specified minLength parameter). Also, there are many repeated words in the Tale of Two Cities book, which was used as benchmark. Out of 135,938 words, only 561 are distinct.
With more search hits than search misses, a sequential search does not have to check all the N words in the symbol table (its worst case) as often. The same applies to a binary search, that does not have to do lg N compares as often than if there were many search misses.
These details of the application make the performance of the symbol tables better than predicted.


Question: Memoryless. Show that red-black BSTs are not memoryless: for example, if you insert a key that is smaller than all the keys in the tree and then immediately delete the minimum, you may get a different tree.

Answer:
3.1.37 - Put/get ratio

For a symbol table with 10-bit int values the amount of time that BinarySearchST spent on put() was lower than the amount of time it spent on get().
However, for all the other scenarios (counting the frequency of 20 and 30-bit values and the frequency of the words in the book Tale of Two Cities) the amount of time that BinarySearchST spends on put() operations is much higher than the amount of time it spends on get().

             Input |   Running Time Put |   Running Time Get |     Ratio
 10-bit int values                 0.11                 0.29        0.39
 20-bit int values               216.62                 0.78      279.51
 30-bit int values              1303.54                 0.81     1613.29
Tale of Two Cities                 0.33                 0.03       12.37


Question: Delete the minimum. Implement the deleteMin() operation for red-black BSTs by maintaining the correspondence with the transformations given in the text for moving down the left spine of the tree while maintaining the invariant that the current node is not a 2-node.

Answer:
3.1.39 - Actual timings

As expected, BinarySearchST total running time is much smaller than SequentialSearchST total running time.
BinarySearchST total running time for all the calls to get() and put() was 204 milliseconds while SequentialSearchST total running time was 2237 milliseconds.


Question: Delete the maximum. Implement the deleteMax() operation for red-black BSTs. Note that the transformations involved differ slightly from those in the previous exercise because red links are left-leaning.

Answer:
3.1.40 - Crossover to binary search

Analysis:
Considering that sequential search has a complexity of O(n) and binary search has a complexity of O(lg (n)):

N = 58: While sequential search does 58 operations (in the worst case), binary search does ~5.8 operations (in the worst case).
Binary search is ~10 times faster than sequential search for N = 58.

N = 996: While sequential search does 996 operations (in the worst case), binary search does ~9.96 operations (in the worst case).
Binary search is ~100 times faster than sequential search for N = 996.

N = 13746: While sequential search does 13,746 operations (in the worst case), binary search does ~13.746 operations (in the worst case).
Binary search is ~1000 times faster than sequential search for N = 13,746.

Experimental results:

Value of N for which binary search becomes 10 times faster than sequential search: 120
Value of N for which binary search becomes 100 times faster than sequential search: 1463
Value of N for which binary search becomes 1000 times faster than sequential search: 22461


Question: Delete. Implement the delete() operation for red-black BSTs, combining the methods of the previous two exercises with the delete() operation for BSTs.

Answer:
3.1.41 - Crossover to interpolation search

Analysis:
Considering that binary search has a complexity of O(lg (n)) and interpolation search has a complexity of O(lg(lg (n))) when the elements are uniformly distributed (and it is O(n) otherwise).
Assuming an array with elements uniformly distributed:

N = 1: Both binary search and interpolation search would be able to finish their search instantly with N = 1.
Interpolation search and binary search are just as fast for N = 1.

N = 4: Binary search does 2 operations (in the worst case) and interpolation search does 1 operation (in the worst case).
Interpolation search is 2 times faster than binary search for N = 4.

N = 490000000000000000 (49 * 10^16): Binary search does 58.76 operations (in the worst case) and interpolation search does 5.876 operations (in the worst case).
Interpolation search is 10 times faster than binary search for N = 49 * 10^16.

Experimental results:

Value of N for which interpolation search becomes the same speed as binary search: 1
Value of N for which interpolation search becomes 2 times faster than binary search: 21
Value of N for which interpolation search becomes 10 times faster than binary search: 1033


Question: Suppose that keys are t-bit integers. For a modular hash function with prime M, prove that each key bit has the property that there exist two keys differing only in that bit that have different hash values.

Answer:
3.1.6

The put() method will be called once for every word.
The get() method will be called once for every word, except on the first time a word in being inserted in the symbol table.

Calls to put() = W
Calls to get() = W - D

Thanks to faame (https://github.com/faame) for suggesting a better answer.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/221


Question: Consider the idea of implementing modular hashing for integer keys with the code (a * k) % M, where a is an arbitrary fixed prime. Does this change mix up the bits sufficiently well that you can use nonprime M?

Answer:
3.1.7

Number of distinct keys among 10 random nonnegative integers less than 1000: 10
Number of distinct keys among 100 random nonnegative integers less than 1000: 97
Number of distinct keys among 1000 random nonnegative integers less than 1000: 635
Number of distinct keys among 10000 random nonnegative integers less than 1000: 999
Number of distinct keys among 100000 random nonnegative integers less than 1000: 1000
Number of distinct keys among 1000000 random nonnegative integers less than 1000: 1000


Question: How many empty lists do you expect to see when you insert N keys into a hash table with SeparateChainingHashST, for N=10, 10^2, 10^3, 10^4, 10^5, and 10^6? Hint: See Exercise 2.5.31.

Answer:
3.1.8

Most frequently used word of ten letters or more in Tale of Two Cities: Monseigneur Frequency: 47


Question: Implement an eager delete() method for SeparateChainingHashST.

Answer:
3.1.9

Length cutoff Last word inserted  Words processed
            1            known."           135937
            8          faltering           135905
           10      disfigurement           135890


Question: Insert the keys E A S Y Q U T I O N in that order into an initially empty table of size M =16 using linear probing. Use the hash function 11 k % M to transform the kth letter of the alphabet into a table index. Redo this exercise for M = 10.

Answer:
3.1.10

key  value  first
 E     0      E0                                         0 compares
 A     1      A1  E0                                     1 compare
 S     2      S2  A1  E0                                 2 compares
 Y     3      Y3  S2  A1  E0                             3 compares
 Q     4      Q4  Y3  S2  A1  E0                         4 compares
 U     5      U5  Q4  Y3  S2  A1  E0                     5 compares
 E     6      U5  Q4  Y3  S2  A1  E6                     6 compares
 S     7      U5  Q4  Y3  S7  A1  E6                     4 compares
 T     8      T8  U5  Q4  Y3  S7  A1  E6                 6 compares
 I     9      I9  T8  U5  Q4  Y3  S7  A1  E6             7 compares
 O    10      O10  I9  T8  U5  Q4  Y3  S7  A1  E6        8 compares
 N    11      N11  O10  I9  T8  U5  Q4  Y3  S7  A1  E6   9 compares

Total: 55 compares

Thanks to ebber22 (https://github.com/ebber22) for finding an issue with the way the nodes were being added.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/102


Question: Give the contents of a linear-probing hash table that results when you insert the keys E A S Y Q U T I O N in that order into an initially empty table of initial size M = 4 that is expanded with doubling whenever half full. Use the hash function 11 k % M to transform the kth letter of the alphabet into a table index.

Answer:
3.1.11

			keys[]				            vals[]
key  value    0  1  2  3  4  5  6  7  8  9    N   0  1  2  3  4  5  6  7  8  9
 E     0      E                               1   0                               0 compares
 A     1      A  E                            2   1  0                            2 compares
 S     2      A  E  S                         3   1  0  2                         2 compares
 Y     3      A  E  S  Y                      4   1  0  2  3                      2 compares
 Q     4      A  E  Q  S  Y                   5   1  0  4  2  3                   3 compares
 U     5      A  E  Q  S  U  Y                6   1  0  4  2  5  3                4 compares
 E     6      A  E  Q  S  U  Y                6   1  6  4  2  5  3                4 compares
 S     7      A  E  Q  S  U  Y                6   1  6  4  7  5  3                4 compares
 T     8      A  E  Q  S  T  U  Y             6   1  6  4  7  8  5  3             4 compares
 I     9      A  E  I  Q  S  T  U  Y          6   1  6  9  4  7  8  5  3          4 compares
 O    10      A  E  I  O  Q  S  T  U  Y       6   1  6  9  10 4  7  8  5  3       4 compares
 N    11      A  E  I  N  O  Q  S  T  U  Y    6   1  6  9  11 10 4  7  8  5  3    5 compares
              A  E  I  N  O  Q  S  T  U  Y        1  6  9  11 10 4  7  8  5  3

Total: 38 compares


Question:
Which of the following scenarios leads to expected linear running time for a random search hit in a linear-probing hash table?
a. All keys hash to the same index.
b. All keys hash to different indices.
c. All keys hash to an even-numbered index.
d. All keys hash to different even-numbered indices.

Answer:
3.1.13

For an application that does 10^3 put() operations and 10^6 get() operations I would use a binary search symbol table implementation.
The application does a lot more get() than put() operations and a binary search symbol table implementation has a O(log(n)) runtime complexity for the get() operation, which is better than the O(n) runtime complexity of the sequential search symbol table implementation.


Question: Answer the previous question for search miss, assuming the search key is equally likely to hash to each table position.

Answer:
3.1.14

For an application that does 10^6 put() operations and 10^3 get() operations I would use a sequential search symbol table implementation.

The worst-case runtime cost of the put() operation for a binary search symbol table is 2N while for a sequential search symbol table it is N.
For the get() operation, the worst-case runtime cost of a binary search symbol table is lg(n) while for the sequential search symbol table it is N.

Comparing the total number of operations in the worst-case:
Sequential search symbol table: 10^6 + 10^3 ~ 10^6 operations
Binary search symbol table: 2 * 10^6 + lg(10^3) ~ 2 * 10^6 operations

The average-case runtime cost of both implementations in this case is similar because most of the operations are put() and both the sequential search symbol table and the binary search symbol table implementations have an average-case cost of N for it.

So for this application the sequential search symbol table performs better than the binary search symbol table in the worst-case and both perform similarly in the average-case, making the sequential search symbol table the best choice.

Thanks to dragon-dreamer (https://github.com/dragon-dreamer) for showing that the sequential search symbol table is the best option in this question.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/119


Question: How many compares could it take, in the worst case, to insert N keys into an initially empty table, using linear probing with array resizing?

Answer:
3.1.15

     Searches Percentage of total time spent on insertions
         1000                                        0.00%
      1000000                                        6.10%
   1000000000                                       96.26%

The average insertion cost is N, so the total insertion cost for N keys is N^2.
The average search cost is lg N, so the total search cost for M keys is M * lg N.
According to the question, M = 1000 * N.

The total cost is then:
total cost = N^2 + M * lg N = N^2 + 1000N * lg N

The insertion percentage is:
P = (N^2) / (N^2 + 1000N * lg N)
N^2 is the higher-order element in the equation, so as N increases, the insertion percentage approaches 100%.

Thanks to faame (https://github.com/faame) for improving this answer.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/222


Question: Add a constructor to SeparateChainingHashST that gives the client the ability to specify the average number of probes to be tolerated for searches. Use array resizing to keep the average list size less than the specified value, and use the technique described on page 478 to ensure that the modulus for hash() is prime.

Answer:
3.1.18

The rank() method in BinarySearchST always starts the search in the middle of the array if it has an odd number of elements.
If the array has an even number of elements, the rank() method starts the search on the left of the two middle elements.
After comparing the middle element with the search key, if it is smaller, the search continues on the right side of the array.
If it is bigger, the search continues on the left side of the array.
If they have the same value, the current element's index is the rank we are looking for.

This guarantees that if an element exists in the symbol table its rank will be found in the rank() method.

When the element does not exist the value of the "low" variable will have passed the value of the "high" variable, pointing to the correct rank of where the key should be. This only happens when both the element on the left of the final rank has been checked and the element on the current (final) rank has been checked. After these checks, low will be pointing to the correct rank location.

Example:

Symbol Table: 0 1 2 3 5 6
Rank of key 4 (non-existent)

1- The initial search range is [0..5]. The rank() method checks the left of the middle elements on index 2 -> value 2
2- 2 is less than 4, so the new range to search is [3..5]. The rank() method checks the middle element on index 4 -> value 5
3- 5 is more than 4, so the new range to search is [3..3]. The rank() method checks the only element left (index 3) -> value 3
4- 3 is less than 4, so the new range to search is [4..3]. Now the "low" variable is bigger than the "high" variable.
5- The rank() method returns the value of the "low" variable, 4. This is the correct rank for key 4.


Question: Add a method to LinearProbingHashST that computes the average cost of a search hit in the table, assuming that each key in the table is equally likely to be sought.

Answer:
3.1.20

Proposition B. Binary search in an ordered array with N keys uses no more than lg N + 1 compares for a search (successful or unsuccessful).

Proof: Let C(N) be the number of compares to search for a key in a symbol table of size N.
We have C(0) = 0, C(1) = 1, and for N > 0 we can write a recurrence relationship that directly mirrors the recursive method:
C(N) <= C(FLOOR(N / 2)) + 1

Whether the search goes to the left or to the right, the size of the subarray is no more than FLOOR(N / 2), and we can use one compare to check for equality and to choose whether to go left or right.

Let's first prove by induction that C(N) is monotonic: C(N) <= C(N + 1) for all N > 0.

It is trivial to prove that:
C(1) = 1
C(2) = 1 or 2

Thus we have:
C(1) <= C(2)

Assume for all N=[0, K]
C(N + 1) >= C(N)

Thus we have:
C(K + 1) >= C(K)

At the beginning of the binary search, low has value 0 and high has value N. Mid is computed as:
mid = low + (high - low) / 2 = high / 2

Let's use L to represent the size of the left half and R to represent the size of the right half.
The mid element has size 1.
When N = K + 1 the total size is L + 1 + R.

When N = K + 2:
mid = low + (high - low) / 2 = (high + 1) / 2 = (N + 1) / 2

When N is incremented by 1 (from K + 1 to K + 2), the mid point either remains in the same place or shifts to the right by 1.
1- If mid remains in the same place, then L remains the same, and R is incremented by 1.
Since (R + 1) <= K + 1, we have:
C(K + 2) = C(L) + 1 + C(R + 1) >= C(L) + 1 + C(R) = C(K + 1)
2- If mid shifts to the right by 1, then L is incremented by 1 and R remains the same.
Since (L + 1) <= K + 1, we have:
C(K + 2) = C(L + 1) + 1 + C(R) >= C(L) + 1 + C(R) = C(K + 1)

Therefore, given C(K + 1) >= C(K) it is also true that C(K + 2) >= C(K + 1), which proves that C(N) is monotonic.

For a general N, we have that:

C(N) <= C(N / 2) + 1 (one comparison to check equality or decide which way of the subarray to go)
C(N / 2) <= C(N / 4) + 1

Putting the value of C(N / 2) in the first equation:
C(N) <= C(N / 4) + 1 + 1

And adding all values to the first equation until C = 1:
C(N) <= C(N / 8) + 1 + 1 + 1
C(N) <= C(N / 16) + 1 + 1 + 1 + 1
C(N) <= C(N / 2^k) + 1 + 1 + 1 + 1 + ... + 1

until we get to
C(N) <= 1 + 1 + 1 + 1 + 1 + ... + 1 (even if N is not divisible by 2 there is still a compare operation)
C(N) <= k + 1

In this case, 2^k = N
k <= lg N + 1

We can also prove it using the Master theorem:

Binary search recurrence relation:
T(N) = T(N/2) + O(1)

Master theorem:
T(N) = aT(N/b) + f(N)

Here, a = 1, b = 2 and f(n) is O(1) (constant)

c = log(a base b) = log(1 base 2) = 0

We can see that this is the case 2 of the Master theorem by taking k = 0 in this equation:
O(n^c * (log n)^k)
O(n^0 * (log n)^0) = O(1) = f(n) -> This means we are in case 2 of the Master theorem

From the Case 2 of the Master Theorem we know that:
T(n) = O(n^(log a base b) * (log n)^(k + 1))
T(n) = O(n^0 * log(n)^1) = O(log n)

With binary search, we achieve a logarithmic-time search guarantee.

Reference: https://en.wikipedia.org/wiki/Master_theorem

Thanks to faame (https://github.com/faame) for adding the section proving that C(N) is monotonic.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/223


Question: Add a method to LinearProbingHashST that computes the average cost of a search miss in the table, assuming a random hash function. Note: You do not have to compute any hash functions to solve this problem.

Answer:
3.1.21 - Memory usage

* BinarySearchST
   object overhead -> 16 bytes
   Key[] reference (keys) -> 8 bytes
   Value[] reference (values) -> 8 bytes
   int value (size) -> 4 bytes
   padding -> 4 bytes
   Key[]
      object overhead -> 16 bytes
      int value (length) -> 4 bytes
      padding -> 4 bytes
      N Key references -> between 8N and 32N bytes (the resizing array may be 25% to 100% full)
   Value[]
      object overhead -> 16 bytes
      int value (length) -> 4 bytes
      padding -> 4 bytes
      N Value references -> between 8N and 32N bytes (the resizing array may be 25% to 100% full)
Amount of memory needed: 16 + 8 + 8 + 4 + 4 + 16 + 4 + 4 + (8N to 32N) + 16 + 4 + 4 + (8N to 32N) = (16N to 64N) + 88 bytes

* SequentialSearchST
   object overhead -> 16 bytes
   Node reference (first) -> 8 bytes
   Node
      object overhead -> 16 bytes
      extra overhead for reference to the enclosing instance -> 8 bytes
      Key reference (key) -> 8 bytes
      Value reference (value) -> 8 bytes
      Node reference (next) -> 8 bytes
            (N Node references -> 48N bytes)
   int value (size) -> 4 bytes
   padding -> 4 bytes
Amount of memory needed: 16 + 8 + (16 + 8 + 8 + 8 + 8)N + 4 + 4 = 48N + 32 bytes


Question: Consider modular hashing for string keys with R = 256 and M = 255. Show that this is a bad choice because any permutation of letters within a string hashes to the same value.

Answer:
3.1.23 - Analysis of binary search

As the book and exercise 3.1.20 have proven, the maximum number of compares used for a binary search in a table of size N is lg N + 1.

A number N has exactly lg N + 1 bits. This is because shifting 1 bit to the right reduces the number by half (rounded down).

For example:

N   Bit representation   Number of bits   lg N + 1
1                    1                1          1
2                   10                2          2
4                  100                3          3
5                  101                3          3
9                 1001                4          4

Therefore, the maximum number of compares used for a binary search in a table of size N is precisely the number of bits in the binary representation of N.


Question: Double probing. Modify SeparateChainingHashST to use a second hash function and pick the shorter of the two lists. Give a trace of the process of inserting the keys E A S Y Q U T I O N in that order into an initially empty table of size M =3 using the function 11 k % M (for the kth letter) as the first hash function and the function 17 k % M (for the kth letter) as the second hash function. Give the average number of probes for random search hit and search miss in this table.

Answer:
3.1.27 - Small tables

Building the binary search symbol table requires N calls to put().
Every put() operation makes a call to rank() and does a search, an operation with order of growth O(lg N).
Assuming that we can choose the order of the keys to insert, we can create the table in a sorted order, starting with the smallest element and ending with the highest element.
By doing this, every element will be inserted at the end of the keys[] and values[] array, making the put() operation use O(lg N) for the rank() operation and O(1) for the insert (there will be no need to move keys and values to the right since the new element is the rightmost element). N inserts will have an order of growth of O(N lg N).

A search operation has the order of growth O(lg N).
Therefore, the order of growth of S should be O(N), with S search operations having an order of growth O(N lg N), making the cost of building the table the same as the cost of all searches.

If the items are inserted in random order, the put operation has an order of growth O(N): O(lg N) for the rank operation and O(N) for inserting an element. N inserts will have an order of growth of O(N^2).
In this case, the order of growth of S should be O(N^2 / lg N).

Thanks to dragon-dreamer (https://github.com/dragon-dreamer) for correcting the orders of growth of S.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/120


Question: Cuckoo hashing. Develop a symbol-table implementation that maintains two hash tables and two hash functions. Any given key is in one of the tables, but not both. When inserting a new key, hash to one of the tables; if the table position is occupied, replace that key with the new key and hash the old key into the other table (again kicking out a key that might reside there). If this process cycles, restart. Keep the tables less than half full. This method uses a constant number of equality tests in the worst case for search (trivial) and amortized constant time for insert.

Answer:
3.1.31 - Performance driver

Number of Experiments |  Array Size |  AVG Time Taken
                    1        100000             12.32
                    2        100000             12.13
                    3        100000             12.15
                    4        100000             14.73
                    5        100000             19.76


Question:
Hash attack. Find 2^N strings, each of length 2^N, that have the same hashCode() value, supposing that the hashCode() implementation for String is the following:
public int hashCode() {
 int hash = 0;
 for (int i = 0; i < length(); i ++)
  hash = (hash * 31) + charAt(i);
 return hash;
}
Strong hint: Aa and BB have the same value.

Answer:
3.1.32 - Exercise driver

               Input Type |  Array Size |  Running Time
                  Ordered        100000             6.8
                  Ordered        200000            37.7
                  Ordered        400000           142.9
          Reverse Ordered        100000            24.1
          Reverse Ordered        200000           102.2
          Reverse Ordered        400000           469.6
                Same Keys        100000             0.0
                Same Keys        200000             0.0
                Same Keys        400000             0.0
  Keys with only 2 Values        100000             0.0
  Keys with only 2 Values        200000             0.0
  Keys with only 2 Values        400000             0.0


Question:
Bad hash function. Consider the following hashCode() implementation for String, which was used in early versions of Java:
public int hashCode() {
 int hash = 0;
 int skip = Math.max(1, length()/8);
 for (int i = 0; i < length(); i += skip)
  hash = (hash * 37) + charAt(i);
 return hash;
}
Explain why you think the designers chose this implementation and then why you think it was abandoned in favor of the one in the previous exercise.

Answer:
3.1.33 - Driver for self-organizing search

Array Size |  Self-Organizing Search Time |  Default Binary Search Time
      1000                           0.01                          0.01
     10000                           0.05                          0.01
    100000                           4.46                          0.10
   1000000                         576.02                          0.87


Question: Develop classes HashSETint and HashSETdouble for maintaining sets of keys of primitive int and double types, respectively. (Eliminate code involving values in your solution to Exercise 3.5.4.)

Answer:
3.1.6

The put() method will be called once for every word.
The get() method will be called once for every word, except on the first time a word in being inserted in the symbol table.

Calls to put() = W
Calls to get() = W - D

Thanks to faame (https://github.com/faame) for suggesting a better answer.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/221


Question: Develop classes SETint and SETdouble for maintaining ordered sets of keys of primitive int and double types, respectively. (Eliminate code involving values in your solution to Exercise 3.5.5.)

Answer:
3.1.7

Number of distinct keys among 10 random nonnegative integers less than 1000: 10
Number of distinct keys among 100 random nonnegative integers less than 1000: 97
Number of distinct keys among 1000 random nonnegative integers less than 1000: 635
Number of distinct keys among 10000 random nonnegative integers less than 1000: 999
Number of distinct keys among 100000 random nonnegative integers less than 1000: 1000
Number of distinct keys among 1000000 random nonnegative integers less than 1000: 1000


Question: Modify LinearProbingHashST to keep duplicate keys in the table. Return any value associated with the given key for get(), and remove all items in the table that have keys equal to the given key for delete().

Answer:
3.1.8

Most frequently used word of ten letters or more in Tale of Two Cities: Monseigneur Frequency: 47


Question: Modify BST to keep duplicate keys in the tree. Return any value associated with the given key for get(), and remove all nodes in the tree that have keys equal to the given key for delete().

Answer:
3.1.9

Length cutoff Last word inserted  Words processed
            1            known."           135937
            8          faltering           135905
           10      disfigurement           135890


Question: Modify RedBlackBST to keep duplicate keys in the tree. Return any value associated with the given key for get(), and remove all nodes in the tree that have keys equal to the given key for delete().

Answer:
3.1.10

key  value  first
 E     0      E0                                         0 compares
 A     1      A1  E0                                     1 compare
 S     2      S2  A1  E0                                 2 compares
 Y     3      Y3  S2  A1  E0                             3 compares
 Q     4      Q4  Y3  S2  A1  E0                         4 compares
 U     5      U5  Q4  Y3  S2  A1  E0                     5 compares
 E     6      U5  Q4  Y3  S2  A1  E6                     6 compares
 S     7      U5  Q4  Y3  S7  A1  E6                     4 compares
 T     8      T8  U5  Q4  Y3  S7  A1  E6                 6 compares
 I     9      I9  T8  U5  Q4  Y3  S7  A1  E6             7 compares
 O    10      O10  I9  T8  U5  Q4  Y3  S7  A1  E6        8 compares
 N    11      N11  O10  I9  T8  U5  Q4  Y3  S7  A1  E6   9 compares

Total: 55 compares

Thanks to ebber22 (https://github.com/ebber22) for finding an issue with the way the nodes were being added.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/102


Question: Develop a MultiSET class that is like SET, but allows equal keys and thus implements a mathematical multiset.

Answer:
3.1.11

			keys[]				            vals[]
key  value    0  1  2  3  4  5  6  7  8  9    N   0  1  2  3  4  5  6  7  8  9
 E     0      E                               1   0                               0 compares
 A     1      A  E                            2   1  0                            2 compares
 S     2      A  E  S                         3   1  0  2                         2 compares
 Y     3      A  E  S  Y                      4   1  0  2  3                      2 compares
 Q     4      A  E  Q  S  Y                   5   1  0  4  2  3                   3 compares
 U     5      A  E  Q  S  U  Y                6   1  0  4  2  5  3                4 compares
 E     6      A  E  Q  S  U  Y                6   1  6  4  2  5  3                4 compares
 S     7      A  E  Q  S  U  Y                6   1  6  4  7  5  3                4 compares
 T     8      A  E  Q  S  T  U  Y             6   1  6  4  7  8  5  3             4 compares
 I     9      A  E  I  Q  S  T  U  Y          6   1  6  9  4  7  8  5  3          4 compares
 O    10      A  E  I  O  Q  S  T  U  Y       6   1  6  9  10 4  7  8  5  3       4 compares
 N    11      A  E  I  N  O  Q  S  T  U  Y    6   1  6  9  11 10 4  7  8  5  3    5 compares
              A  E  I  N  O  Q  S  T  U  Y        1  6  9  11 10 4  7  8  5  3

Total: 38 compares


Question: Modify LookupCSV to make a program RangeLookupCSV that takes two key values from the standard input and prints all key-value pairs in the .csv file such that the key falls within the range specified.

Answer:
3.1.13

For an application that does 10^3 put() operations and 10^6 get() operations I would use a binary search symbol table implementation.
The application does a lot more get() than put() operations and a binary search symbol table implementation has a O(log(n)) runtime complexity for the get() operation, which is better than the O(n) runtime complexity of the sequential search symbol table implementation.


Question: Develop and test a static method invert() that takes as argument an ST<String, Bag<String>> and produces as return value the inverse of the given symbol table (a symbol table of the same type).

Answer:
3.1.14

For an application that does 10^6 put() operations and 10^3 get() operations I would use a sequential search symbol table implementation.

The worst-case runtime cost of the put() operation for a binary search symbol table is 2N while for a sequential search symbol table it is N.
For the get() operation, the worst-case runtime cost of a binary search symbol table is lg(n) while for the sequential search symbol table it is N.

Comparing the total number of operations in the worst-case:
Sequential search symbol table: 10^6 + 10^3 ~ 10^6 operations
Binary search symbol table: 2 * 10^6 + lg(10^3) ~ 2 * 10^6 operations

The average-case runtime cost of both implementations in this case is similar because most of the operations are put() and both the sequential search symbol table and the binary search symbol table implementations have an average-case cost of N for it.

So for this application the sequential search symbol table performs better than the binary search symbol table in the worst-case and both perform similarly in the average-case, making the sequential search symbol table the best choice.

Thanks to dragon-dreamer (https://github.com/dragon-dreamer) for showing that the sequential search symbol table is the best option in this question.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/119


Question: Write a program that takes a string on standard input and an integer k as command-line argument and puts on standard output a sorted list of the k-grams found in the string, each followed by its index in the string.

Answer:
3.1.15

     Searches Percentage of total time spent on insertions
         1000                                        0.00%
      1000000                                        6.10%
   1000000000                                       96.26%

The average insertion cost is N, so the total insertion cost for N keys is N^2.
The average search cost is lg N, so the total search cost for M keys is M * lg N.
According to the question, M = 1000 * N.

The total cost is then:
total cost = N^2 + M * lg N = N^2 + 1000N * lg N

The insertion percentage is:
P = (N^2) / (N^2 + 1000N * lg N)
N^2 is the higher-order element in the equation, so as N increases, the insertion percentage approaches 100%.

Thanks to faame (https://github.com/faame) for improving this answer.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/222


Question: Multisets. After referring to Exercises 3.5.2 and 3.5.3 and the previous exercise, develop APIs MultiHashSET and MultiSET for multisets (sets that can have equal keys) and implementations SeparateChainingMultiSET and BinarySearchMultiSET for multisets and ordered multisets, respectively.

Answer:
3.1.18

The rank() method in BinarySearchST always starts the search in the middle of the array if it has an odd number of elements.
If the array has an even number of elements, the rank() method starts the search on the left of the two middle elements.
After comparing the middle element with the search key, if it is smaller, the search continues on the right side of the array.
If it is bigger, the search continues on the left side of the array.
If they have the same value, the current element's index is the rank we are looking for.

This guarantees that if an element exists in the symbol table its rank will be found in the rank() method.

When the element does not exist the value of the "low" variable will have passed the value of the "high" variable, pointing to the correct rank of where the key should be. This only happens when both the element on the left of the final rank has been checked and the element on the current (final) rank has been checked. After these checks, low will be pointing to the correct rank location.

Example:

Symbol Table: 0 1 2 3 5 6
Rank of key 4 (non-existent)

1- The initial search range is [0..5]. The rank() method checks the left of the middle elements on index 2 -> value 2
2- 2 is less than 4, so the new range to search is [3..5]. The rank() method checks the middle element on index 4 -> value 5
3- 5 is more than 4, so the new range to search is [3..3]. The rank() method checks the only element left (index 3) -> value 3
4- 3 is less than 4, so the new range to search is [4..3]. Now the "low" variable is bigger than the "high" variable.
5- The rank() method returns the value of the "low" variable, 4. This is the correct rank for key 4.


Question: Concordance. Write an ST client Concordance that puts on standard output a concordance of the strings in the standard input stream (see page 498).

Answer:
3.1.20

Proposition B. Binary search in an ordered array with N keys uses no more than lg N + 1 compares for a search (successful or unsuccessful).

Proof: Let C(N) be the number of compares to search for a key in a symbol table of size N.
We have C(0) = 0, C(1) = 1, and for N > 0 we can write a recurrence relationship that directly mirrors the recursive method:
C(N) <= C(FLOOR(N / 2)) + 1

Whether the search goes to the left or to the right, the size of the subarray is no more than FLOOR(N / 2), and we can use one compare to check for equality and to choose whether to go left or right.

Let's first prove by induction that C(N) is monotonic: C(N) <= C(N + 1) for all N > 0.

It is trivial to prove that:
C(1) = 1
C(2) = 1 or 2

Thus we have:
C(1) <= C(2)

Assume for all N=[0, K]
C(N + 1) >= C(N)

Thus we have:
C(K + 1) >= C(K)

At the beginning of the binary search, low has value 0 and high has value N. Mid is computed as:
mid = low + (high - low) / 2 = high / 2

Let's use L to represent the size of the left half and R to represent the size of the right half.
The mid element has size 1.
When N = K + 1 the total size is L + 1 + R.

When N = K + 2:
mid = low + (high - low) / 2 = (high + 1) / 2 = (N + 1) / 2

When N is incremented by 1 (from K + 1 to K + 2), the mid point either remains in the same place or shifts to the right by 1.
1- If mid remains in the same place, then L remains the same, and R is incremented by 1.
Since (R + 1) <= K + 1, we have:
C(K + 2) = C(L) + 1 + C(R + 1) >= C(L) + 1 + C(R) = C(K + 1)
2- If mid shifts to the right by 1, then L is incremented by 1 and R remains the same.
Since (L + 1) <= K + 1, we have:
C(K + 2) = C(L + 1) + 1 + C(R) >= C(L) + 1 + C(R) = C(K + 1)

Therefore, given C(K + 1) >= C(K) it is also true that C(K + 2) >= C(K + 1), which proves that C(N) is monotonic.

For a general N, we have that:

C(N) <= C(N / 2) + 1 (one comparison to check equality or decide which way of the subarray to go)
C(N / 2) <= C(N / 4) + 1

Putting the value of C(N / 2) in the first equation:
C(N) <= C(N / 4) + 1 + 1

And adding all values to the first equation until C = 1:
C(N) <= C(N / 8) + 1 + 1 + 1
C(N) <= C(N / 16) + 1 + 1 + 1 + 1
C(N) <= C(N / 2^k) + 1 + 1 + 1 + 1 + ... + 1

until we get to
C(N) <= 1 + 1 + 1 + 1 + 1 + ... + 1 (even if N is not divisible by 2 there is still a compare operation)
C(N) <= k + 1

In this case, 2^k = N
k <= lg N + 1

We can also prove it using the Master theorem:

Binary search recurrence relation:
T(N) = T(N/2) + O(1)

Master theorem:
T(N) = aT(N/b) + f(N)

Here, a = 1, b = 2 and f(n) is O(1) (constant)

c = log(a base b) = log(1 base 2) = 0

We can see that this is the case 2 of the Master theorem by taking k = 0 in this equation:
O(n^c * (log n)^k)
O(n^0 * (log n)^0) = O(1) = f(n) -> This means we are in case 2 of the Master theorem

From the Case 2 of the Master Theorem we know that:
T(n) = O(n^(log a base b) * (log n)^(k + 1))
T(n) = O(n^0 * log(n)^1) = O(log n)

With binary search, we achieve a logarithmic-time search guarantee.

Reference: https://en.wikipedia.org/wiki/Master_theorem

Thanks to faame (https://github.com/faame) for adding the section proving that C(N) is monotonic.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/223


Question: Inverted concordance. Write a program InvertedConcordance that takes a concordance on standard input and puts the original string on standard output stream. Note: This computation is associated with a famous story having to do with the Dead Sea Scrolls. The team that discovered the original tablets enforced a secrecy rule that essentially resulted in their making public only a concordance. After a while, other researchers figured out how to invert the concordance, and the full text was eventually made public.

Answer:
3.1.21 - Memory usage

* BinarySearchST
   object overhead -> 16 bytes
   Key[] reference (keys) -> 8 bytes
   Value[] reference (values) -> 8 bytes
   int value (size) -> 4 bytes
   padding -> 4 bytes
   Key[]
      object overhead -> 16 bytes
      int value (length) -> 4 bytes
      padding -> 4 bytes
      N Key references -> between 8N and 32N bytes (the resizing array may be 25% to 100% full)
   Value[]
      object overhead -> 16 bytes
      int value (length) -> 4 bytes
      padding -> 4 bytes
      N Value references -> between 8N and 32N bytes (the resizing array may be 25% to 100% full)
Amount of memory needed: 16 + 8 + 8 + 4 + 4 + 16 + 4 + 4 + (8N to 32N) + 16 + 4 + 4 + (8N to 32N) = (16N to 64N) + 88 bytes

* SequentialSearchST
   object overhead -> 16 bytes
   Node reference (first) -> 8 bytes
   Node
      object overhead -> 16 bytes
      extra overhead for reference to the enclosing instance -> 8 bytes
      Key reference (key) -> 8 bytes
      Value reference (value) -> 8 bytes
      Node reference (next) -> 8 bytes
            (N Node references -> 48N bytes)
   int value (size) -> 4 bytes
   padding -> 4 bytes
Amount of memory needed: 16 + 8 + (16 + 8 + 8 + 8 + 8)N + 4 + 4 = 48N + 32 bytes


Question: Sparse matrices. Develop an API and an implementation for sparse 2D matrices. Support matrix addition and matrix multiplication. Include constructors for row and column vectors.

Answer:
3.1.23 - Analysis of binary search

As the book and exercise 3.1.20 have proven, the maximum number of compares used for a binary search in a table of size N is lg N + 1.

A number N has exactly lg N + 1 bits. This is because shifting 1 bit to the right reduces the number by half (rounded down).

For example:

N   Bit representation   Number of bits   lg N + 1
1                    1                1          1
2                   10                2          2
4                  100                3          3
5                  101                3          3
9                 1001                4          4

Therefore, the maximum number of compares used for a binary search in a table of size N is precisely the number of bits in the binary representation of N.


Question:
List. Develop an implementation of the following API:
public class List<Item> implements Iterable<Item>
 List() create a list
 void addFront(Item item) add item to the front
 void addBack(Item item) add item to the back
 Item deleteFront() remove from the front
 Item deleteBack() remove from the back
 void delete(Item item) remove item from the list
 void add(int i, Item item) add item as the ith in the list
 Item delete(int i) remove the ith item from the list
 boolean contains(Item item) is key in the list?
 boolean isEmpty() is the list empty?
 int size() number of items in the list
Hint: Use two symbol tables, one to find the ith item in the list efficiently, and the other to efficiently search by item. (Java’s java.util.List interface contains methods like these but does not supply any implementation that efficiently supports all operations.)

Answer:
3.1.27 - Small tables

Building the binary search symbol table requires N calls to put().
Every put() operation makes a call to rank() and does a search, an operation with order of growth O(lg N).
Assuming that we can choose the order of the keys to insert, we can create the table in a sorted order, starting with the smallest element and ending with the highest element.
By doing this, every element will be inserted at the end of the keys[] and values[] array, making the put() operation use O(lg N) for the rank() operation and O(1) for the insert (there will be no need to move keys and values to the right since the new element is the rightmost element). N inserts will have an order of growth of O(N lg N).

A search operation has the order of growth O(lg N).
Therefore, the order of growth of S should be O(N), with S search operations having an order of growth O(N lg N), making the cost of building the table the same as the cost of all searches.

If the items are inserted in random order, the put operation has an order of growth O(N): O(lg N) for the rank operation and O(N) for inserting an element. N inserts will have an order of growth of O(N^2).
In this case, the order of growth of S should be O(N^2 / lg N).

Thanks to dragon-dreamer (https://github.com/dragon-dreamer) for correcting the orders of growth of S.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/120


Question: What is the maximum number of edges in a graph with V vertices and no parallel edges? What is the minimum number of edges in a graph with V vertices, none of which are isolated?

Answer:
4.1.1

The maximum number of edges in a graph with V vertices and no parallel edges is V * (V - 1) / 2.
Since we do not have self-loops or parallel edges, each vertex can connect to V - 1 other vertices. In an undirected graph vertex v connected to vertex w is the same as vertex w connected to vertex v, so we divide the result by 2.

Example:
o - o
| X |
o — o

The minimum number of edges in a graph with V vertices, none of which are isolated (have degree 0) is V - 1.
Example:
o — o — o


Question: Draw, in the style of the figure in the text (page 524), the adjacency lists built by Graph’s input stream constructor for the file tinyGex2.txt depicted at left.

Answer:
4.1.2

adj[]
  0  -> 5 -> 2 -> 6
  1  -> 4 -> 8 -> 11
  2  -> 5 -> 6 -> 0 -> 3
  3  -> 10 -> 6 -> 2
  4  -> 1 -> 8
  5  -> 0 -> 10 -> 2
  6  -> 2 -> 3 -> 0
  7  -> 8 -> 11
  8  -> 1 -> 11 -> 7 -> 4
  9  ->
 10  -> 5 -> 3
 11  -> 8 -> 7 -> 1


Question: Consider the four-vertex graph with edges 0-1, 1-2, 2-3, and 3-0. Draw an array of adjacency-lists that could not have been built calling addEdge() for these edges no matter what order.

Answer:
4.1.6

The edges form a cycle, so changing the connection order of one of the vertices' adjacency list creates an impossible sequence of connections.

adj[]
  0  -> 1 -> 3 (the original was 0 -> 3 -> 1)
  1  -> 2 -> 0
  2  -> 3 -> 1
  3  -> 0 -> 2


Question: Show, in the style of the figure on page 533, a detailed trace of the call dfs(0) for the graph built by Graph’s input stream constructor for the file tinyGex2.txt (see Exercise 4.1.2). Also, draw the tree represented by edgeTo[].

Answer:
4.1.9
                marked[]    adj[]
dfs (0)          0 T        0 5 2 6
                 1          1 4 8 11
                 2          2 5 6 0 3
                 3          3 10 6 2
                 4          4 1 8
                 5          5 0 10 2
                 6          6 2 3 0
                 7          7 8 11
                 8          8 1 11 7 4
                 9          9
                10         10 5 3
                11         11 8 7 1

 dfs (5)         0 T        0 5 2 6
  check 0        1          1 4 8 11
                 2          2 5 6 0 3
                 3          3 10 6 2
                 4          4 1 8
                 5 T        5 0 10 2
                 6          6 2 3 0
                 7          7 8 11
                 8          8 1 11 7 4
                 9          9
                10         10 5 3
                11         11 8 7 1

  dfs (10)       0 T        0 5 2 6
   check 5       1          1 4 8 11
                 2          2 5 6 0 3
                 3          3 10 6 2
                 4          4 1 8
                 5 T        5 0 10 2
                 6          6 2 3 0
                 7          7 8 11
                 8          8 1 11 7 4
                 9          9
                10 T       10 5 3
                11         11 8 7 1

   dfs (3)       0 T        0 5 2 6
    check 10     1          1 4 8 11
                 2          2 5 6 0 3
                 3 T        3 10 6 2
                 4          4 1 8
                 5 T        5 0 10 2
                 6          6 2 3 0
                 7          7 8 11
                 8          8 1 11 7 4
                 9          9
                10 T       10 5 3
                11         11 8 7 1

    dfs (6)      0 T        0 5 2 6
                 1          1 4 8 11
                 2          2 5 6 0 3
                 3 T        3 10 6 2
                 4          4 1 8
                 5 T        5 0 10 2
                 6 T        6 2 3 0
                 7          7 8 11
                 8          8 1 11 7 4
                 9          9
                10 T       10 5 3
                11         11 8 7 1

     dfs (2)     0 T        0 5 2 6
      check 5    1          1 4 8 11
      check 6    2 T        2 5 6 0 3
      check 0    3 T        3 10 6 2
      check 3    4          4 1 8
     2 done      5 T        5 0 10 2
                 6 T        6 2 3 0
                 7          7 8 11
                 8          8 1 11 7 4
                 9          9
                10 T       10 5 3
                11         11 8 7 1
     check 3
     check 0
    6 done
    check 2
   3 done
  10 done
  check 2
 5 done
 check 2
 check 6
0 done

edgeTo[] tree

           0
          5
        10
       3
      6
     2