Question: Shortest paths in Euclidean graphs. Adapt our APIs to speed up Dijkstra’s algorithm in the case where it is known that vertices are points in the plane.

Answer:
4.1.27

Integer
* object overhead -> 16 bytes
* int value -> 4 bytes
* padding -> 4 bytes
Amount of memory needed: 16 + 4 + 4 = 24 bytes

Node
* object overhead -> 16 bytes
* extra overhead for reference to the enclosing instance -> 8 bytes
* Item reference (item) -> 8 bytes
* Node reference (next) -> 8 bytes
Amount of memory needed: 16 + 8 + 8 + 8 = 40 bytes

Bag
* object overhead -> 16 bytes
* Node reference (first) -> 8 bytes
* int value (size) -> 4 bytes
* padding -> 4 bytes
* N Nodes -> 40N bytes
* Integer (item) -> 24N bytes
Amount of memory needed: 16 + 8 + 4 + 4 + 40N + 24N = 64N + 32 bytes

Graph
* object overhead -> 16 bytes
* int value (V) -> 4 bytes
* int value (E) -> 4 bytes
* Bag<Integer>[] reference (adj) -> 8 bytes
* Bag<Integer>[] (adj)
   object overhead -> 16 bytes
   int value (length) -> 4 bytes
   padding -> 4 bytes
   Bag references -> 8V
   Bag -> 64E + 32 bytes -> There are V Bags and in total, they have 2E nodes -> 128E + 32V
Amount of memory needed: 16 + 4 + 4 + 8 + 16 + 4 + 4 + 8V + 128E + 32V = 128E + 40V + 56 bytes


Question: Longest paths in DAGs. Develop an implementation AcyclicLP that can solve the longest-paths problem in edge-weighted DAGs, as described in PROPOSITION T.

Answer:
4.1.28

Non-isomorphic graphs:

There are 2 non-isomorphic graphs with 2 vertices:
o o

o-o

There are 4 non-isomorphic graphs with 3 vertices:
 o
o o

 o
o-o

o-o-o

 o
/ \
o-o

There are 11 non-isomorphic graphs with 4 vertices:
o o o o

o-o o o

o-o-o o

o-o-o-o

o-o o-o

  o
  |
  o
 / \
o   o

  o
 / \
o---o o

o-o
| |
o-o

  o
  |
  o
 / \
o---o

  o
 / \
o---o
 \ /
  o

   o
  /|\
 / o \
/ / \ \
o------o

There are 34 non-isomorphic graphs with 5 vertices:
  o
o   o
 o o

  o
o   o
 o-o

  o
o    o
    /
 o-o

   o
o     o
 \   /
  o o

   o
 / | \
o  o  o
     o

  o
 / \
o   o
 o-o

   o
 /  \
o    o
 \
  o o

   o
 /  \
o----o
  o o

   o
 /|\\
o o oo

o-o
| |
o-o o

o
|
o-o
| |
o o

  o
  |
  o
 / \
o---o o

o-o-o-o-o

  o  o
 /|  |
o |  |
 \|  |
  o  o

o--o
|  |
o--o
|
o

o   o
|   |
o---o
 \ /
  o

 o--o
  \/
o--o--o

   o
 /  \
o    o
 \   /
  o-o

    o   (This is a complete graph, where all vertices have degree = 4)
 / / \\
o------o
\ /\ /\/
 \|/\ |/
  o---o

    o
 / / \\
o------o
\ /\ /\/
 \|/\ |/
  o   o

o---o
|\ /|\
| X | o
|/ \|/
o---o

o---o
|\ /|
| o |
|/ \|
o---o

o---o
|\ /|\
| X | o
|/ \|
o---o

   o
 // \
o-o  o
   \/
    o

o-o-o-o
 \| | /
   o

    o
   /\
  /  \
 /    \
o------o
\  \ / /
 \ /\ /
  o  o

o---o
| X | o
o---o

o   o
|\ /|
| o |
|/ \|
o   o

     o
    /|\
o--o | o
    \|/
     o

  o
  |
  o
 /|\
o | o
 \|/
  o

  o
 / \
o---o
|   |
o---o

o---o
|  /|
| o |
|/  |
o---o

  o
  |
  o
  |
  o
 / \
o---o

o---o
| \ |
o---o o

Based on: http://www.graphclasses.org/smallgraphs.html


Question: All-pairs shortest path in graphs with negative cycles. Articulate an API like the one implemented on page 656 for the all-pairs shortest-paths problem in graphs with no negative cycles. Develop an implementation that runs a version of Bellman-Ford to identify weights pi[v] such that for any edge v->w, the edge weight plus the difference between pi[v] and pi[w] is nonnegative. Then use these weights to reweight the graph, so that Dijkstra’s algorithm is effective for finding all shortest paths in the reweighted graph.

Answer:
4.1.30 - Eulerian and Hamiltonian cycles

An Eulerian cycle (or Eulerian circuit) is a path which starts and ends at the same vertex and includes every edge exactly once.
A Hamiltonian cycle is a path which starts and ends at the same vertex and includes every vertex exactly once (except for the source, which is visited twice).

According to Euler theorems a graph has an Eulerian cycle/circuit if and only if it does not have any vertices of odd degree.

First graph:
0-1 0-2 0-3 1-3 1-4 2-5 2-9 3-6 4-7 4-8 5-8 5-9 6-7 6-9 7-8

It does not have an Eulerian cycle because it has vertices of odd degree (0, 1, 2, 3, 4, 5, 6, 7, 8 and 9).
It has a Hamiltonian cycle: 1-4 4-8 8-7 7-6 6-9 9-5 5-2 2-0 0-3 3-1

Second graph:
0-1 0-2 0-3 1-3 0-3 2-5 5-6 3-6 4-7 4-8 5-8 5-9 6-7 6-9 8-8

It has an Eulerian cycle (all the vertices have even degrees):
0-3 3-0 0-2 2-5 5-9 9-6 6-5 5-8 8-8 8-4 4-7 7-6 6-3 3-1 1-0
There is no Hamiltonian cycle.

Third graph:
0-1 1-2 1-3 0-3 0-4 2-5 2-9 3-6 4-7 4-8 5-8 5-9 6-7 6-9 7-8

It does not have an Eulerian cycle because it has vertices of odd degree (0, 1, 2, 3, 4, 5, 6, 7, 8, 9).
It has a Hamiltonian cycle: 4-8 8-7 7-6 6-9 9-5 5-2 2-1 1-3 3-0 0-4

Fourth graph:
4-1 7-9 6-2 7-3 5-0 0-2 0-8 1-6 3-9 6-3 2-8 1-5 9-8 4-5 4-7

It does not have an Eulerian cycle because it has vertices of odd degree (0, 1, 2, 3, 4, 5, 6, 7, 8, 9).
It has a Hamiltonian cycle: 0-5 5-4 4-1 1-6 6-3 3-7 7-9 9-8 8-2 2-0


Question: All-pairs shortest path on a line. Given a weighted line graph (undirected connected graph, all vertices of degree 2, except two endpoints which have degree 1), devise an algorithm that preprocesses the graph in linear time and can return the distance of the shortest path between any two vertices in constant time.

Answer:
4.1.31 - Graph enumeration

If we were considering graphs with no self-loops:

There are (V) ways to choose a set {u, v} of two vertices. From this set there are E ways to choose the vertices to connect.
          (2)
So there are ((V)) = (V! / 2! * (V - 2)!)! / E! * ((V! / 2! * (V - 2)!) - E)!
             ((2))
             ( E )
different undirected graphs with V vertices and E edges (and no parallel edges and no self-loops).

Since we are also considering graphs with self-loops, there are (V) * 2^V ways to choose a set {u, v} of two vertices in which
                                                                (2)
vertices may or may not have a self-loop.

So there are ((V) * 2^V) = ((V! / 2! * (V - 2)!) * 2^V)! / E! * (((V! / 2! * (V - 2)!) * 2^V) - E)!
             ((2)      )
             (    E    )
different undirected graphs with V vertices and E edges (and no parallel edges).

Reference:
Handbook of Discrete and Combinatorial Mathematics by Kenneth H. Rosen, page 580
https://math.stackexchange.com/questions/1072726/counting-simple-connected-labeled-graphs-with-n-vertices-and-k-edges
https://math.stackexchange.com/questions/128439/how-to-determine-the-number-of-directed-undirected-graphs


Question: Shortest path in a grid. Given an N-by-N matrix of positive integers, find the shortest path from the (0, 0) entry to the (N-1, N-1) entry, where the length of the path is the sum of the integers in the path. Repeat the problem but assume you can only move right and down.

Answer:
4.1.33 - Odd cycles

A graph is two-colorable (bipartite) if and only if it contains no odd-length cycle.

Proof:
1- Proving that a graph with an odd-length cycle cannot be bipartite:
If a graph G is bipartite with vertex sets V1 and V2, every step along a walk takes you either from V1 to V2 or from V2 to V1. To end up where you started, therefore, you must take an even number of steps.

2- Proving that a graph with only even-length cycles is bipartite:
Consider G to be a graph with only even-length cycles. Let v0 be any vertex. For each vertex v in the same component C0 as v0 let d(v) be the length of the shortest path from v0 to v. Color red every vertex in C0 whose distance from v0 is even, and color the other vertices of C0 blue. Do the same for each component of G. Check that if G had any edge between two red vertices or between two blue vertices, it would have an odd cycle. Thus, G is bipartite, the red vertices and the blue vertices being the two parts.

Reference:
https://math.stackexchange.com/questions/311665/proof-a-graph-is-bipartite-if-and-only-if-it-contains-no-odd-cycles


Question: Bitonic shortest path. Given a digraph, find a bitonic shortest path from s to every other vertex (if one exists). A path is bitonic if there is an intermediate vertex v such that the edges on the path from s to v are strictly increasing and the edges on the path from v to t are strictly decreasing. The path should be simple (no repeated vertices).

Answer:
4.1.35 - Biconnectedness

Any graph with no articulation points is biconnected.

Proof:
Consider two vertices, s, t, and a path P1 connecting s to t.
We know that no vertex in P1 is an articulation point, so for each vertex v in the path, there is always another path P2 connecting s to t that does not include it. Also, P2 does not include any of the vertices of P1, otherwise any vertex included in both paths would be an articulation point (being the only way to connect s to t).
This means that every pair of vertices is connected by two vertex-disjoint paths (such as P1 and P2), making the graph biconnected.

Graph illustration (P1 is the path s-v1 v1-v2 v2-t and P2 is the path s-v3 v3-v4 v4-t):

s
|\
| \
v1 v3
|   \
|   v4
v2 /
| /
|/
t


Question:
Give a trace for LSD string sort for the keys
 no is th ti fo al go pe to co to th ai of th pa

Answer:
5.1.2

Trace for LSD string sort (same model as used in the book):

input   d=1   d=0   output
no      pa    ai    ai
is      pe    al    al
th      of    co    co
ti      th    fo    fo
fo      th    go    go
al      th    is    is
go      ti    no    no
pe      ai    of    of
to      al    pa    pa
co      no    pe    pe
to      fo    th    th
th      go    th    th
ai      to    th    th
of      co    ti    ti
th      to    to    to
pa      is    to    to


Question:
Give a trace for MSD string sort for the keys
 no is th ti fo al go pe to co to th ai of th pa

Answer:
5.1.3

According to ASC II values, indices of chars 'a' through 'z' start at 97.
But this trace indices follow the book convention of 'a' starting at 0.

Trace for MSD string sort (same model as used in the book):

Top level of sort(array, 0, 15, 0):

input
0  no
1  is
2  th
3  ti
4  fo
5  al
6  go
7  pe
8  to
9  co
10 to
11 th
12 ai
13 of
14 th
15 pa

d=0
Count frequencies
0    0
1    0
2  a 2
3  b 0
4  c 1
5  d 0
6  e 0
7  f 1
8  g 1
9  h 0
10 i 1
11 j 0
12 k 0
13 l 0
14 m 0
15 n 1
16 o 1
17 p 2
18 q 0
19 r 0
20 s 0
21 t 6
22 u 0
23 v 0
24 x 0
25 w 0
26 y 0
27 z 0

Transform counts to indices
0    0
1    0
2  a 2
3  b 2
4  c 3
5  d 3
6  e 3
7  f 4
8  g 5
9  h 5
10 i 6
11 j 6
12 k 6
13 l 6
14 m 6
15 n 7
16 o 8
17 p 10
18 q 10
19 r 10
20 s 10
21 t 16
22 u 16
23 v 16
24 x 16
25 w 16
26 y 16
27 z 16

Distribute and copy back
0  al
1  ai
2  co
3  fo
4  go
5  is
6  no
7  of
8  pe
9  pa
10 th
11 ti
12 to
13 to
14 th
15 th

Indices at completion of distribute phase
0    0
1    2
2  a 2
3  b 3
4  c 3
5  d 3
6  e 4
7  f 5
8  g 5
9  h 6
10 i 6
11 j 6
12 k 6
13 l 6
14 m 7
15 n 8
16 o 10
17 p 10
18 q 10
19 r 10
20 s 16
21 t 16
22 u 16
23 v 16
24 x 16
25 w 16
26 y 16
27 z 16

Recursively sort subarrays
sort(a, 0, 1, 1);
sort(a, 2, 1, 1);
sort(a, 2, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 3, 1);
sort(a, 4, 4, 1);
sort(a, 5, 4, 1);
sort(a, 5, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 6, 1);
sort(a, 7, 7, 1);
sort(a, 8, 9, 1);
sort(a, 10, 9, 1);
sort(a, 10, 9, 1);
sort(a, 10, 9, 1);
sort(a, 10, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);

Sorted result
0  ai
1  al
2  co
3  fo
4  go
5  is
6  no
7  of
8  pa
9  pe
10 th
11 th
12 th
13 ti
14 to
15 to

Trace of recursive calls for MSD string sort (no cutoff for small subarrays, subarrays of size 0 and 1 omitted)

input   __      __                      output
no      al      ai      ai      ai      ai
is      ai      al      al      al      al
th      co      --      co      co      co
ti      fo      co      fo      fo      fo
fo      go      fo      go      go      go
al      is      go      is      is      is
go      no      is      no      no      no
pe      of      no      of      of      of
to      pe      of      --      pa      pa
co      pa      pe      pa      pe      pe
to      th      pa      pe      --      th
th      ti      th      --      th      th
ai      to      ti      th      th      th
of      to      to      ti      th      ti
th      th      to      to      ti      to
pa      th      th      to      to      to
        --      th      th      to
                        th      --


Question:
Give a trace for 3-way string quicksort for the keys
 no is th ti fo al go pe to co to th ai of th pa

Answer:
5.1.4

Trace for 3-way string quicksort (same model as used in the book):

                   --      --      --
0  no      is      ai      ai      ai      ai
1  is      ai      co      al      --      al
2  th      co      fo      --      al      co
3  ti      fo      al      fo      --      fo
4  fo      al      go      go      co      go
5  al      go      --      co      --      is
6  go      --      is      --      fo      no
7  pe      no      --      is      --      of
8  to      --      no      --      go      pa
9  co      to      --      no      --      pe
10 to      pe      pe      --      is      th
11 th      to      of      of      --      th
12 ai      th      pa      --      no      th
13 of      ti      --      pe      --      ti
14 th      of      th      pa      of      to
15 pa      th      ti      --      --      to
           pa      to      th      pa
           th      th      th      --
                   to      th      pe
                   th      --      --
                   --      to      th
                           to      th
                           ti      th
                                   --
                                   ti
                                   --
                                   to
                                   to
                                   --


Question:
Give a trace for MSD string sort for the keys
 now is the time for all good people to come to the aid of

Answer:
5.1.5

According to ASC II values, indices of chars 'a' through 'z' start at 97.
But this trace indices follow the book convention of 'a' starting at 0.

Trace for MSD string sort (same model as used in the book):

Top level of sort(array, 0, 13, 0):

input
0  now
1  is
2  the
3  time
4  for
5  all
6  good
7  people
8  to
9  come
10 to
11 the
12 aid
13 of

d=0
Count frequencies
0    0
1    0
2  a 2
3  b 0
4  c 1
5  d 0
6  e 0
7  f 1
8  g 1
9  h 0
10 i 1
11 j 0
12 k 0
13 l 0
14 m 0
15 n 1
16 o 1
17 p 1
18 q 0
19 r 0
20 s 0
21 t 5
22 u 0
23 v 0
24 x 0
25 w 0
26 y 0
27 z 0

Transform counts to indices
0    0
1    0
2  a 2
3  b 2
4  c 3
5  d 3
6  e 3
7  f 4
8  g 5
9  h 5
10 i 6
11 j 6
12 k 6
13 l 6
14 m 6
15 n 7
16 o 8
17 p 9
18 q 9
19 r 9
20 s 9
21 t 14
22 u 14
23 v 14
24 x 14
25 w 14
26 y 14
27 z 14

Distribute and copy back
0  all
1  aid
2  come
3  for
4  good
5  is
6  now
7  of
8  people
9  the
10 time
11 to
12 to
13 the

Indices at completion of distribute phase
0    0
1    2
2  a 2
3  b 3
4  c 3
5  d 3
6  e 4
7  f 5
8  g 5
9  h 6
10 i 6
11 j 6
12 k 6
13 l 6
14 m 7
15 n 8
16 o 9
17 p 9
18 q 9
19 r 9
20 s 14
21 t 14
22 u 14
23 v 14
24 x 14
25 w 14
26 y 14
27 z 14

Recursively sort subarrays
sort(a, 0, 1, 1);
sort(a, 2, 1, 1);
sort(a, 2, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 3, 1);
sort(a, 4, 4, 1);
sort(a, 5, 4, 1);
sort(a, 5, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 6, 1);
sort(a, 7, 7, 1);
sort(a, 8, 8, 1);
sort(a, 9, 8, 1);
sort(a, 9, 8, 1);
sort(a, 9, 8, 1);
sort(a, 9, 13, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);

Sorted result
0  aid
1  all
2  come
3  for
4  good
5  is
6  now
7  of
8  people
9  the
10 the
11 time
12 to
13 to

Trace of recursive calls for MSD string sort (no cutoff for small subarrays, subarrays of size 0 and 1 omitted)

input       ____        ___                                 output
now         all         aid         aid         aid         aid
is          aid         all         all         all         all
the         come        ---         come        come        come
time        for         come        for         for         for
for         good        for         good        good        good
all         is          good        is          is          is
good        now         is          now         now         now
people      of          now         of          of          of
to          people      of          people      people      people
come        the         people      ---         ---         the
to          time        the         the         the         the
the         to          time        the         the         time
aid         to          to          time        ---         to
of          the         to          to          time        to
            ----        the         to          to
                                    ---         to


Question:
Give a trace for 3-way string quicksort for the keys
 now is the time for all good people to come to the aid of

Answer:
5.1.6

Trace for 3-way string quicksort (same model as used in the book):

                    ----     ---      ---      ---
0  now      is      aid      aid      aid      aid      aid
1  is       aid     come     all      ---      ---      all
2  the      come    for      ---      all      all      come
3  time     for     all      for      ---      ---      for
4  for      all     good     good     come     come     good
5  all      good    ----     come     ----     ----     is
6  good     ---     is       ----     for      for      now
7  people   now     ---      is       ----     ----     of
8  to       ---     now      ---      good     good     people
9  come     to      ---      now      ----     ----     the
10 to       people  people   ---      is       is       the
11 the      to      of       of       ---      ---      time
12 aid      the     ---      ------   now      now      to
13 of       time    to       people   ---      ---      to
            of      the      ------   of       of
            the     time     the      ------   ------
                    to       time     people   people
                    the      the      ------   ------
                    ---      ----     the      the
                             to       the      the
                             to       ----     ----
                             ----     time     time
                                      ----     ----
                                      to       to
                                      to       to
                                      --       --


Question: Give the number of characters examined by MSD string sort and 3-way string quicksort for a file of N keys a, aa, aaa, aaaa, aaaaa, . . .

Answer:
5.1.8

Both MSD string sort and 3-way string quicksort examine all characters in the N keys.
That number is equal to 1 + 2 + ... + N = (N^2 + N) / 2 characters.
MSD string sort, however, generates (R - 1) * N empty subarrays (an empty subarray for all digits in R other than 'a', in every pass) while 3-way string quicksort generates 2N empty subarrays (empty subarrays for digits smaller than 'a' and for digits higher than 'a', or empty subarrays for digits smaller than '-1' and for digits equal to '-1', in every pass).

MSD string sort trace (no cutoff for small subarrays, subarrays of size 0 and 1 omitted):

input   ----
a       a       a       a       a       a       a
aa      aa      ----    aa      aa      aa      aa
aaa     aaa     aa      ----    aaa     aaa     aaa
aaaa    aaaa    aaa     aaa     ----    aaaa    aaaa
...     ...     aaaa    aaaa    aaaa    ----    ...
        ----    ...     ...     ...     ...
                ----    ----    ----    ----

3-way string quicksort trace:

input   ----
a       a       a       a       a       a       a
aa      aa      ----    aa      aa      aa      aa
aaa     aaa     aa      ----    aaa     aaa     aaa
aaaa    aaaa    aaa     aaa     ----    aaaa    aaaa
...     ...     aaaa    aaaa    aaaa    ----    ...
        ----    ...     ...     ...     ...
                ----    ----    ----    ----


Question: What is the total number of characters examined by 3-way string quicksort when sorting N fixed-length strings (all of length W), in the worst case?

Answer:
5.1.10

The total number of characters examined by 3-way string quicksort when sorting N fixed-length strings (all of length W) in the worst case is O(N * W * R).

This can be seen with a recurrence relation T(W).

The base case T(1) is when all the strings have length 1.
An example with R = 3 is { "a", "b", "c" }.
In the worst case they are in reverse order.
For example: { "c", "b", "a" }.
In this case we only remove one string from the list in each pass.
If we consider N = R^W (in this case, W = 1), the number of comparisons is equal to:
Characters examined = Sum[i=0..R] i
Characters examined = R * (R + 1) / 2

To build the worst case for strings of length 2 (T(2)), we take each string from T(1) and append it to the end of each character in R.
So for single character strings "a", "b", "c", with R = 3, the two character list is: "aa", "ab", "ac", "ba", "bb", "bc", "ca", "cb", "cc".
The list can then be split into R groups: one for each character in R that is a prefix to every string of length W - 1.
During the partitioning phase all strings that start with "a" will be in the same partition and the algorithm will do the same process as in T(1) because removing the first character 'a' will lead to the same 1-length strings { "c", "b", "a" } as before.
The same thing happens for strings starting with "b" and "c".
So, for R = 3, the algorithm will check 3 * R + 2 * R + R characters in the first position of the strings (which is 3 + 2 + 1 characters times R groups).
Then it will check the second characters in the strings in each of the R groups.

For T(W), where W > 2, the list will then again be split into R groups: one for each character in R that is a prefix to every string of length W - 2.
Quicksort will then remove R strings from the list in each partition.
It will then check R * T(W - 1) more characters for each of those groups.
This gives the recurrence T(W) = (R^(W - 1) * Sum[i=0..R] R - i) + R * T(W - 1), which simplifies to:
T(W) = R^(W + 1) + R^W  + R * T(W - 1)
      -----------------
              2
Solving the recurrence gives us:
T(W) = W * (R^W) * (R + 1)
      ---------------------
                2
Substituting N = R^W:
T(W) = W * N * (R + 1)
      -----------------
              2
Which is O(N * W * R).

Thanks to dragon-dreamer (https://github.com/dragon-dreamer) for finding a more accurate worst case.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/153
Thanks to GenevaS (https://github.com/GenevaS) for finding a more accurate worst case.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/245


Question: Hybrid sort. Investigate the idea of using standard MSD string sort for large arrays, in order to get the advantage of multiway partitioning, and 3-way string quicksort for smaller arrays, in order to avoid the negative effects of large numbers of empty bins.

Answer:
5.1.13 - Hybrid sort

Idea: using standard MSD string sort for large arrays, in order to get the advantage of multiway partitioning, and 3-way string quicksort for smaller arrays, in order to avoid the negative effects of large numbers of empty bins.

This idea will work well for random strings because, in general, the higher the number of keys to be sorted, the higher the number of non-empty subarrays generated on each pass of MSD string sort. Such scenario would work well due to the advantage of having multiway partitioning.
However, MSD string sort will still generate a large number of empty subarrays if there is a large number of equal keys (or a large number of keys with long common prefixes).

3-way string quicksort will avoid the negative effects of large numbers of empty bins not only for smaller arrays, but also for large arrays, while also having the benefit of using less space than MSD string sort since it does not require space for frequency counts or for an auxiliary array. On the other hand, it envolves more data movement than MSD string sort when the number of nonempty subarrays is large because it has to do a series of 3-way partitions to get the effect of the multiway partition. This would not be a problem in the hybrid sort if there were many equal keys in smaller arrays, since 3-way string quicksort would be the algorithm of choice in such situation.

Overall, hybrid sort would be a good choice for random strings. However, a version of hybrid sort that chooses between MSD string sort and 3-way string quicksort based on the percentage of equal keys (choosing MSD string sort if there is a low percentage of equal keys and choosing 3-way string quicksort if there is a high number of equal keys) would be more effective than a version that makes the choice based on the number of keys.


Question: In-place key-indexed counting. Develop a version of key-indexed counting that uses only a constant amount of extra space. Prove that your version is stable or provide a counterexample.

Answer:
5.1.17 - In-place key-indexed counting

LSD and MSD sorts that use only a constant amount of extra space are not stable.

Counterexample for LSD sort:

The array ["4PGC938", "2IYE230", "3CIO720", "1ICK750", "1OHV845", "4JZY524", "1ICK750", "3CIO720", "1OHV845", "1OHV845", "2RLA629", "2RLA629", "3ATW723"] after being sorted by in-place LSD becomes:
["1OHV845", "1OHV845", "1OHV845", "1ICK750", "1ICK750", "2RLA629", "2IYE230", "2RLA629", "3ATW723", "3CIO720", "3CIO720", "4PGC938", "4JZY524"]
If it were sorted by non-in-place LSD the output would be:
["1ICK750", "1ICK750", "1OHV845", "1OHV845", "1OHV845", "2IYE230", "2RLA629", "2RLA629", "3ATW723", "3CIO720", "3CIO720", "4JZY524", "4PGC938"]

Counterexample for both LSD and MSD sorts:

The array ["CAA" (index 0), "ABB" (index 1), "ABB" (index 2)] after being sorted by either in-place LSD or in-place MSD becomes:
["ABB" (original index 2), "ABB" (original index 1), "CAA" (original index 0)]
If it were sorted by non-in-place LSD or MSD the output would be:
["ABB" (original index 1), "ABB" (original index 2), "CAA" (original index 0)]


Question: Timings. Compare the running times of MSD string sort and 3-way string quicksort, using various key generators. For fixed-length keys, include LSD string sort.

Answer:
5.1.22 - Timings

Running 10 experiments with 1000000 strings for random decimal keys (with fixed-length of 10 characters), random CA license plates, random fixed-length words (with fixed-length of 10 characters) and random variable length items (with given values 'A' and 'B'). The cutoff for small subarrays used in Most-Significant-Digit sort was equal to 15.

     Random string type |                 Sort type |  Average time spent
           Decimal keys     Least-Significant-Digit                  2.30
           Decimal keys      Most-Significant-Digit                  0.45
           Decimal keys      3-way string quicksort                  0.32
      CA license plates     Least-Significant-Digit                  1.48
      CA license plates      Most-Significant-Digit                  0.41
      CA license plates      3-way string quicksort                  0.33
     Fixed length words     Least-Significant-Digit                  2.52
     Fixed length words      Most-Significant-Digit                  0.28
     Fixed length words      3-way string quicksort                  0.35
  Variable length items      Most-Significant-Digit                  1.80
  Variable length items      3-way string quicksort                  0.55

The experiment results show that for all random string types, LSD sort had the worst results.
For random decimal keys, random CA license plates and random variable length items 3-way string quicksort had the best results.
For random fixed-length words, MSD had the best running time.
Having to always scan all characters in all keys may explain why LSD sort had the slowest running times when sorting all random string types. 3-way string quicksort may have had good results because it does not create a high number of empty subarrays, as MSD sort does, and because it can handle well keys with long common prefixes (which are likely to happen in random decimal keys, random CA license plates and random variable length items).
Random fixed-length words are less likely to have long common prefixes (because all their characters are in the range [40, 125]), which may explain why MSD sort had better results than both LSD sort and 3-way string quicksort during their sort.


Question: Array accesses. Compare the number of array accesses used by MSD string sort and 3-way string sort, using various key generators. For fixed-length keys, include LSD string sort.

Answer:
5.1.23 - Array accesses

Running 10 experiments with 1000000 strings for random decimal keys (with fixed-length of 10 characters), random CA license plates, random fixed-length words (with fixed-length of 10 characters) and random variable length items (with given values 'A' and 'B'). The cutoff for small subarrays used in Most-Significant-Digit sort was equal to 15.

     Random string type |                 Sort type |  Number of array accesses
           Decimal keys     Least-Significant-Digit                    40000000
           Decimal keys      Most-Significant-Digit                    35443947
           Decimal keys      3-way string quicksort                    78124405
      CA license plates     Least-Significant-Digit                    28000000
      CA license plates      Most-Significant-Digit                    25703588
      CA license plates      3-way string quicksort                    82889002
     Fixed length words     Least-Significant-Digit                    40000000
     Fixed length words      Most-Significant-Digit                    14841196
     Fixed length words      3-way string quicksort                    95310075
  Variable length items      Most-Significant-Digit                    72121457
  Variable length items      3-way string quicksort                    97523400

The experiment results show that for all random string types, 3-way string quicksort accessed the array more times than LSD and MSD sort; LSD sort accessed the array more times than MSD sort; and MSD sort had the lowest number of array accesses.
A possible explanation for these results is the fact that 3-way string quicksort accesses the array 4 times for each exchange operation, which leads to more array accesses than both LSD and MSD sorts, that do not make inplace exchanges.
LSD sort will always access the array 4 * N * W times, where N is the number of strings and W is the length of the strings (which is equivalent to 4 array accesses for each character in the keys) and MSD sort will only access the array while the strings have common prefixes, which explains why MSD sort has the lowest number of array accesses of all sort types.


Question: Rightmost character accessed. Compare the position of the rightmost character accessed for MSD string sort and 3-way string quicksort, using various key generators.

Answer:
5.1.24 - Rightmost character accessed

Running 1 experiment with 1000000 strings for random decimal keys (with fixed-length of 10 characters), random CA license plates, random fixed-length words (with fixed-length of 10 characters) and random variable length items (with given values 'A' and 'B'). The cutoff for small subarrays used in Most-Significant-Digit sort was equal to 15.

     Random string type |                 Sort type |  Rightmost character accessed
           Decimal keys      Most-Significant-Digit                               9
           Decimal keys      3-way string quicksort                               9
      CA license plates      Most-Significant-Digit                               6
      CA license plates      3-way string quicksort                               6
     Fixed length words      Most-Significant-Digit                               5
     Fixed length words      3-way string quicksort                               5
  Variable length items      Most-Significant-Digit                              20
  Variable length items      3-way string quicksort                              20

In all experiments the rightmost character position accessed in MSD sort and in 3-way string quicksort was the same, which shows that both algorithms scan the same characters.


Question:
Draw the TST that results when the keys
 no is th ti fo al go pe to co to th ai of th pa
are inserted in that order into an initially empty TST.

Answer:
5.1.2

Trace for LSD string sort (same model as used in the book):

input   d=1   d=0   output
no      pa    ai    ai
is      pe    al    al
th      of    co    co
ti      th    fo    fo
fo      th    go    go
al      th    is    is
go      ti    no    no
pe      ai    of    of
to      al    pa    pa
co      no    pe    pe
to      fo    th    th
th      go    th    th
ai      to    th    th
of      co    ti    ti
th      to    to    to
pa      is    to    to


Question:
Draw the R-way trie that results when the keys
 now is the time for all good people to come to the aid of
are inserted in that order into an initially empty trie (do not draw null links).

Answer:
5.1.3

According to ASC II values, indices of chars 'a' through 'z' start at 97.
But this trace indices follow the book convention of 'a' starting at 0.

Trace for MSD string sort (same model as used in the book):

Top level of sort(array, 0, 15, 0):

input
0  no
1  is
2  th
3  ti
4  fo
5  al
6  go
7  pe
8  to
9  co
10 to
11 th
12 ai
13 of
14 th
15 pa

d=0
Count frequencies
0    0
1    0
2  a 2
3  b 0
4  c 1
5  d 0
6  e 0
7  f 1
8  g 1
9  h 0
10 i 1
11 j 0
12 k 0
13 l 0
14 m 0
15 n 1
16 o 1
17 p 2
18 q 0
19 r 0
20 s 0
21 t 6
22 u 0
23 v 0
24 x 0
25 w 0
26 y 0
27 z 0

Transform counts to indices
0    0
1    0
2  a 2
3  b 2
4  c 3
5  d 3
6  e 3
7  f 4
8  g 5
9  h 5
10 i 6
11 j 6
12 k 6
13 l 6
14 m 6
15 n 7
16 o 8
17 p 10
18 q 10
19 r 10
20 s 10
21 t 16
22 u 16
23 v 16
24 x 16
25 w 16
26 y 16
27 z 16

Distribute and copy back
0  al
1  ai
2  co
3  fo
4  go
5  is
6  no
7  of
8  pe
9  pa
10 th
11 ti
12 to
13 to
14 th
15 th

Indices at completion of distribute phase
0    0
1    2
2  a 2
3  b 3
4  c 3
5  d 3
6  e 4
7  f 5
8  g 5
9  h 6
10 i 6
11 j 6
12 k 6
13 l 6
14 m 7
15 n 8
16 o 10
17 p 10
18 q 10
19 r 10
20 s 16
21 t 16
22 u 16
23 v 16
24 x 16
25 w 16
26 y 16
27 z 16

Recursively sort subarrays
sort(a, 0, 1, 1);
sort(a, 2, 1, 1);
sort(a, 2, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 3, 1);
sort(a, 4, 4, 1);
sort(a, 5, 4, 1);
sort(a, 5, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 6, 1);
sort(a, 7, 7, 1);
sort(a, 8, 9, 1);
sort(a, 10, 9, 1);
sort(a, 10, 9, 1);
sort(a, 10, 9, 1);
sort(a, 10, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);

Sorted result
0  ai
1  al
2  co
3  fo
4  go
5  is
6  no
7  of
8  pa
9  pe
10 th
11 th
12 th
13 ti
14 to
15 to

Trace of recursive calls for MSD string sort (no cutoff for small subarrays, subarrays of size 0 and 1 omitted)

input   __      __                      output
no      al      ai      ai      ai      ai
is      ai      al      al      al      al
th      co      --      co      co      co
ti      fo      co      fo      fo      fo
fo      go      fo      go      go      go
al      is      go      is      is      is
go      no      is      no      no      no
pe      of      no      of      of      of
to      pe      of      --      pa      pa
co      pa      pe      pa      pe      pe
to      th      pa      pe      --      th
th      ti      th      --      th      th
ai      to      ti      th      th      th
of      to      to      ti      th      ti
th      th      to      to      ti      to
pa      th      th      to      to      to
        --      th      th      to
                        th      --


Question:
Draw the TST that results when the keys
 now is the time for all good people to come to the aid of
are inserted in that order into an initially empty TST.

Answer:
5.1.4

Trace for 3-way string quicksort (same model as used in the book):

                   --      --      --
0  no      is      ai      ai      ai      ai
1  is      ai      co      al      --      al
2  th      co      fo      --      al      co
3  ti      fo      al      fo      --      fo
4  fo      al      go      go      co      go
5  al      go      --      co      --      is
6  go      --      is      --      fo      no
7  pe      no      --      is      --      of
8  to      --      no      --      go      pa
9  co      to      --      no      --      pe
10 to      pe      pe      --      is      th
11 th      to      of      of      --      th
12 ai      th      pa      --      no      th
13 of      ti      --      pe      --      ti
14 th      of      th      pa      of      to
15 pa      th      ti      --      --      to
           pa      to      th      pa
           th      th      th      --
                   to      th      pe
                   th      --      --
                   --      to      th
                           to      th
                           ti      th
                                   --
                                   ti
                                   --
                                   to
                                   to
                                   --


Question: Develop nonrecursive versions of TrieST and TST.

Answer:
5.1.5

According to ASC II values, indices of chars 'a' through 'z' start at 97.
But this trace indices follow the book convention of 'a' starting at 0.

Trace for MSD string sort (same model as used in the book):

Top level of sort(array, 0, 13, 0):

input
0  now
1  is
2  the
3  time
4  for
5  all
6  good
7  people
8  to
9  come
10 to
11 the
12 aid
13 of

d=0
Count frequencies
0    0
1    0
2  a 2
3  b 0
4  c 1
5  d 0
6  e 0
7  f 1
8  g 1
9  h 0
10 i 1
11 j 0
12 k 0
13 l 0
14 m 0
15 n 1
16 o 1
17 p 1
18 q 0
19 r 0
20 s 0
21 t 5
22 u 0
23 v 0
24 x 0
25 w 0
26 y 0
27 z 0

Transform counts to indices
0    0
1    0
2  a 2
3  b 2
4  c 3
5  d 3
6  e 3
7  f 4
8  g 5
9  h 5
10 i 6
11 j 6
12 k 6
13 l 6
14 m 6
15 n 7
16 o 8
17 p 9
18 q 9
19 r 9
20 s 9
21 t 14
22 u 14
23 v 14
24 x 14
25 w 14
26 y 14
27 z 14

Distribute and copy back
0  all
1  aid
2  come
3  for
4  good
5  is
6  now
7  of
8  people
9  the
10 time
11 to
12 to
13 the

Indices at completion of distribute phase
0    0
1    2
2  a 2
3  b 3
4  c 3
5  d 3
6  e 4
7  f 5
8  g 5
9  h 6
10 i 6
11 j 6
12 k 6
13 l 6
14 m 7
15 n 8
16 o 9
17 p 9
18 q 9
19 r 9
20 s 14
21 t 14
22 u 14
23 v 14
24 x 14
25 w 14
26 y 14
27 z 14

Recursively sort subarrays
sort(a, 0, 1, 1);
sort(a, 2, 1, 1);
sort(a, 2, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 3, 1);
sort(a, 4, 4, 1);
sort(a, 5, 4, 1);
sort(a, 5, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 6, 1);
sort(a, 7, 7, 1);
sort(a, 8, 8, 1);
sort(a, 9, 8, 1);
sort(a, 9, 8, 1);
sort(a, 9, 8, 1);
sort(a, 9, 13, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);

Sorted result
0  aid
1  all
2  come
3  for
4  good
5  is
6  now
7  of
8  people
9  the
10 the
11 time
12 to
13 to

Trace of recursive calls for MSD string sort (no cutoff for small subarrays, subarrays of size 0 and 1 omitted)

input       ____        ___                                 output
now         all         aid         aid         aid         aid
is          aid         all         all         all         all
the         come        ---         come        come        come
time        for         come        for         for         for
for         good        for         good        good        good
all         is          good        is          is          is
good        now         is          now         now         now
people      of          now         of          of          of
to          people      of          people      people      people
come        the         people      ---         ---         the
to          time        the         the         the         the
the         to          time        the         the         time
aid         to          to          time        ---         to
of          the         to          to          time        to
            ----        the         to          to
                                    ---         to


Question:
Implement the following API, for a StringSET data type:
public class StringSET
StringSET() create a string set
void add(String key) put key into the set
void delete(String key) remove key from the set
boolean contains(String key) is key in the set?
boolean isEmpty() is the set empty?
int size() number of keys in the set
int toString() string representation of the set

Answer:
5.1.6

Trace for 3-way string quicksort (same model as used in the book):

                    ----     ---      ---      ---
0  now      is      aid      aid      aid      aid      aid
1  is       aid     come     all      ---      ---      all
2  the      come    for      ---      all      all      come
3  time     for     all      for      ---      ---      for
4  for      all     good     good     come     come     good
5  all      good    ----     come     ----     ----     is
6  good     ---     is       ----     for      for      now
7  people   now     ---      is       ----     ----     of
8  to       ---     now      ---      good     good     people
9  come     to      ---      now      ----     ----     the
10 to       people  people   ---      is       is       the
11 the      to      of       of       ---      ---      time
12 aid      the     ---      ------   now      now      to
13 of       time    to       people   ---      ---      to
            of      the      ------   of       of
            the     time     the      ------   ------
                    to       time     people   people
                    the      the      ------   ------
                    ---      ----     the      the
                             to       the      the
                             to       ----     ----
                             ----     time     time
                                      ----     ----
                                      to       to
                                      to       to
                                      --       --


Question: Ordered operations for tries. Implement the floor(), ceil(), rank(), and select() (from our standard ordered ST API from Chapter 3) for TrieST.

Answer:
5.1.8

Both MSD string sort and 3-way string quicksort examine all characters in the N keys.
That number is equal to 1 + 2 + ... + N = (N^2 + N) / 2 characters.
MSD string sort, however, generates (R - 1) * N empty subarrays (an empty subarray for all digits in R other than 'a', in every pass) while 3-way string quicksort generates 2N empty subarrays (empty subarrays for digits smaller than 'a' and for digits higher than 'a', or empty subarrays for digits smaller than '-1' and for digits equal to '-1', in every pass).

MSD string sort trace (no cutoff for small subarrays, subarrays of size 0 and 1 omitted):

input   ----
a       a       a       a       a       a       a
aa      aa      ----    aa      aa      aa      aa
aaa     aaa     aa      ----    aaa     aaa     aaa
aaaa    aaaa    aaa     aaa     ----    aaaa    aaaa
...     ...     aaaa    aaaa    aaaa    ----    ...
        ----    ...     ...     ...     ...
                ----    ----    ----    ----

3-way string quicksort trace:

input   ----
a       a       a       a       a       a       a
aa      aa      ----    aa      aa      aa      aa
aaa     aaa     aa      ----    aaa     aaa     aaa
aaaa    aaaa    aaa     aaa     ----    aaaa    aaaa
...     ...     aaaa    aaaa    aaaa    ----    ...
        ----    ...     ...     ...     ...
                ----    ----    ----    ----


Question: Size. Implement very eager size() (that keeps in each node the number of keys in its subtree) for TrieST and TST.

Answer:
5.1.10

The total number of characters examined by 3-way string quicksort when sorting N fixed-length strings (all of length W) in the worst case is O(N * W * R).

This can be seen with a recurrence relation T(W).

The base case T(1) is when all the strings have length 1.
An example with R = 3 is { "a", "b", "c" }.
In the worst case they are in reverse order.
For example: { "c", "b", "a" }.
In this case we only remove one string from the list in each pass.
If we consider N = R^W (in this case, W = 1), the number of comparisons is equal to:
Characters examined = Sum[i=0..R] i
Characters examined = R * (R + 1) / 2

To build the worst case for strings of length 2 (T(2)), we take each string from T(1) and append it to the end of each character in R.
So for single character strings "a", "b", "c", with R = 3, the two character list is: "aa", "ab", "ac", "ba", "bb", "bc", "ca", "cb", "cc".
The list can then be split into R groups: one for each character in R that is a prefix to every string of length W - 1.
During the partitioning phase all strings that start with "a" will be in the same partition and the algorithm will do the same process as in T(1) because removing the first character 'a' will lead to the same 1-length strings { "c", "b", "a" } as before.
The same thing happens for strings starting with "b" and "c".
So, for R = 3, the algorithm will check 3 * R + 2 * R + R characters in the first position of the strings (which is 3 + 2 + 1 characters times R groups).
Then it will check the second characters in the strings in each of the R groups.

For T(W), where W > 2, the list will then again be split into R groups: one for each character in R that is a prefix to every string of length W - 2.
Quicksort will then remove R strings from the list in each partition.
It will then check R * T(W - 1) more characters for each of those groups.
This gives the recurrence T(W) = (R^(W - 1) * Sum[i=0..R] R - i) + R * T(W - 1), which simplifies to:
T(W) = R^(W + 1) + R^W  + R * T(W - 1)
      -----------------
              2
Solving the recurrence gives us:
T(W) = W * (R^W) * (R + 1)
      ---------------------
                2
Substituting N = R^W:
T(W) = W * N * (R + 1)
      -----------------
              2
Which is O(N * W * R).

Thanks to dragon-dreamer (https://github.com/dragon-dreamer) for finding a more accurate worst case.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/153
Thanks to GenevaS (https://github.com/GenevaS) for finding a more accurate worst case.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/245


Question: Hybrid TST with R2-way branching at the root. Add code to TST to do multiway branching at the first two levels, as described in the text.

Answer:
5.1.13 - Hybrid sort

Idea: using standard MSD string sort for large arrays, in order to get the advantage of multiway partitioning, and 3-way string quicksort for smaller arrays, in order to avoid the negative effects of large numbers of empty bins.

This idea will work well for random strings because, in general, the higher the number of keys to be sorted, the higher the number of non-empty subarrays generated on each pass of MSD string sort. Such scenario would work well due to the advantage of having multiway partitioning.
However, MSD string sort will still generate a large number of empty subarrays if there is a large number of equal keys (or a large number of keys with long common prefixes).

3-way string quicksort will avoid the negative effects of large numbers of empty bins not only for smaller arrays, but also for large arrays, while also having the benefit of using less space than MSD string sort since it does not require space for frequency counts or for an auxiliary array. On the other hand, it envolves more data movement than MSD string sort when the number of nonempty subarrays is large because it has to do a series of 3-way partitions to get the effect of the multiway partition. This would not be a problem in the hybrid sort if there were many equal keys in smaller arrays, since 3-way string quicksort would be the algorithm of choice in such situation.

Overall, hybrid sort would be a good choice for random strings. However, a version of hybrid sort that chooses between MSD string sort and 3-way string quicksort based on the percentage of equal keys (choosing MSD string sort if there is a low percentage of equal keys and choosing 3-way string quicksort if there is a high number of equal keys) would be more effective than a version that makes the choice based on the number of keys.


Question: Spell checking. Write a TST client SpellChecker that takes as command-line argument the name of a file containing a dictionary of words in the English language, and then reads a string from standard input and prints out any word that is not in the dictionary. Use a string set.

Answer:
5.1.17 - In-place key-indexed counting

LSD and MSD sorts that use only a constant amount of extra space are not stable.

Counterexample for LSD sort:

The array ["4PGC938", "2IYE230", "3CIO720", "1ICK750", "1OHV845", "4JZY524", "1ICK750", "3CIO720", "1OHV845", "1OHV845", "2RLA629", "2RLA629", "3ATW723"] after being sorted by in-place LSD becomes:
["1OHV845", "1OHV845", "1OHV845", "1ICK750", "1ICK750", "2RLA629", "2IYE230", "2RLA629", "3ATW723", "3CIO720", "3CIO720", "4PGC938", "4JZY524"]
If it were sorted by non-in-place LSD the output would be:
["1ICK750", "1ICK750", "1OHV845", "1OHV845", "1OHV845", "2IYE230", "2RLA629", "2RLA629", "3ATW723", "3CIO720", "3CIO720", "4JZY524", "4PGC938"]

Counterexample for both LSD and MSD sorts:

The array ["CAA" (index 0), "ABB" (index 1), "ABB" (index 2)] after being sorted by either in-place LSD or in-place MSD becomes:
["ABB" (original index 2), "ABB" (original index 1), "CAA" (original index 0)]
If it were sorted by non-in-place LSD or MSD the output would be:
["ABB" (original index 1), "ABB" (original index 2), "CAA" (original index 0)]


Question: Typing monkeys. Suppose that a typing monkey creates random words by appending each of 26 possible letter with probability p to the current word and finishes the word with probability 1 - 26p. Write a program to estimate the frequency distribution of the lengths of words produced. If "abc" is produced more than once, count it only once.

Answer:
5.1.22 - Timings

Running 10 experiments with 1000000 strings for random decimal keys (with fixed-length of 10 characters), random CA license plates, random fixed-length words (with fixed-length of 10 characters) and random variable length items (with given values 'A' and 'B'). The cutoff for small subarrays used in Most-Significant-Digit sort was equal to 15.

     Random string type |                 Sort type |  Average time spent
           Decimal keys     Least-Significant-Digit                  2.30
           Decimal keys      Most-Significant-Digit                  0.45
           Decimal keys      3-way string quicksort                  0.32
      CA license plates     Least-Significant-Digit                  1.48
      CA license plates      Most-Significant-Digit                  0.41
      CA license plates      3-way string quicksort                  0.33
     Fixed length words     Least-Significant-Digit                  2.52
     Fixed length words      Most-Significant-Digit                  0.28
     Fixed length words      3-way string quicksort                  0.35
  Variable length items      Most-Significant-Digit                  1.80
  Variable length items      3-way string quicksort                  0.55

The experiment results show that for all random string types, LSD sort had the worst results.
For random decimal keys, random CA license plates and random variable length items 3-way string quicksort had the best results.
For random fixed-length words, MSD had the best running time.
Having to always scan all characters in all keys may explain why LSD sort had the slowest running times when sorting all random string types. 3-way string quicksort may have had good results because it does not create a high number of empty subarrays, as MSD sort does, and because it can handle well keys with long common prefixes (which are likely to happen in random decimal keys, random CA license plates and random variable length items).
Random fixed-length words are less likely to have long common prefixes (because all their characters are in the range [40, 125]), which may explain why MSD sort had better results than both LSD sort and 3-way string quicksort during their sort.


Question: Duplicates (revisited again). Redo Exercise 3.5.30 using StringSET (see Exercise 5.2.6) instead of HashSET. Compare the running times of the two approaches. Then use Dedup to run the experiments for N = 10^7, 10^8, and10^9, repeat the experiments for random long values and discuss the results.

Answer:
5.1.23 - Array accesses

Running 10 experiments with 1000000 strings for random decimal keys (with fixed-length of 10 characters), random CA license plates, random fixed-length words (with fixed-length of 10 characters) and random variable length items (with given values 'A' and 'B'). The cutoff for small subarrays used in Most-Significant-Digit sort was equal to 15.

     Random string type |                 Sort type |  Number of array accesses
           Decimal keys     Least-Significant-Digit                    40000000
           Decimal keys      Most-Significant-Digit                    35443947
           Decimal keys      3-way string quicksort                    78124405
      CA license plates     Least-Significant-Digit                    28000000
      CA license plates      Most-Significant-Digit                    25703588
      CA license plates      3-way string quicksort                    82889002
     Fixed length words     Least-Significant-Digit                    40000000
     Fixed length words      Most-Significant-Digit                    14841196
     Fixed length words      3-way string quicksort                    95310075
  Variable length items      Most-Significant-Digit                    72121457
  Variable length items      3-way string quicksort                    97523400

The experiment results show that for all random string types, 3-way string quicksort accessed the array more times than LSD and MSD sort; LSD sort accessed the array more times than MSD sort; and MSD sort had the lowest number of array accesses.
A possible explanation for these results is the fact that 3-way string quicksort accesses the array 4 times for each exchange operation, which leads to more array accesses than both LSD and MSD sorts, that do not make inplace exchanges.
LSD sort will always access the array 4 * N * W times, where N is the number of strings and W is the length of the strings (which is equivalent to 4 array accesses for each character in the keys) and MSD sort will only access the array while the strings have common prefixes, which explains why MSD sort has the lowest number of array accesses of all sort types.


Question: Spell checker. Redo Exercise 3.5.31, which uses the file dictionary.txt from the booksite and the BlackFilter client on page 491 to print all misspelled words in a text file. Compare the performance of TrieST and TST for the file war.txt with this client and discuss the results.

Answer:
5.1.24 - Rightmost character accessed

Running 1 experiment with 1000000 strings for random decimal keys (with fixed-length of 10 characters), random CA license plates, random fixed-length words (with fixed-length of 10 characters) and random variable length items (with given values 'A' and 'B'). The cutoff for small subarrays used in Most-Significant-Digit sort was equal to 15.

     Random string type |                 Sort type |  Rightmost character accessed
           Decimal keys      Most-Significant-Digit                               9
           Decimal keys      3-way string quicksort                               9
      CA license plates      Most-Significant-Digit                               6
      CA license plates      3-way string quicksort                               6
     Fixed length words      Most-Significant-Digit                               5
     Fixed length words      3-way string quicksort                               5
  Variable length items      Most-Significant-Digit                              20
  Variable length items      3-way string quicksort                              20

In all experiments the rightmost character position accessed in MSD sort and in 3-way string quicksort was the same, which shows that both algorithms scan the same characters.


Question: Give the dfa[][] array for the Knuth-Morris-Pratt algorithm for the pattern A A A A A A A A A, and draw the DFA, in the style of the figures in the text.

Answer:
5.1.2

Trace for LSD string sort (same model as used in the book):

input   d=1   d=0   output
no      pa    ai    ai
is      pe    al    al
th      of    co    co
ti      th    fo    fo
fo      th    go    go
al      th    is    is
go      ti    no    no
pe      ai    of    of
to      al    pa    pa
co      no    pe    pe
to      fo    th    th
th      go    th    th
ai      to    th    th
of      co    ti    ti
th      to    to    to
pa      is    to    to


Question: Give the dfa[][] array for the Knuth-Morris-Pratt algorithm for the pattern A B R A C A D A B R A, and draw the DFA, in the style of the figures in the text.

Answer:
5.1.3

According to ASC II values, indices of chars 'a' through 'z' start at 97.
But this trace indices follow the book convention of 'a' starting at 0.

Trace for MSD string sort (same model as used in the book):

Top level of sort(array, 0, 15, 0):

input
0  no
1  is
2  th
3  ti
4  fo
5  al
6  go
7  pe
8  to
9  co
10 to
11 th
12 ai
13 of
14 th
15 pa

d=0
Count frequencies
0    0
1    0
2  a 2
3  b 0
4  c 1
5  d 0
6  e 0
7  f 1
8  g 1
9  h 0
10 i 1
11 j 0
12 k 0
13 l 0
14 m 0
15 n 1
16 o 1
17 p 2
18 q 0
19 r 0
20 s 0
21 t 6
22 u 0
23 v 0
24 x 0
25 w 0
26 y 0
27 z 0

Transform counts to indices
0    0
1    0
2  a 2
3  b 2
4  c 3
5  d 3
6  e 3
7  f 4
8  g 5
9  h 5
10 i 6
11 j 6
12 k 6
13 l 6
14 m 6
15 n 7
16 o 8
17 p 10
18 q 10
19 r 10
20 s 10
21 t 16
22 u 16
23 v 16
24 x 16
25 w 16
26 y 16
27 z 16

Distribute and copy back
0  al
1  ai
2  co
3  fo
4  go
5  is
6  no
7  of
8  pe
9  pa
10 th
11 ti
12 to
13 to
14 th
15 th

Indices at completion of distribute phase
0    0
1    2
2  a 2
3  b 3
4  c 3
5  d 3
6  e 4
7  f 5
8  g 5
9  h 6
10 i 6
11 j 6
12 k 6
13 l 6
14 m 7
15 n 8
16 o 10
17 p 10
18 q 10
19 r 10
20 s 16
21 t 16
22 u 16
23 v 16
24 x 16
25 w 16
26 y 16
27 z 16

Recursively sort subarrays
sort(a, 0, 1, 1);
sort(a, 2, 1, 1);
sort(a, 2, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 3, 1);
sort(a, 4, 4, 1);
sort(a, 5, 4, 1);
sort(a, 5, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 6, 1);
sort(a, 7, 7, 1);
sort(a, 8, 9, 1);
sort(a, 10, 9, 1);
sort(a, 10, 9, 1);
sort(a, 10, 9, 1);
sort(a, 10, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);

Sorted result
0  ai
1  al
2  co
3  fo
4  go
5  is
6  no
7  of
8  pa
9  pe
10 th
11 th
12 th
13 ti
14 to
15 to

Trace of recursive calls for MSD string sort (no cutoff for small subarrays, subarrays of size 0 and 1 omitted)

input   __      __                      output
no      al      ai      ai      ai      ai
is      ai      al      al      al      al
th      co      --      co      co      co
ti      fo      co      fo      fo      fo
fo      go      fo      go      go      go
al      is      go      is      is      is
go      no      is      no      no      no
pe      of      no      of      of      of
to      pe      of      --      pa      pa
co      pa      pe      pa      pe      pe
to      th      pa      pe      --      th
th      ti      th      --      th      th
ai      to      ti      th      th      th
of      to      to      ti      th      ti
th      th      to      to      ti      to
pa      th      th      to      to      to
        --      th      th      to
                        th      --


Question: Write an efficient method that takes a string txt and an integer M as arguments and returns the position of the first occurrence of M consecutive blanks in the string, txt.length if there is no such occurrence. Estimate the number of character compares used by your method, on typical text and in the worst case.

Answer:
5.1.4

Trace for 3-way string quicksort (same model as used in the book):

                   --      --      --
0  no      is      ai      ai      ai      ai
1  is      ai      co      al      --      al
2  th      co      fo      --      al      co
3  ti      fo      al      fo      --      fo
4  fo      al      go      go      co      go
5  al      go      --      co      --      is
6  go      --      is      --      fo      no
7  pe      no      --      is      --      of
8  to      --      no      --      go      pa
9  co      to      --      no      --      pe
10 to      pe      pe      --      is      th
11 th      to      of      of      --      th
12 ai      th      pa      --      no      th
13 of      ti      --      pe      --      ti
14 th      of      th      pa      of      to
15 pa      th      ti      --      --      to
           pa      to      th      pa
           th      th      th      --
                   to      th      pe
                   th      --      --
                   --      to      th
                           to      th
                           ti      th
                                   --
                                   ti
                                   --
                                   to
                                   to
                                   --


Question: Develop a brute-force substring search implementation BruteForceRL that processes the pattern from right to left (a simplified version of ALGORITHM 5.7).

Answer:
5.1.5

According to ASC II values, indices of chars 'a' through 'z' start at 97.
But this trace indices follow the book convention of 'a' starting at 0.

Trace for MSD string sort (same model as used in the book):

Top level of sort(array, 0, 13, 0):

input
0  now
1  is
2  the
3  time
4  for
5  all
6  good
7  people
8  to
9  come
10 to
11 the
12 aid
13 of

d=0
Count frequencies
0    0
1    0
2  a 2
3  b 0
4  c 1
5  d 0
6  e 0
7  f 1
8  g 1
9  h 0
10 i 1
11 j 0
12 k 0
13 l 0
14 m 0
15 n 1
16 o 1
17 p 1
18 q 0
19 r 0
20 s 0
21 t 5
22 u 0
23 v 0
24 x 0
25 w 0
26 y 0
27 z 0

Transform counts to indices
0    0
1    0
2  a 2
3  b 2
4  c 3
5  d 3
6  e 3
7  f 4
8  g 5
9  h 5
10 i 6
11 j 6
12 k 6
13 l 6
14 m 6
15 n 7
16 o 8
17 p 9
18 q 9
19 r 9
20 s 9
21 t 14
22 u 14
23 v 14
24 x 14
25 w 14
26 y 14
27 z 14

Distribute and copy back
0  all
1  aid
2  come
3  for
4  good
5  is
6  now
7  of
8  people
9  the
10 time
11 to
12 to
13 the

Indices at completion of distribute phase
0    0
1    2
2  a 2
3  b 3
4  c 3
5  d 3
6  e 4
7  f 5
8  g 5
9  h 6
10 i 6
11 j 6
12 k 6
13 l 6
14 m 7
15 n 8
16 o 9
17 p 9
18 q 9
19 r 9
20 s 14
21 t 14
22 u 14
23 v 14
24 x 14
25 w 14
26 y 14
27 z 14

Recursively sort subarrays
sort(a, 0, 1, 1);
sort(a, 2, 1, 1);
sort(a, 2, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 3, 1);
sort(a, 4, 4, 1);
sort(a, 5, 4, 1);
sort(a, 5, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 6, 1);
sort(a, 7, 7, 1);
sort(a, 8, 8, 1);
sort(a, 9, 8, 1);
sort(a, 9, 8, 1);
sort(a, 9, 8, 1);
sort(a, 9, 13, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);

Sorted result
0  aid
1  all
2  come
3  for
4  good
5  is
6  now
7  of
8  people
9  the
10 the
11 time
12 to
13 to

Trace of recursive calls for MSD string sort (no cutoff for small subarrays, subarrays of size 0 and 1 omitted)

input       ____        ___                                 output
now         all         aid         aid         aid         aid
is          aid         all         all         all         all
the         come        ---         come        come        come
time        for         come        for         for         for
for         good        for         good        good        good
all         is          good        is          is          is
good        now         is          now         now         now
people      of          now         of          of          of
to          people      of          people      people      people
come        the         people      ---         ---         the
to          time        the         the         the         the
the         to          time        the         the         time
aid         to          to          time        ---         to
of          the         to          to          time        to
            ----        the         to          to
                                    ---         to


Question: Give the right[] array computed by the constructor in ALGORITHM 5.7 for the pattern A B R A C A D A B R A.

Answer:
5.1.6

Trace for 3-way string quicksort (same model as used in the book):

                    ----     ---      ---      ---
0  now      is      aid      aid      aid      aid      aid
1  is       aid     come     all      ---      ---      all
2  the      come    for      ---      all      all      come
3  time     for     all      for      ---      ---      for
4  for      all     good     good     come     come     good
5  all      good    ----     come     ----     ----     is
6  good     ---     is       ----     for      for      now
7  people   now     ---      is       ----     ----     of
8  to       ---     now      ---      good     good     people
9  come     to      ---      now      ----     ----     the
10 to       people  people   ---      is       is       the
11 the      to      of       of       ---      ---      time
12 aid      the     ---      ------   now      now      to
13 of       time    to       people   ---      ---      to
            of      the      ------   of       of
            the     time     the      ------   ------
                    to       time     people   people
                    the      the      ------   ------
                    ---      ----     the      the
                             to       the      the
                             to       ----     ----
                             ----     time     time
                                      ----     ----
                                      to       to
                                      to       to
                                      --       --


Question: Add to KMP a count() method to count occurrences and a searchAll() method to print all occurrences.

Answer:
5.1.8

Both MSD string sort and 3-way string quicksort examine all characters in the N keys.
That number is equal to 1 + 2 + ... + N = (N^2 + N) / 2 characters.
MSD string sort, however, generates (R - 1) * N empty subarrays (an empty subarray for all digits in R other than 'a', in every pass) while 3-way string quicksort generates 2N empty subarrays (empty subarrays for digits smaller than 'a' and for digits higher than 'a', or empty subarrays for digits smaller than '-1' and for digits equal to '-1', in every pass).

MSD string sort trace (no cutoff for small subarrays, subarrays of size 0 and 1 omitted):

input   ----
a       a       a       a       a       a       a
aa      aa      ----    aa      aa      aa      aa
aaa     aaa     aa      ----    aaa     aaa     aaa
aaaa    aaaa    aaa     aaa     ----    aaaa    aaaa
...     ...     aaaa    aaaa    aaaa    ----    ...
        ----    ...     ...     ...     ...
                ----    ----    ----    ----

3-way string quicksort trace:

input   ----
a       a       a       a       a       a       a
aa      aa      ----    aa      aa      aa      aa
aaa     aaa     aa      ----    aaa     aaa     aaa
aaaa    aaaa    aaa     aaa     ----    aaaa    aaaa
...     ...     aaaa    aaaa    aaaa    ----    ...
        ----    ...     ...     ...     ...
                ----    ----    ----    ----


Question: Add to RabinKarp a count() method to count occurrences and a searchAll() method to print all occurrences.

Answer:
5.1.10

The total number of characters examined by 3-way string quicksort when sorting N fixed-length strings (all of length W) in the worst case is O(N * W * R).

This can be seen with a recurrence relation T(W).

The base case T(1) is when all the strings have length 1.
An example with R = 3 is { "a", "b", "c" }.
In the worst case they are in reverse order.
For example: { "c", "b", "a" }.
In this case we only remove one string from the list in each pass.
If we consider N = R^W (in this case, W = 1), the number of comparisons is equal to:
Characters examined = Sum[i=0..R] i
Characters examined = R * (R + 1) / 2

To build the worst case for strings of length 2 (T(2)), we take each string from T(1) and append it to the end of each character in R.
So for single character strings "a", "b", "c", with R = 3, the two character list is: "aa", "ab", "ac", "ba", "bb", "bc", "ca", "cb", "cc".
The list can then be split into R groups: one for each character in R that is a prefix to every string of length W - 1.
During the partitioning phase all strings that start with "a" will be in the same partition and the algorithm will do the same process as in T(1) because removing the first character 'a' will lead to the same 1-length strings { "c", "b", "a" } as before.
The same thing happens for strings starting with "b" and "c".
So, for R = 3, the algorithm will check 3 * R + 2 * R + R characters in the first position of the strings (which is 3 + 2 + 1 characters times R groups).
Then it will check the second characters in the strings in each of the R groups.

For T(W), where W > 2, the list will then again be split into R groups: one for each character in R that is a prefix to every string of length W - 2.
Quicksort will then remove R strings from the list in each partition.
It will then check R * T(W - 1) more characters for each of those groups.
This gives the recurrence T(W) = (R^(W - 1) * Sum[i=0..R] R - i) + R * T(W - 1), which simplifies to:
T(W) = R^(W + 1) + R^W  + R * T(W - 1)
      -----------------
              2
Solving the recurrence gives us:
T(W) = W * (R^W) * (R + 1)
      ---------------------
                2
Substituting N = R^W:
T(W) = W * N * (R + 1)
      -----------------
              2
Which is O(N * W * R).

Thanks to dragon-dreamer (https://github.com/dragon-dreamer) for finding a more accurate worst case.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/153
Thanks to GenevaS (https://github.com/GenevaS) for finding a more accurate worst case.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/245


Question: In the Boyer-Moore implementation in ALGORITHM 5.7, show that you can set right[c] to the penultimate occurrence of c when c is the last character in the pattern.

Answer:
5.1.13 - Hybrid sort

Idea: using standard MSD string sort for large arrays, in order to get the advantage of multiway partitioning, and 3-way string quicksort for smaller arrays, in order to avoid the negative effects of large numbers of empty bins.

This idea will work well for random strings because, in general, the higher the number of keys to be sorted, the higher the number of non-empty subarrays generated on each pass of MSD string sort. Such scenario would work well due to the advantage of having multiway partitioning.
However, MSD string sort will still generate a large number of empty subarrays if there is a large number of equal keys (or a large number of keys with long common prefixes).

3-way string quicksort will avoid the negative effects of large numbers of empty bins not only for smaller arrays, but also for large arrays, while also having the benefit of using less space than MSD string sort since it does not require space for frequency counts or for an auxiliary array. On the other hand, it envolves more data movement than MSD string sort when the number of nonempty subarrays is large because it has to do a series of 3-way partitions to get the effect of the multiway partition. This would not be a problem in the hybrid sort if there were many equal keys in smaller arrays, since 3-way string quicksort would be the algorithm of choice in such situation.

Overall, hybrid sort would be a good choice for random strings. However, a version of hybrid sort that chooses between MSD string sort and 3-way string quicksort based on the percentage of equal keys (choosing MSD string sort if there is a low percentage of equal keys and choosing 3-way string quicksort if there is a high number of equal keys) would be more effective than a version that makes the choice based on the number of keys.


Question:
Draw the KMP DFA for the following pattern strings.
a. AAAAAAB
b. AACAAAB
c. ABABABAB
d. ABAABAAABAAAB
e. ABAABCABAABCB

Answer:
5.1.17 - In-place key-indexed counting

LSD and MSD sorts that use only a constant amount of extra space are not stable.

Counterexample for LSD sort:

The array ["4PGC938", "2IYE230", "3CIO720", "1ICK750", "1OHV845", "4JZY524", "1ICK750", "3CIO720", "1OHV845", "1OHV845", "2RLA629", "2RLA629", "3ATW723"] after being sorted by in-place LSD becomes:
["1OHV845", "1OHV845", "1OHV845", "1ICK750", "1ICK750", "2RLA629", "2IYE230", "2RLA629", "3ATW723", "3CIO720", "3CIO720", "4PGC938", "4JZY524"]
If it were sorted by non-in-place LSD the output would be:
["1ICK750", "1ICK750", "1OHV845", "1OHV845", "1OHV845", "2IYE230", "2RLA629", "2RLA629", "3ATW723", "3CIO720", "3CIO720", "4JZY524", "4PGC938"]

Counterexample for both LSD and MSD sorts:

The array ["CAA" (index 0), "ABB" (index 1), "ABB" (index 2)] after being sorted by either in-place LSD or in-place MSD becomes:
["ABB" (original index 2), "ABB" (original index 1), "CAA" (original index 0)]
If it were sorted by non-in-place LSD or MSD the output would be:
["ABB" (original index 1), "ABB" (original index 2), "CAA" (original index 0)]


Question: How would you modify the Rabin-Karp algorithm to search for an H-by-V pattern in an N-by-N text?

Answer:
5.1.22 - Timings

Running 10 experiments with 1000000 strings for random decimal keys (with fixed-length of 10 characters), random CA license plates, random fixed-length words (with fixed-length of 10 characters) and random variable length items (with given values 'A' and 'B'). The cutoff for small subarrays used in Most-Significant-Digit sort was equal to 15.

     Random string type |                 Sort type |  Average time spent
           Decimal keys     Least-Significant-Digit                  2.30
           Decimal keys      Most-Significant-Digit                  0.45
           Decimal keys      3-way string quicksort                  0.32
      CA license plates     Least-Significant-Digit                  1.48
      CA license plates      Most-Significant-Digit                  0.41
      CA license plates      3-way string quicksort                  0.33
     Fixed length words     Least-Significant-Digit                  2.52
     Fixed length words      Most-Significant-Digit                  0.28
     Fixed length words      3-way string quicksort                  0.35
  Variable length items      Most-Significant-Digit                  1.80
  Variable length items      3-way string quicksort                  0.55

The experiment results show that for all random string types, LSD sort had the worst results.
For random decimal keys, random CA license plates and random variable length items 3-way string quicksort had the best results.
For random fixed-length words, MSD had the best running time.
Having to always scan all characters in all keys may explain why LSD sort had the slowest running times when sorting all random string types. 3-way string quicksort may have had good results because it does not create a high number of empty subarrays, as MSD sort does, and because it can handle well keys with long common prefixes (which are likely to happen in random decimal keys, random CA license plates and random variable length items).
Random fixed-length words are less likely to have long common prefixes (because all their characters are in the range [40, 125]), which may explain why MSD sort had better results than both LSD sort and 3-way string quicksort during their sort.


Question: Write a program that reads characters one at a time and reports at each instant if the current string is a palindrome. Hint : Use the Rabin-Karp hashing idea.

Answer:
5.1.23 - Array accesses

Running 10 experiments with 1000000 strings for random decimal keys (with fixed-length of 10 characters), random CA license plates, random fixed-length words (with fixed-length of 10 characters) and random variable length items (with given values 'A' and 'B'). The cutoff for small subarrays used in Most-Significant-Digit sort was equal to 15.

     Random string type |                 Sort type |  Number of array accesses
           Decimal keys     Least-Significant-Digit                    40000000
           Decimal keys      Most-Significant-Digit                    35443947
           Decimal keys      3-way string quicksort                    78124405
      CA license plates     Least-Significant-Digit                    28000000
      CA license plates      Most-Significant-Digit                    25703588
      CA license plates      3-way string quicksort                    82889002
     Fixed length words     Least-Significant-Digit                    40000000
     Fixed length words      Most-Significant-Digit                    14841196
     Fixed length words      3-way string quicksort                    95310075
  Variable length items      Most-Significant-Digit                    72121457
  Variable length items      3-way string quicksort                    97523400

The experiment results show that for all random string types, 3-way string quicksort accessed the array more times than LSD and MSD sort; LSD sort accessed the array more times than MSD sort; and MSD sort had the lowest number of array accesses.
A possible explanation for these results is the fact that 3-way string quicksort accesses the array 4 times for each exchange operation, which leads to more array accesses than both LSD and MSD sorts, that do not make inplace exchanges.
LSD sort will always access the array 4 * N * W times, where N is the number of strings and W is the length of the strings (which is equivalent to 4 array accesses for each character in the keys) and MSD sort will only access the array while the strings have common prefixes, which explains why MSD sort has the lowest number of array accesses of all sort types.


Question: Find all occurrences. Add a method findAll() to each of the four substring search algorithms given in the text that returns an Iterable<Integer> that allows clients to iterate through all offsets of the pattern in the text.

Answer:
5.1.24 - Rightmost character accessed

Running 1 experiment with 1000000 strings for random decimal keys (with fixed-length of 10 characters), random CA license plates, random fixed-length words (with fixed-length of 10 characters) and random variable length items (with given values 'A' and 'B'). The cutoff for small subarrays used in Most-Significant-Digit sort was equal to 15.

     Random string type |                 Sort type |  Rightmost character accessed
           Decimal keys      Most-Significant-Digit                               9
           Decimal keys      3-way string quicksort                               9
      CA license plates      Most-Significant-Digit                               6
      CA license plates      3-way string quicksort                               6
     Fixed length words      Most-Significant-Digit                               5
     Fixed length words      3-way string quicksort                               5
  Variable length items      Most-Significant-Digit                              20
  Variable length items      3-way string quicksort                              20

In all experiments the rightmost character position accessed in MSD sort and in 3-way string quicksort was the same, which shows that both algorithms scan the same characters.


Question: Tandem repeat search. A tandem repeat of a base string b in a string s is a substring of s having at least two consecutive copies b (nonoverlapping). Develop and implement a linear-time algorithm that, given two strings b and s, returns the index of the beginning of the longest tandem repeat of b in s. For example, your program should return 3 when b is abcab and s is abcabcababcababcababcab.

Answer:
5.5.27 - Long repeats

Compressing 2 * 1000 random characters (for N = 1000), with 16000 bits (8 bits per character).

% java -cp algs4.jar:. RunLengthEncoding - < 5.5.27_random.txt | java -cp algs4.jar:. BinaryDump 0
53736 bits

Compression ratio: 53736 / 16000 = 335%

% java -cp algs4.jar:. edu.princeton.cs.algs4.Huffman - < 5.5.27_random.txt | java -cp algs4.jar:. BinaryDump 0
12624 bits

Compression ratio: 12624 / 16000 = 79%

% java -cp algs4.jar:. edu.princeton.cs.algs4.LZW - < 5.5.27_random.txt | java -cp algs4.jar:. BinaryDump 0
13416 bits

Compression ratio: 13416 / 16000 = 83%


Question:
Random patterns. How many character compares are needed to do a substring search for a random pattern of length 100 in a given text?
Answer: None. The method
public boolean search(char[] txt)
{ return false; }
is quite effective for this problem, since the chances of a random pattern of length 100 appearing in any text are so low that you may consider it to be 0.

Answer:
5.3.31 - Random patterns

None. The method
public boolean search(char[] text) {
    return false;
}
is quite effective for this problem, since the chances of a random pattern of length 100 appearing in any text are so low that you may consider it to be 0.


Question:
Timings. Write a program that times the four methods for the task of searchng for the substring
it is a far far better thing that i do than i have ever done
in the text of Tale of Two Cities (tale.txt). Discuss the extent to which your results validate the hypthotheses about performance that are stated in the text.

Answer:
5.3.39 - Timings

            Method | Time spent
        Bruteforce         0.01
Knuth-Morris-Pratt         0.01
       Boyer-Moore         0.00
        Rabin-Karp         0.04

The results validate the hypotheses about performance stated in the text:
Both brute force and Knuth-Morris-Pratt methods took 0.01 seconds to do the search, which is aligned with their expected performance of 1.1N operations when searching in typical texts.
Boyer-Moore took 0.00 seconds to do the search, which suggests that it did a sublinear number of operations. This is aligned with its expected performance of N / M operations when searching in typical texts.
Rabin-Karp took 0.04 seconds to do the search, a longer time than all other three methods. However, this is expected according to the hypothesis in the text that it performs 7N operations when searching in typical texts.


Question:
Give a brief English description of each of the following REs:
a. .*
b. A.*A | A
c. .*ABBABBA.*
d. .* A.*A.*A.*A.*

Answer:
5.1.2

Trace for LSD string sort (same model as used in the book):

input   d=1   d=0   output
no      pa    ai    ai
is      pe    al    al
th      of    co    co
ti      th    fo    fo
fo      th    go    go
al      th    is    is
go      ti    no    no
pe      ai    of    of
to      al    pa    pa
co      no    pe    pe
to      fo    th    th
th      go    th    th
ai      to    th    th
of      co    ti    ti
th      to    to    to
pa      is    to    to


Question: What is the maximum number of different strings that can be described by a regular expression with M or operators and no closure operators (parentheses and concatenation are allowed)?

Answer:
5.1.3

According to ASC II values, indices of chars 'a' through 'z' start at 97.
But this trace indices follow the book convention of 'a' starting at 0.

Trace for MSD string sort (same model as used in the book):

Top level of sort(array, 0, 15, 0):

input
0  no
1  is
2  th
3  ti
4  fo
5  al
6  go
7  pe
8  to
9  co
10 to
11 th
12 ai
13 of
14 th
15 pa

d=0
Count frequencies
0    0
1    0
2  a 2
3  b 0
4  c 1
5  d 0
6  e 0
7  f 1
8  g 1
9  h 0
10 i 1
11 j 0
12 k 0
13 l 0
14 m 0
15 n 1
16 o 1
17 p 2
18 q 0
19 r 0
20 s 0
21 t 6
22 u 0
23 v 0
24 x 0
25 w 0
26 y 0
27 z 0

Transform counts to indices
0    0
1    0
2  a 2
3  b 2
4  c 3
5  d 3
6  e 3
7  f 4
8  g 5
9  h 5
10 i 6
11 j 6
12 k 6
13 l 6
14 m 6
15 n 7
16 o 8
17 p 10
18 q 10
19 r 10
20 s 10
21 t 16
22 u 16
23 v 16
24 x 16
25 w 16
26 y 16
27 z 16

Distribute and copy back
0  al
1  ai
2  co
3  fo
4  go
5  is
6  no
7  of
8  pe
9  pa
10 th
11 ti
12 to
13 to
14 th
15 th

Indices at completion of distribute phase
0    0
1    2
2  a 2
3  b 3
4  c 3
5  d 3
6  e 4
7  f 5
8  g 5
9  h 6
10 i 6
11 j 6
12 k 6
13 l 6
14 m 7
15 n 8
16 o 10
17 p 10
18 q 10
19 r 10
20 s 16
21 t 16
22 u 16
23 v 16
24 x 16
25 w 16
26 y 16
27 z 16

Recursively sort subarrays
sort(a, 0, 1, 1);
sort(a, 2, 1, 1);
sort(a, 2, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 3, 1);
sort(a, 4, 4, 1);
sort(a, 5, 4, 1);
sort(a, 5, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 6, 1);
sort(a, 7, 7, 1);
sort(a, 8, 9, 1);
sort(a, 10, 9, 1);
sort(a, 10, 9, 1);
sort(a, 10, 9, 1);
sort(a, 10, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);
sort(a, 16, 15, 1);

Sorted result
0  ai
1  al
2  co
3  fo
4  go
5  is
6  no
7  of
8  pa
9  pe
10 th
11 th
12 th
13 ti
14 to
15 to

Trace of recursive calls for MSD string sort (no cutoff for small subarrays, subarrays of size 0 and 1 omitted)

input   __      __                      output
no      al      ai      ai      ai      ai
is      ai      al      al      al      al
th      co      --      co      co      co
ti      fo      co      fo      fo      fo
fo      go      fo      go      go      go
al      is      go      is      is      is
go      no      is      no      no      no
pe      of      no      of      of      of
to      pe      of      --      pa      pa
co      pa      pe      pa      pe      pe
to      th      pa      pe      --      th
th      ti      th      --      th      th
ai      to      ti      th      th      th
of      to      to      ti      th      ti
th      th      to      to      ti      to
pa      th      th      to      to      to
        --      th      th      to
                        th      --


Question: Draw the NFA corresponding to the pattern ( ( ( A | B ) * | C D * | E F G ) * ) * .

Answer:
5.1.4

Trace for 3-way string quicksort (same model as used in the book):

                   --      --      --
0  no      is      ai      ai      ai      ai
1  is      ai      co      al      --      al
2  th      co      fo      --      al      co
3  ti      fo      al      fo      --      fo
4  fo      al      go      go      co      go
5  al      go      --      co      --      is
6  go      --      is      --      fo      no
7  pe      no      --      is      --      of
8  to      --      no      --      go      pa
9  co      to      --      no      --      pe
10 to      pe      pe      --      is      th
11 th      to      of      of      --      th
12 ai      th      pa      --      no      th
13 of      ti      --      pe      --      ti
14 th      of      th      pa      of      to
15 pa      th      ti      --      --      to
           pa      to      th      pa
           th      th      th      --
                   to      th      pe
                   th      --      --
                   --      to      th
                           to      th
                           ti      th
                                   --
                                   ti
                                   --
                                   to
                                   to
                                   --


Question: Draw the digraph of ε-transitions for the NFA from Exercise 5.4.4.

Answer:
5.1.5

According to ASC II values, indices of chars 'a' through 'z' start at 97.
But this trace indices follow the book convention of 'a' starting at 0.

Trace for MSD string sort (same model as used in the book):

Top level of sort(array, 0, 13, 0):

input
0  now
1  is
2  the
3  time
4  for
5  all
6  good
7  people
8  to
9  come
10 to
11 the
12 aid
13 of

d=0
Count frequencies
0    0
1    0
2  a 2
3  b 0
4  c 1
5  d 0
6  e 0
7  f 1
8  g 1
9  h 0
10 i 1
11 j 0
12 k 0
13 l 0
14 m 0
15 n 1
16 o 1
17 p 1
18 q 0
19 r 0
20 s 0
21 t 5
22 u 0
23 v 0
24 x 0
25 w 0
26 y 0
27 z 0

Transform counts to indices
0    0
1    0
2  a 2
3  b 2
4  c 3
5  d 3
6  e 3
7  f 4
8  g 5
9  h 5
10 i 6
11 j 6
12 k 6
13 l 6
14 m 6
15 n 7
16 o 8
17 p 9
18 q 9
19 r 9
20 s 9
21 t 14
22 u 14
23 v 14
24 x 14
25 w 14
26 y 14
27 z 14

Distribute and copy back
0  all
1  aid
2  come
3  for
4  good
5  is
6  now
7  of
8  people
9  the
10 time
11 to
12 to
13 the

Indices at completion of distribute phase
0    0
1    2
2  a 2
3  b 3
4  c 3
5  d 3
6  e 4
7  f 5
8  g 5
9  h 6
10 i 6
11 j 6
12 k 6
13 l 6
14 m 7
15 n 8
16 o 9
17 p 9
18 q 9
19 r 9
20 s 14
21 t 14
22 u 14
23 v 14
24 x 14
25 w 14
26 y 14
27 z 14

Recursively sort subarrays
sort(a, 0, 1, 1);
sort(a, 2, 1, 1);
sort(a, 2, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 2, 1);
sort(a, 3, 3, 1);
sort(a, 4, 4, 1);
sort(a, 5, 4, 1);
sort(a, 5, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 5, 1);
sort(a, 6, 6, 1);
sort(a, 7, 7, 1);
sort(a, 8, 8, 1);
sort(a, 9, 8, 1);
sort(a, 9, 8, 1);
sort(a, 9, 8, 1);
sort(a, 9, 13, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);
sort(a, 13, 14, 1);

Sorted result
0  aid
1  all
2  come
3  for
4  good
5  is
6  now
7  of
8  people
9  the
10 the
11 time
12 to
13 to

Trace of recursive calls for MSD string sort (no cutoff for small subarrays, subarrays of size 0 and 1 omitted)

input       ____        ___                                 output
now         all         aid         aid         aid         aid
is          aid         all         all         all         all
the         come        ---         come        come        come
time        for         come        for         for         for
for         good        for         good        good        good
all         is          good        is          is          is
good        now         is          now         now         now
people      of          now         of          of          of
to          people      of          people      people      people
come        the         people      ---         ---         the
to          time        the         the         the         the
the         to          time        the         the         time
aid         to          to          time        ---         to
of          the         to          to          time        to
            ----        the         to          to
                                    ---         to


Question: Give the sets of states reachable by your NFA from EXERCISE 5.4.4 after each character match and susbsequent ε-transitions for the input A B B A C E F G E F G C A A B .

Answer:
5.1.6

Trace for 3-way string quicksort (same model as used in the book):

                    ----     ---      ---      ---
0  now      is      aid      aid      aid      aid      aid
1  is       aid     come     all      ---      ---      all
2  the      come    for      ---      all      all      come
3  time     for     all      for      ---      ---      for
4  for      all     good     good     come     come     good
5  all      good    ----     come     ----     ----     is
6  good     ---     is       ----     for      for      now
7  people   now     ---      is       ----     ----     of
8  to       ---     now      ---      good     good     people
9  come     to      ---      now      ----     ----     the
10 to       people  people   ---      is       is       the
11 the      to      of       of       ---      ---      time
12 aid      the     ---      ------   now      now      to
13 of       time    to       people   ---      ---      to
            of      the      ------   of       of
            the     time     the      ------   ------
                    to       time     people   people
                    the      the      ------   ------
                    ---      ----     the      the
                             to       the      the
                             to       ----     ----
                             ----     time     time
                                      ----     ----
                                      to       to
                                      to       to
                                      --       --


Question:
Write a regular expression for each of the following sets of binary strings:
a. Contains at least three consecutive 1s
b. Contains the substring 110
c. Contains the substring 1101100
d. Does not contain the substring 110

Answer:
5.1.8

Both MSD string sort and 3-way string quicksort examine all characters in the N keys.
That number is equal to 1 + 2 + ... + N = (N^2 + N) / 2 characters.
MSD string sort, however, generates (R - 1) * N empty subarrays (an empty subarray for all digits in R other than 'a', in every pass) while 3-way string quicksort generates 2N empty subarrays (empty subarrays for digits smaller than 'a' and for digits higher than 'a', or empty subarrays for digits smaller than '-1' and for digits equal to '-1', in every pass).

MSD string sort trace (no cutoff for small subarrays, subarrays of size 0 and 1 omitted):

input   ----
a       a       a       a       a       a       a
aa      aa      ----    aa      aa      aa      aa
aaa     aaa     aa      ----    aaa     aaa     aaa
aaaa    aaaa    aaa     aaa     ----    aaaa    aaaa
...     ...     aaaa    aaaa    aaaa    ----    ...
        ----    ...     ...     ...     ...
                ----    ----    ----    ----

3-way string quicksort trace:

input   ----
a       a       a       a       a       a       a
aa      aa      ----    aa      aa      aa      aa
aaa     aaa     aa      ----    aaa     aaa     aaa
aaaa    aaaa    aaa     aaa     ----    aaaa    aaaa
...     ...     aaaa    aaaa    aaaa    ----    ...
        ----    ...     ...     ...     ...
                ----    ----    ----    ----


Question:
Write a regular expression for each of the following sets of binary strings:
a. Has at least 3 characters, and the third character is 0
b. Number of 0s is a multiple of 3
c. Starts and ends with the same character
d. Odd length
e. Starts with 0 and has odd length, or starts with 1 and has even length
f. Length is at least 1 and at most 3

Answer:
5.1.10

The total number of characters examined by 3-way string quicksort when sorting N fixed-length strings (all of length W) in the worst case is O(N * W * R).

This can be seen with a recurrence relation T(W).

The base case T(1) is when all the strings have length 1.
An example with R = 3 is { "a", "b", "c" }.
In the worst case they are in reverse order.
For example: { "c", "b", "a" }.
In this case we only remove one string from the list in each pass.
If we consider N = R^W (in this case, W = 1), the number of comparisons is equal to:
Characters examined = Sum[i=0..R] i
Characters examined = R * (R + 1) / 2

To build the worst case for strings of length 2 (T(2)), we take each string from T(1) and append it to the end of each character in R.
So for single character strings "a", "b", "c", with R = 3, the two character list is: "aa", "ab", "ac", "ba", "bb", "bc", "ca", "cb", "cc".
The list can then be split into R groups: one for each character in R that is a prefix to every string of length W - 1.
During the partitioning phase all strings that start with "a" will be in the same partition and the algorithm will do the same process as in T(1) because removing the first character 'a' will lead to the same 1-length strings { "c", "b", "a" } as before.
The same thing happens for strings starting with "b" and "c".
So, for R = 3, the algorithm will check 3 * R + 2 * R + R characters in the first position of the strings (which is 3 + 2 + 1 characters times R groups).
Then it will check the second characters in the strings in each of the R groups.

For T(W), where W > 2, the list will then again be split into R groups: one for each character in R that is a prefix to every string of length W - 2.
Quicksort will then remove R strings from the list in each partition.
It will then check R * T(W - 1) more characters for each of those groups.
This gives the recurrence T(W) = (R^(W - 1) * Sum[i=0..R] R - i) + R * T(W - 1), which simplifies to:
T(W) = R^(W + 1) + R^W  + R * T(W - 1)
      -----------------
              2
Solving the recurrence gives us:
T(W) = W * (R^W) * (R + 1)
      ---------------------
                2
Substituting N = R^W:
T(W) = W * N * (R + 1)
      -----------------
              2
Which is O(N * W * R).

Thanks to dragon-dreamer (https://github.com/dragon-dreamer) for finding a more accurate worst case.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/153
Thanks to GenevaS (https://github.com/GenevaS) for finding a more accurate worst case.
https://github.com/reneargento/algorithms-sedgewick-wayne/issues/245