[SUMMARY] Verbal Arithmetic (#128)
 

I'm always impressed by the creativity of the solutions, but I think this week
stands out even more than usual. I literally spent hours going through the
solutions and learned some really great tricks from them. I wish I could take
you on the same tour of the code, but that would take this summary into the
range of a full text book in length.

Because I'm going to miss all of the following, let me point out some highlights
for your own explorations:

* Though the brute-force solutions are slow, most of them handle any
math equations Ruby can. That is an interesting advantage.
* Andreas Launila sent in a fun preview of his Google Summer of Code
project that looks to simplify many of these search problems we
commonly use as quizzes.
* Glen's solution is a nifty metaprogramming solution that customizes
itself to the equation entered. It's lightning quick too.
* Morton Goldberg solved the quiz with some genetic programming and
that code is still quite a bit zippier than a brute-force search.

The solution I will show is from Eric I. It has an interesting state machine
design that tries to fail fast in an attempt to aggressively prune the search
space. It too finds solutions quite rapidly, though it only works for addition
problems.

Eric's code breaks the equation down into a small series of steps. Instead of
searching for a match for all numbers and then checking the result, this
solution checks as many little sub-criteria as possible. Does just this column
add up correctly, given what we know at this point? Is this digit a zero,
because it starts a term somewhere else?

These smaller tests lead to failures that allow the search to skip large groups
in the set of possible solutions. For example, if S can't be seven in just one
column, it's impossible to have any scenario where S is seven and all such
attempts can be safely skipped. That allows the code to zoom in on a correct
answer faster.

Now that we understand the logic, let's start tackling the code:

require 'set'

# State represents the stage of a partially solved word equation. It
# keeps track of what digits letters map to, which digits have not yet
# been assigned to letters, and the results of the last summed column,
# including the resulting digit and any carry if there is one.
class State
attr_accessor :sum, :carry
attr_reader :letters

def initialize()
@available_digits = Set.new(0..9)
@letters = Hash.new
@sum, @carry = 0, 0
end

# Return digit for letter.
def [](letter)
@letters[letter]
end

# The the digit for a letter.
def []=(letter, digit)
# if the letter is currently assigned, return its digit to the
# available set
@available_digits.add @letters[letter] if @letters[letter]

@letters[letter] = digit
@available_digits.delete digit
end

# Clear the digit for a letter.
def clear(letter)
@available_digits.add @letters[letter]
@letters[letter] = nil
end

# Return the available digits as an array copied from the set.
def available_digits
@available_digits.to_a
end

# Tests whether a given digit is still available.
def available?(digit)
@available_digits.member? digit
end

# Receives the total for a column and keeps track of it as the
# summed-to digit and any carry.
def column_total=(total)
@sum = total % 10
@carry = total / 10
end
end

# ...

This State object tracks progress through the equation, which will be solved
column by column. It has operations to track what each letter is currently
assigned to, assign letters as they are determined, examine which digits have
and have not been used, and track the sum of the last column plus any value
carried over to the next column. There's nothing too tricky in this data
structure code.

What we need to go with this, is an algorithm that drives this State object to a
solution. That code begins here:

# ...

# Step is an "abstract" base level class from which all the "concrete"
# steps can be derived. It simply handles the storage of the next
# step in the sequence. Subclasses should provide 1) a to_s method to
# describe the step being performed and 2) a perform method to
# actually perform the step.
class Step
attr_writer :next_step
end

# ...

This base Step is about as simple as things get. I merely provides a means of
storing the next step in the process.

Note that this class's abstract status and the required implementation for
subclasses are all handled through the documentation. That's perfectly
reasonable in a dynamic language like Ruby where we can count on duck typing to
resolve to the proper methods when the search is actually being performed.

Let's advance to a concrete implementation of the Step class:

# ...

# This step tries assigning each available digit to a given letter and
# continuing from there.
class ChooseStep < Step
def initialize(letter)
@letter = letter
end

def to_s
"Choose a digit for \"#{@letter}\"."
end

def perform(state)
state.available_digits.each do |v|
state[@letter] = v
@next_step.perform(state)
end
state.clear(@letter)
end
end

# ...

This ChooseStep handles the digit guessing. It is created for some letter and
when perform() is triggered, it will try each unused in turn digit in that
position. After a new guess is set, the ChooseStep just hands off to a later
step to verify that the current guess works.

Here's another Step subclass:

# ...

# This step sums up the given letters and changes to state to reflect
# the sum. Because we may have to backtrack, it stores the previous
# saved sum and carry for later restoration.
class SumColumnStep < Step
def initialize(letters)
@letters = letters
end

def to_s
list = @letters.map { |l| "\"#{l}\"" }.join(', ')
"Sum the column using letters #{list} (and include carry)."
end

def perform(state)
# save sum and carry
saved_sum, saved_carry = state.sum, state.carry

state.column_total =
state.carry +
@letters.inject(0) { |sum, letter| sum + state[letter] }
@next_step.perform(state)

# restore sum and carry
state.sum, state.carry = saved_sum, saved_carry
end
end

# ...

This SumColumnStep will be added whenever guesses had been made for an entire
column. It's job is to add up that column and update the State with this new
total. You can see that it must save old State values and restore them when
backtracking.

Once we know a column total, we can use that to set a letter from the solution
side of the equation:

# ...

# This step determines the digit for a letter given the last column
# summed. If the digit is not available, then we cannot continue.
class AssignOnSumStep < Step
def initialize(letter)
@letter = letter
end

def to_s
"Set the digit for \"#{@letter}\" based on last column summed."
end

def perform(state)
if state.available? state.sum
state[@letter] = state.sum
@next_step.perform(state)
state.clear(@letter)
end
end
end

# ...

This AssignOnSumStep is added for letters in the solution of the equation. It
will set the value of that letter to the calculated sum of the column, provided
that is a legal non-duplicate digit choice.

When we have assigned that letter, we need to verify that the whole column makes
sense mathematically:

# ...

# This step will occur after a column is summed, and the result must
# match a letter that's already been assigned.
class CheckOnSumStep < Step
def initialize(letter)
@letter = letter
end

def to_s
"Verify that last column summed matches current " +
"digit for \"#{@letter}\"."
end

def perform(state)
@next_step.perform(state) if state[@letter] == state.sum
end
end

# ...

Now, if we did all the guessing, summing, and assigning everything probably adds
up. But as we continue through the equation, some numbers will already be
filled in. Sums created using those may not balance with the total digit. This
CheckOnSumStep watches for such a case.

If the sum doesn't check out, this class causes backtracking. Note that all it
has to do is not forward to the following steps which will cause recursion to
unwind the stack until it has another option.

One last check can trim the search space further:

# ...

# This step will occur after a letter is assigned to a digit if the
# letter is not allowed to be a zero, because one or more terms begins
# with that letter.
class CheckNotZeroStep < Step
def initialize(letter)
@letter = letter
end

def to_s
"Verify that \"#{@letter}\" has not been assigned to zero."
end

def perform(state)
@next_step.perform(state) unless state[@letter] == 0
end
end

# ...

This CheckNotZeroStep is used to ensure that a leading letter in a term is
non-zero. Again, it fails to forward calls when this is not the case.

One more step is needed to catch correct solutions:

# ...

# This step represents finishing the equation. The carry must be zero
# for the perform to have found an actual result, so check that and
# display a digit -> letter conversion table and dispaly the equation
# with the digits substituted in for the letters.
class FinishStep < Step
def initialize(equation)
@equation = equation
end

def to_s
"Display a solution (provided carry is zero)!"
end

def perform(state)
# we're supposedly done, so there can't be anything left in carry
return unless state.carry == 0

# display a letter to digit table on a single line
table = state.letters.invert
puts
puts table.keys.sort.map { |k| "#{table[k]}=#{k}" }.join(' ')

# display the equation with digits substituted for the letters
equation = @equation.dup
state.letters.each { |k, v| equation.gsub!(k, v.to_s) }
puts
puts equation
end
end

# ...

This method first ensures that we are successful by validating that we have no
remaining carry value. If that's true, our equation balanced out.

The rest of the work here is just in printing the found result. Nothing tricky
there.

We're now ready to get into the application code:

# ...

# Do a basic test for the command-line arguments validity.
unless ARGV[0] =~ Regexp.new('^[a-z]+(\+[a-z]+)*=[a-z]+$')
STDERR.puts "invalid argument"
exit 1
end

# Split the command-line argument into terms and figure out how many
# columns we're dealing with.
terms = ARGV[0].split(/\+|=/)
column_count = terms.map { |e| e.size }.max

# Build the display of the equation a line at a time. The line
# containing the final term of the sum has to have room for the plus
# sign.
display_columns = [column_count, terms[-2].size + 1].max
display = []
terms[0..-3].each do |term|
display << term.rjust(display_columns)
end
display << "+" + terms[-2].rjust(display_columns - 1)
display << "-" * display_columns
display << terms[-1].rjust(display_columns)
display = display.join("\n")
puts display

# AssignOnSumStep which letters cannot be zero since they're the first
# letter of a term.
nonzero_letters = Set.new
terms.each { |e| nonzero_letters.add(e[0, 1]) }

# A place to keep track of which letters have so-far been assigned.
chosen_letters = Set.new

# ...

This code validates the input and breaks it into terms. After that, the big
chunk of code here displays the equation in a pretty format, like the examples
from the quiz description.

The rest of the code begins to divide up the input as needed to build the proper
steps. The first tactic is to locate and letters that must be nonzero, because
they start a term. A set is also prepared to hold letters that have be given
values at any point in the process.

Here's the heart of the process code:

# ...

# Build up the steps needed to solve the equation.
steps = []
column_count.times do |column|
index = -column - 1
letters = [] # letters for this column to be added

terms[0..-2].each do |term| # for each term that's being added...
letter = term[index, 1]
next if letter.nil? # skip term if no letter in column
letters << letter # note that this letter is part of sum

# if the letter does not have a digit, create a ChooseStep
unless chosen_letters.member? letter
steps << ChooseStep.new(letter)
chosen_letters.add(letter)
steps << CheckNotZeroStep.new(letter) if
nonzero_letters.member? letter
end
end

# create a SumColumnStep for the column
steps << SumColumnStep.new(letters)

summed_letter = terms[-1][index, 1] # the letter being summed to

# check whether the summed to letter should already have a digit
if chosen_letters.member? summed_letter
# should already have a digit, check that summed digit matches it
steps << CheckOnSumStep.new(summed_letter)
else
# doesn't already have digit, so create a AssignOnSumStep for
# letter
steps << AssignOnSumStep.new(summed_letter)
chosen_letters.add(summed_letter)

# check whether this letter cannot be zero and if so add a
# CheckNotZeroStep
steps << CheckNotZeroStep.new(summed_letter) if
nonzero_letters.member? summed_letter
end
end

# ...

This code breaks down the provided equation into the steps we've seen defined up
to this point. Though it's a fair bit of code, it's pretty straightforward and
very well commented. In short:

1. Values are selected for the numbers in each column as needed.
2. Columns are summed
3. Sums are assigned and or validated as needed.

With the setup complete, here's the code that kicks the solver into action:

# ...

# should be done, so add a FinishStep
steps << FinishStep.new(display)

# print out all the steps
# steps.each_with_index { |step, i| puts "#{i + 1}. #{step}" }

# let each step know about the one that follows it.
steps.each_with_index { |step, i| step.next_step = steps[i + 1] }

# start performing with the first step.
steps.first.perform(State.new)

Here the FinishStep is added, all steps are linked, and the perform() call is
made to get the ball rolling. You can uncomment the second chunk of code to
have a human-readable explanation of the steps added to the output.

My thanks to all the super clever solvers who tackled this problem. I was blown
away with the creativity.

Tomorrow we will put Ruby Quiz to work helping some friends of ours...