Simulated Annealing - When brute-force takes too long

[RTFC]

Simulated Annealing (SA) is a simple technique for finding an acceptable solution (but not necessarily always the absolute best one that exists!) to very hard combinatorial problems, that is, ones for which a brute-force approach of cycling through all possible alternatives to find the global optimum just takes too darn long. Typically, one would be seeking some specific sequence or permutation of a (sub)set, and the number of possibilities is astronomically large. In addition, for SA to be applicable, you'll need to be able to quantify in some way how good any particular trial solution is, or how far distant from the ideal result. The real power of SA lies in these so-called cost functions; you can define as many as you like, with different weights if you like, and these conditions are even allowed to be conflicting. User-defined settings define how tenaciously it should be exploring solution space before homing in on a region with desirable properties, and finding its minimum. You can think of it as a learning algorithm that is allowed to make lots of mistakes in the beginning, before gradually avoiding ever more potentially bad decisions.

A simple Analogy:

You're standing in the middle of an extensive, rough terrain with hills, ridges, bumps, valleys, and tiny, deep depressions. You've got a GPS altimeter to tell you how high above sea-level you currently are, but this area is perpetually shrouded in thick fog. Now find the lowest point. And quickly please. Now what? There's no time to systematically grid and traverse the entire area, and you cannot see more than a few feet ahead. Following the local gradient down-slope will likely get you stuck at a very local minimum, whereas a much lower point may be just beyond the hext hill.

Luckily, you brought your magic boots (the red ones with the twinkling silver stars). These allow you to make huge leaps through the fog, landing safely somewhere else. And although you don't know in advance where you'll end up, the boots magically remember their last-previous departure point, so if you don't like your new surroundings, you can go back one jump (but never more than that). Now to find the lowest point in the landscape, you keep tracking your altimeter changes with every jump. Some jumps will get you to higher ground, others might land you in a deep valley. The trick is not to settle for ever-lower heights immediately, but to allow going back up ever so often, so you can get beyond some high ridge behind which you might find a much lower depression than your best-previous result. Crucially, to decide whether or not to gain height in the misty terrain, you roll some dice you've got in your pocket, and the more jumps you've made, the lower you make the upper bound below which you would still go back uphill. So in the beginning you'll be jumping all over the place (allowing you to sample the terrain extensively), but eventually you'll be limiting yourself to some deep valley you've found, which might be the deepest one all round, but even if not, it will still be a pretty good guess. And you will have found this deep valley in a tiny fraction of the time it would take to do a full land survey.

Simulated annealing needs to be defined in terms of the specific problem you're trying to solve. So it's not possible to provide you with a generic UDF that'll figure out in advance what your ideal solution would look like (for example, in the analogy above, we might be looking for the highest point instead of the lowest one). You need to define that ideal in terms of one or more cost functions that SA will then attempt to minimise. What can be done is to provide you with specific examples.

Example 1.

From a list of user-defined pre-supplied values, select the fewest terms that sum to (or approximate) a predefined target total. This solution was written in response to this thread. Current cost updates are periodically written to console. This example attempts to satisfy two conditions simultaneously: getting a sum that matches the target, and using the smallest number of terms.

expandcollapse popup

; Simulated Annealing example (combinatorial minimisation), by RTFC (22 Feb 2016)

; Note that this algorithm converges on A *local* minimum (in terms of the
; user-defined cost-function(s)), which is not necessarily THE *global* minimum.
; Note also that the search path, duration, and final result may differ from run to run.
; Several parameters can be tweaked to adjust this.

#include <Array.au3>
#include <Math.au3>

Global $temperat,$path,$kk,$nswap,$nswapstep,$cost,$altcost,$tempstep
Global $costadjust,$ttlsites,$totalcost,$factor,$maxsumlength,$maxsize
Global $site1,$site2,$index1,$index2,$weight_sum,$weight_length
Global $options,$sumlength,$prevlength

; initialise
SRandom(@SEC+@AutoItPID)    ; initialise radnomising seed
$verbose=True           ; T: write regular progress updates to console
$factor=1               ; optional Oracle-response adjustment (not used here)
$prevlength=$sumlength  ; to enable reverting to previous state after _TryChange
$minsumlength=0         ; if nothing else is know, we'll start with a single array entry (base-0)

$options=10             ; larger value = larger likelihood of swapping vs changing sumlength
if $options<3 then Exit ; minimum size for this set-up (see Func _TrySwap)

; adjust the balance of conditions here (see _Cost function)
$weight_sum=1           ; relative importance of matching condition 1 (sum = target)
$weight_length=1        ; relative importance of matching condition 2 (lowest number of terms)

; summation results buffer
Global $bestsum[10]
Global $bestsumlength=UBound($bestsum)

; define the summation result we wish to achieve (try different values here!)
$target = 27
; Note: if you change this value, you may have to adjust $minsumlength below as well!

; define our array of summation terms to select from
$dim=9                  ; this is just the way this problem was presented
$ttlsites=$dim*$dim
$maxsize=$ttlsites-1
$maxsumlength=$ttlsites-2   ; need at least one tail slot for swapping

; enable this for the predefined problem...
$doIntegerTest=True ; False
if $doIntegerTest Then
    Global $aArray = [9,2,2,3,1,1,6,3,4, _
                    4,2,3,4,5,6,7,8,7, _
                    7,2,3,4,5,1,7,2,2, _
                    2,2,3,1,5,5,7,1,4, _
                    3,2,1,2,3,6,6,6,4, _
                    3,2,3,4,5,6,7,8,3, _
                    2,2,3,8,1,4,7,1,2, _
                    1,7,3,5,5,6,7,1,2, _
                    7,2,3,7,5,1,7,8,9]

    ; in this specific case, we already know that we'll need at least 4 terms
    ; because a) 9=max value in array, b) there are only 2 nines in the array, and c) target =3x9
    $minsumlength=3     ; base-0, so 4 entries altogether

Else
    ; OR use this for random floats in range 1-$dim (just as an example)
    Global $aArray[$ttlsites]
    For $cc=0 to $maxsize
        $aArray[$cc]=random(1,$dim,0)   ; non-integer values used here!
    Next

    ; no clue about this constraint in this case
    $minsumlength=0 ; one entry (base-0)
EndIf

;______START OF ANNEALING ROUTINE____________

$nover =1000            ; maximum number of changes at any temperature (for more complicated problems, set this several orders of magnitude higher)
$nlimit=Int($nover/4)   ; maximum number of successful changes before continuing
$nwrite=Int($nover/5)   ; default status update interval if verbose=.t.
$tempsteps=100          ; number of temperature steps to try
$tfactor=0.95           ; annealing schedule: temperature is reduced by this factor after each step

While True
    $temperat=0.5       ; initial temperature; smaller = more aggressive + more myopic search
    $absimp=0           ; counter
    $nswapstepzero=0    ; counter
    $sumlength=$minsumlength        ; base-0

    ; prep the cost vars
    $totalcost=_Cost()
    $cost=$totalcost
    $lowestcost=$totalcost
    $initcost=$totalcost

    ; main loop starts here
    For $tempstep=1 to $tempsteps       ; try up to N temperature steps
        $nswap=0
        $nswapstep=0
        For $kk=1 to $nover
            _TrySwap()      ;   swap and determine cost adjustment

            Switch _AskOracle()     ; Feel the Force, Luke.
                Case True
                    $nswap+=1
                    $totalcost+=$costadjust
                    $cost=$altcost

                    If $lowestcost>$totalcost Then
                        $nswapstep+=1
                        $absimp+=1
                        $lowestcost=$totalcost

                        ; ensure results buffer is sufficiently large
                        If $bestsumlength<=$sumlength Then
                            $bestsumlength+=5
                            ReDim $bestsum[$bestsumlength]
                        EndIf

                        ; flush current-best summation
                        For $bc=0 to $sumlength-1
                            $bestsum[$bc]=$aArray[$bc]
                        Next

                        ; pad tail with zeroes
                        For $bc=$sumlength to $bestsumlength-1
                            $bestsum[$bc]=0
                        Next

                        _ScreenOut()
                        If $totalcost<=0 Then ExitLoop
                    Endif

                Case Else   ; restore the previous state
                    $sumlength=$prevlength
                    $aArray[$index1]=$site1
                    $aArray[$index2]=$site2
            EndSwitch

            ; show we're still alive
            If $verbose And mod($kk,$nwrite)=0 Then _ScreenOut()
            If $nswap>=$nlimit Or $lowestcost<=0 then ExitLoop
        Next

        ; optional early-out scenario (disable for a more thorough search)
        If $nswapstep=0 then $nswapstepzero+=1
        If $nswapstepzero=10 then ExitLoop      ; no more improvements in the last N temperature steps

        ; reduce temperature = likelihood of following a trajectory away from the nearest LOCAL optimum (in the hope of getting nearer to the GLOBAL optimum)
        $temperat*=$tfactor
    Next

    ; present final result
    _Arraysort($bestsum)    ; just for clarity

    $summation="Best result so far (at a cost of " & $lowestcost & ") is: " & @CRLF
    $terms=0
    $result=0
    For $cc=0 to $sumlength
        If $aArray[$cc]>0 then
            $summation&=$aArray[$cc] & "+"
            $result+=$aArray[$cc]
            $terms+=1
        Endif
    Next
    $summation=StringTrimRight($summation,1) & " = " & $result & " (target = " & $target & ")" & @CRLF
    $summation&="Number of summation terms: " & $terms & @CR & "Temperature steps: " & $tempstep & @CR & @CR & "Press <Ok> to try again, <Cancel> to Quit"

    if Msgbox($MB_OKCANCEL,"Simulated Annealing Test Result",$summation)=$IDCANCEL then Exit

    ; shuffle entries a bit for variety
    For $cc=1 to $maxsize*10
        $index1=random(0,$maxsize,1)
        $index2=random(0,$maxsize,1)
        $tmp=$aArray[$index1]
        $aArray[$index1]=$aArray[$index2]
        $aArray[$index2]=$tmp
    Next
WEnd

Exit


Func _AskOracle()

    If $costadjust<0 Then
        Return True
    Else        ; this is where all the magic happens!
        Return (random()<Exp(-($costadjust*$factor)/$temperat))
    Endif

EndFunc


Func _TrySwap()

    $index1=0       ; these vars are all Globals
    $index2=0
    $altcost=0
    $prevlength=$sumlength

    ; decide whether to reduce/increase number of terms, or swap an existing term
    Switch Random(1,$options,1)
        Case 1  ; crop
            $sumlength=_Max($minsumlength,$sumlength-1)

        Case 2  ; extend
            $sumlength=_Min($maxsumlength,$sumlength+1)

        Case Else   ; this likelhood is determined by the value of $options (>=3)
            $index1=random(0,$sumlength,1)
            $index2=random($sumlength+1,$maxsize,1)
    EndSwitch

    ; store current contents, in case we decide later that this was a bad idea
    $site1=$aArray[$index1]
    $site2=$aArray[$index2]

    ; swap contents for now
    $aArray[$index1]=$site2
    $aArray[$index2]=$site1

    ; compute the new sum (as either length or content has changed)
    $altcost=_Cost()

    ; performance difference between original and new state
    $costadjust=$altcost-$cost  ; $cost is already filled in previous pass

EndFunc


Func _Cost()

    Local $cc,$result=0
    For $cc=0 to $sumlength
        $result+=$aArray[$cc]
    Next
    Return (Abs($result-$target)*$weight_sum) + (($sumlength-$minsumlength)*$weight_length)

EndFunc


Func _ScreenOut()

    ConsoleWrite("Simulated Annealing. Initial total cost: " & $initcost & @CRLF)
    ConsoleWrite("Step: " & $tempstep & " of " & $tempsteps & "; Temperature: " & $temperat & @CRLF)
    ConsoleWrite("Executed Swaps: " & $nswap & "; Lowest Cost so far: " & $lowestcost & @CRLF)
    ConsoleWrite("Total Improvements: " & $absimp & "; Improvements this step: " & $nswapstep & @CRLF  & @CRLF)

EndFunc

 

Example 2. The Travelling Salesman problem (TSP)

This is a classic combinatorial minimisation problem, and relevant to real-world logistics: to find the shortest route for visiting all cities exactly once, before returning to the original starting point. As it is quite entertaining to see how the algorithm gradually solves this brain teaser,  I've added a simple GUI that visualises the cities (red circles) and the changing routes between them (blue). The problem becomes exponentially harder to solve when the number of cities is increased. This example (adapted from Press et al., Numerical recipes, 2nd ed., pp. 438-443) employs a single cost function of the total route distance.

TSP.au3

It's important to stress that simulated annealing cannot guarantee that the global optimum will always be found, only that it will likely come up with a fairly good solution, and much faster than brute force ever could. If that's good enough for you, then those red, silver-starred boots might fit you too.

 

 

Edited May 15, 2016 by RTFC
example added

[JohnOne]

Cant see myself ever needing something like this, but I can't stop watching the blue lines, they're great.

[Melba23]

RTFC,

Wow, I expected something complex but this is going to take a fair while to understand. I am now reading this to try and get some idea of what is going on in there.

M23

[RTFC]

@JohnOne: Yeah, best not play too much with it, or you might go blind.

@M23: It really works mostly like I describe in the analogy. Rolling the dice = _AskOracle(), using an exponential distribution. We always accept a lower cost, but if it's higher, then we only accept the jump if we sample below our annealing temperature (well technically, cost scaled by temperature), and the temperature itself is gradually lowered. And the name annealing is apt, as the metal is cooled slowly to give the atoms the opportunity to settle into the crystal lattice.

Edited February 22, 2016 by RTFC

[RTFC]

Added a 3rd example in the first post, a Sudoku Generator & Solver.

Paraphrasing the script remarks section:

This example illustrates how some types of problem can cause Simulated Annealing to get stuck in a local optimum other than the global one. To get around this, we can apply a thorough reshuffle of all non-fixed parameters, and try again. The harder the problem is, the larger the average number of required reshuffles to find the full solution (see listed examples in script).

Sudoku puzzles with very few given clues (or none) are easy to solve, because many paths exist that lead to full solutions (non-unique for number of clues < 17). Sudoku's with many clues are also easy to solve, because there are only relatively few paths left, many of which yield the full solution.

Sudoku's with (or close to) the minimum number of clues that identify a unique solution are the hardest, because many paths do exist, but most lead to a sub-optimal, incomplete solution. The location of the clues also becomes increasingly important the closer we get to this minimum. Reshuffling allows us to explore this landscape from different starting points.

The new example script (#3) displays a temporary result (with timing) each time it gets stuck; when the true solution is found, it plays a sound before exiting. Note that this may take a long time.

NB This is obviously not the fastest way to solve a Sudoku; the point of this example is to show that SimAnn can (eventually) find it, without knowing how to solve it, just by getting feedback on its current attempts, and despite the solution space itself being rather large. Furthermore, this example does not imply that all intractable problems can be solved by repeated reshuffle + retry. For example, it would be useless to attempt to quickly generate bitcoins this way.

 

EDIT: after fixing a bug in updating $totalcost, it turns out sudoku isn't a particularly good example of simulated annealing after all (it just takes too long). Until I find a better way to implement it in this context I've removed this example.

Edited May 15, 2016 by RTFC