View source

[[Category:Walnut|3]]

==Distributed Computing==

Multiple threads form the basis of conventional models of concurrent programming. Remarkably, human beings engage in concurrent distributed computing every day even though people are generally single threaded (there are exceptions: Walter Cronkite routinely listened to one broadcast with one ear, a different broadcast with the other, while simultaneously taking notes on another topic and speaking authoritatively to an audience of millions. But most people, including this author, live single-threaded lives).

How do we simple creatures pull off the feat of distributed computing without multiple threads? Why do we not see groups of people, deadlocked in frozen tableau around the coffepot, one person holding the sugar waiting for milk, the other holding milk waiting for sugar?

The answer is, we use a sophisticated computational mechanism known as a ''nonblocking promise''.

Let us look at a conventional human distributed computation. Alice, the CEO of Evernet Computing, needs a new version of the budget including R&D numbers from the VP of Engineering, Bob. Alice calls Bob: "Could you get me those numbers?"

Bob jots Alice's request on his to-do list. "Sure thing, Alice, I promise I'll get them for you after I solve this engineering problem."

Bob has handed Alice a promise for the answer. He has ''not '' handed her the answer. But neither Bob nor Alice sits on their hands, blocked, waiting for the resolution.

Rather, Bob continues to work his current problem. And Alice goes to Carol, the CFO: "Carol, when Bob gets those numbers, plug 'em into the spreadsheet and give me the new budget,okay?"

Carol: "No problem." Carol writes Alice's request on her own to-do list, but does not put it either first or last in the list. Rather, she puts it in the conditional part of the list, to be done when the condition is met--in this case, when Bob fulfills his promise.

Conceptually, Alice has handed to Carol a copy of Bob's promise for numbers, and Carol has handed to Alice a promise for a new integrated spreadsheet. Once again, no one waits around, blocked. Carol ambles down the hall for a contract negotiation, Alice goes back to preparing for the IPO.

When Bob finishes his calculations, he signals that his promise has been fulfilled; when Carol receives the signal, she uses Bob's fulfilled promise to fulfill her own promise; when Carol fulfills her promise, Alice gets her spreadsheet. A sophisticated distributed computation has been completed so simply that no one realizes an advanced degree in computer science should have been required.

In this simple example, we see why human beings never get trapped in the thread-based deadlock situations described endlessly in books on concurrent programming. People don't deadlock because they live in a concurrent world managed with a promise-based architecture. And so it is with <span class="e">''E''</span>.

This chapter on distributed computing is the longest and most challenging part of the book. The Promise Architecture of <span class="e">''E''</span> is very different from the threads, synchronized methods, guarded objects, and RMI of Java. One might expect the Security chapter and its Capability paradigm to be equally new and challenging. But in fact, the implementation of security is embedded so deeply in E's infrastructure that much of the effort merely involves realizing the power inherent in constructs we have already learned. Security with <span class="e">''E''</span> is more a matter of architecture than of coding.

Distributed computation, however, still requires gritty effort. We start simply, with the eventually operator.

{{:Walnut/Distributed Computing/Eventually Operator}}
{{:Walnut/Distributed Computing/Promises}}
{{:Walnut/Distributed Computing/Multiway when-catch}}
{{:Walnut/Distributed Computing/Partial Ordering Of Sent Messages}}
{{:Walnut/Distributed Computing/Naming conventions}}
{{:Walnut/Distributed Computing/Testing for Equality in a Distributed System}}

===Letting When-Catch make a Promise For You===

As noted in chapter 1 under the Secret Lives of Flow Control Structures, control structures return values. When used as described up to this point, the return value of the when--catch expression has been ignored. In fact, the when-catch expression returns a promise for the last value that the executed clause (the body or catch block) will return.

Rewriting the calcMilesBeforeEmptyVow using this feature, we get:

<pre>

 <nowiki># E sample 
 def calcMilesBeforeEmptyVow(carRcvr)  {  
     def fuelRemainingVow := carRcvr <- getFuelRemaining() 
     def mpgVow := carRcvr <- getEfficiency() 
     return when (fuelRemainingVow, mpgVow) -> { 
         mpgVow * fuelRemainingVow 
     }
 }</nowiki>

</pre>

Here the when-catch creates the promise which is returned by the function, and resolution is automatic at the end of the when-catch execution. 
So when the promise that the when-catch depends on is resolved the promise it returned will be resolved to the result of the last seqExp in the when clause (i.e. the "mpgVow * fuelRemainingVow" expression).

Note that we don't need a catch block here. The default behaviour if the promise that the when-catch depends on is smashed is to smash the promise of the when--catch block, which will trigger the catch block in the main program. Normally, you should either report an error or propagate it, so that every error gets reported exactly once.

Whatever happens, the finally clause has no effect on the resolution of the promise returned by when-catch.

Warning: A common mistake is to put the "return" statement inside the when block. This will lead to the confusing error message "Ejector must be enabled" (return is a type of "ejector", which causes a function to return a value, but by the time the when block executes, the original function call has already returned).

===Broken Promise Contagion===

When chaining promises, what happens if one of the promises in the middle or the far end breaks? The problem that breaks the far promise is propagated down all the promises dependent on it. As a consequence, broken promises work their way through the system in a similar way to exceptions being caught in try/catch blocks.

===State Transitions for References===

Having worked our way through near, far, broken, and promised references, let us put all the transitions among these together in a single place. The following diagram shows all the kinds of references <span class="e">''E''</span> distinguishes between, and it shows the two possible transitions:

<font color="rgb(255, 0, 0)"> [[Image:ref-mech-0.gif|state transition diagram]]</font>

The 2 transitions are:

* A promise may transition (resolve) into a resolved reference, which can be near or far or broken..
* A far ref may transition to a broken ref.

That is the complete collection of transitions.

===Guards for distributed systems===

Very early in the book, we presented a list of <span class="e">''E''</span> types, including any, integer, float64, etc. In that list were "rcvr", "vow", "pbc" and "near". "pbc" means that the type of the value is required to be "pass by construction". Transparent immutables are generally pbc, and can be passed through this type declaration. Strings, integers, floats, booleans, ConstMaps and ConstLists are all pbc. Objects that are pbc are copied when they are sent, so that even if the object originated on a remote computer, when you get it you can use immediate calls on it. Also, since pbc objects do not embody any code, pbc objects passed across a trust boundary can be used with far fewer security concerns: the pbc object may contain bad data, but it cannot play any tricky games in its execution.

"Near" means that the value must be local. Promises and far references will not pass the near guard. Anything that passes a near guard can be sent messages with immediate calls.

At this time, "rcvr" and "vow" have the same algorithmic meaning as "any", but can be used as hints to the programmer about how to interact with the object (a vow should be referenced only eventually until it is resolved; a rcvr should always be referenced only eventually).

===Nested When-Catch constructs===

As discussed earlier, we can guarantee the ordering of messages sent from one object to another object. But when a program sends messages to several objects, on different machines, the order in which answers will be received cannot be predicted. Suppose you need to perform a computation involving two far answers, and the algorithm depends on the result from the first answer. You can nest when-catch clauses to await the outcome of the dependency:

<pre>

 <nowiki># E syntax
 def fileRcvr := fileCacheRcvr <- getFile()
 when (def exists := fileRcvr <- exists()) -> {
     if (exists) {
         when (def text := fileRcvr <- getText()) -> {
             println(`File has text: $text`)
         } catch p2 {println(`Problem with existing file: $p2`)}
     } else {println("File does not exist")}
 } catch prob {println(`Problem reaching file: $prob`)}
     </nowiki>

</pre>

Here, we want to print the text of a farFile, but we want to be sure the file exists first. So we ask the file if it exists, and only after we determine that it does indeed exist do we proceed to get the text.

===Live Refs, Sturdy Refs, and URIs===

Up to this point we have carefully dodged a critical question. How does a program acquire its very first reference to an object on a different computer? In all our examples up to this point, we have always started out with a reference to at least one remote object, and we retrieved other remote objects by asking that object for other objects: for example, we asked a remote carMakerRcvr for a new (remote) carRcvr, and were able to immediately work with the car just like any other remote object. How did we get the reference to the carMakerRcvr in the first place?

In <span class="e">''E''</span>, the reference to an object can be encoded as a Universal Resource Identifier string, known as a ''uri'' (the familiar url of the Web is a type of uri). This uri string can be passed around in many fashions; one good secure way of passing a uri is to save it as a text file, encrypt and sign it with PGP, and send it in email. If you wish to run a seriously secure distributed <span class="e">''E''</span> system, encrypting the uris is crucial: indeed, the passing of the uris from machine to machine is the main security issue that <span class="e">''E''</span> cannot address for you (and is a problem shared by all security systems, a problem that will diminish with time as secure systems like PGP email are deployed). Other ways uris have been passed in operational <span class="e">''E''</span> systems have been to send the uri over an ssh connection, and (less securely) by reading the uri off over a telephone! If you are using <span class="e">''E''</span> on a local area network and have no security concerns, but are using <span class="e">''E''</span> simply because it is simpler, safer, and more maintainable for distributed computing, the uris can be stored in files on a shared file system and read directly by the programs on different computers as they start up.

The functions makeURI(object) and objFromURI(uri) detailed in the <span class="e">''E''</span> Quick Reference Card perform the basic transformations you need to hook up objects on multiple computers. These routines use sturdy refs and liverefs in their computations. A sturdyRef is an object that contains an enduring description of where an object can be found; sturdyRefs and URIs are simple transformations of one another. LiveRefs are actual connections to objects that disappear any time a connection goes down. LiveRefs carry the actual traffic in <span class="e">''E''</span> communication. When you request a remote object from another remote object (as in farCarMaker <- ("mercedes")), what you actually get is a liveRef. If you want to continue to connect with that particular car across multiple sessions, you will need to explicitly get a sturdyRef from the car (and the car will have to have an explicit method for granting the sturdyRef. The sturdyref function is an important capability, not one handed out by default).

Each program that expects to work with remote objects needs to invoke the command

<pre>

 <nowiki># E sample
 introducer.onTheAir() </nowiki>

</pre>

before starting any remote connections, including the making or using of uris. An example of these functions working together can be found in eChat.

===blockAtTop, continueAtTop, waitAtTop===

E programs never block. They never just sit and wait around for results. However, most programs will toss out some windows, or set up some services, and then desire to wait for someone or some thing to use them. Often, the first version of a program written by a new <span class="e">''E''</span> programmer will set everything up, execute the last line of the program, and then shut down. It is quite disconcerting to watch your windows briefly light up and then disappear, along with the java virtual machine underlying the entire program.

The command

<pre>

 interp.blockAtTop()

</pre>

causes the <span class="e">''E''</span> interpreter to wait for requests to come in. The command

<pre>

 interp.continueAtTop()

</pre>

causes the <span class="e">''E''</span> interpreter to stop waiting. These commands are used in the eChat sample at the end of this chapter: the last line of the program initiates blockAtTop, and continueAtTop is invoked when the user closes his chat window.

An alternative is to use:

<pre>
 interp.waitAtTop(promise)
</pre>

This is similar to blockAtTop, but continues automatically when the promise resolves.

===A bit of Philosophy when using Promises for the first time===

If a method accepts a promise or far reference as a parameter, the returned value almost certainly cannot be a local object. Why is this? If the computation of the returned object depends on something eventual, the computation will have to wait for an answer, which means the best the object can do for immediately returning something is to return a promise. So a method like getCar(licensePlateVow) has almost certainly violated the naming convention, one way or the other.

As a consequence of this and other characteristics of promises, beginning E programmers often experience a moment of breathlessness soon after they first try to use them. This moment occurs when the programmer erroneously concludes that, once you let a promise into your system, the program has tumbled down a slippery slope from which there is no recovery. It feels as if you will never be able to have a real value in your hands again; you'll spend the rest of your life making eventual sends, coupled with endless when-catch constructs just to get to the next step.

The breathlessness can metastasize into a feeling of having lost control of the software: everything is off computing somewhere else and all you have is a bunch of promises. Since <span class="e">''E''</span> never (really, never!) blocks, you can never tell things, "just stop for a moment while I catch my breath and gather up all the values" :-)

In fact, there are several patterns and tools for getting control of things again. The basic multi-promise when-catch, and the promiseAllResolved and Summoner patterns at the end of this chapter are just a few examples of structures specifically designed to reduce the frequency with which you need when-catch. They allow you to get quickly back into local computation after a distributed action. You really can catch your breath.

And once the moment of breathlessness passes, you will feel freedom, for you will never have to deal with thread synchronization again; you will never lie awake at night wondering if an unexplored race condition will cause deadlock a few days after the production load level hits your system. <font color="#ff0000">How early can I put this paragraph easily? AHK proposes right after Promises.</font>

===Additional Promise-Related Operations===

* whenBroken
* E isBroken
* whenResolved Use this when your action will be the same once the promise resolves, regardless of whether it resolves broken or not.
* E isResolved
* E send(object,verb,arguments) This function is identical to the E call() function described earlier, but does an eventual send rather than an immediate call.
* E sendOnly
* E join(a,b) Guarantees sends to result of join are delivered only after messages to a and b have been delivered, sort of a test for equality on-the-fly

===Under the Covers: Vats and Game Turns===

<font color="#ff0000">Each vat has a separate event loop, each game turn corresponds to the processing of a single event to conclusion with no interruptions. There are frontend vats, which all share the swing event queue for queuing messages to their event loops, and there are backend queues, each of which has its own separate event queue and runs on a separate thread. Frontend vats have the advantage that they can interact directly with user interface widgets. The initial vat created by default at the start of an E program is a frontend vat. You might want multiple frontend vats if you want to shut down whole subsystems in one fell swoop: you can tell a vat to shut down, and it will terminate all its connections to the outside world, making it available for garbage collection. Have markm review this and tell me what is wrong with this description.</font>

<font color="#ff0000">creating a second frontend vat on a single jvm. creating a backend vat on a jvm</font>

===Example: Alice, Bob, and Carol as Code===

We started this chapter with a description of a human interaction, with Alice as the CEO of Evernet Computing in search of a new spreadsheet for her IPO preparation. What would that distributed problem look like in code? Though we present only a skeleton of this human operation here, it is perhaps nonetheless informative. It might also be informative, after one has read how to do the example in <span class="e">''E''</span>, to think about how it would be done with threads, in Java or C++. That, however, is left as an exercise for the reader.

<pre>

 <nowiki># E sample
 #---- Bob Code
 
 def bobRcvr {
     to handleExplosionInNetworkServerRoom() {
        # handle the explosion
     }
     to getCostNumbers() :pbc {
         return ["R&D" => 10000]
     }
 }
 
 #------------ Carol Code
 
 def carolRcvr {
     to attendMeeting(room, time) {
         # attend the meeting
     }
     to integrateSpreadsheetVow(RDNumbersVow) {
         def sheetVow := when (RDNumbersVow) -> {
             ["Marketing" => 50000] | RDNumbersVow
         } catch prob {
             println("No numbers!")
             prob
         }
         return sheetVow
     }
 }
 
 #--------- Alice Code
 
 def aliceRcvr {
     to prepareIPOVow()  {
         def RDNumbersVow := bobRcvr <- getCostNumbers()
         def spreadsheet := carolRcvr <- integrateSpreadsheetVow(RDNumbersVow)
         return when (spreadsheet) -> {
             println(`Appendix: Budget Spreadsheet: $\n$spreadsheet`)
         } catch prob {
             aliceRcvr <- fireSomebody([bobRcvr, carolRcvr])
         }
         # do other IPO preparations
     }
     to fireSomebody(candidateRcvrs) {
         # get rid of culprit
     }
 }
 
 bobRcvr <- handleExplosionInNetworkServerRoom()
 carolRcvr <- attendMeeting(1,2)
 aliceRcvr <- prepareIPOVow()</nowiki>

</pre>

===Example: minChat version 1===

In order to focus on the distributed computation features of this chat tool, we have gone to extreme lengths to make the user interface machinery a very short piece of code. Hence, this should properly be called "minChat", because the user interface is so brutally minimal. Even with such minimization, however, the user interface still requires more lines of code than the actual distributed computation!

The chatController at the bottom of the example is the heart and soul of the computation.

<code>


 #?? in new vat minChatVat.e-swt
 ## E sample
 #!/usr/bin/env rune
 #eChat with minimalist user interface
 pragma.syntax("0.9")
 def <widget> := <swt:widgets.*>
 def SWT := <swt:SWT>
 introducer.onTheAir()
 # return the object represented by the URI
 def getObjectFromURI(uri)  {return introducer.sturdyFromURI(uri).getRcvr()}
 
 def makeURIFromObject(obj) :String {
     # This implementation assumes a non-persistent single incarnation
     def [sr, _, _] := identityMgr.makeKnown(obj)
     <span style="color:#FF0000">#XXX not a uri if bracketed, bug, markm?</span>
     def bracketed := introducer.sturdyToURI(sr)
     if (bracketed =~ `<@uri>`) {return uri}
     return bracketed
 }
 
 def chatController
 def chatArea
 def chatUI {
     to show() {
         def frame := <widget:Shell>(currentDisplay)
         frame.setText("eChat"); frame.setBounds(30, 30, 600, 300)
         def winDisposeListener {to widgetDisposed(event) {interp.continueAtTop()}}        
         frame.addDisposeListener(winDisposeListener)
         bind chatArea := <widget:Text>(frame, 
             (SWT.getMULTI() | SWT.getWRAP()) | (SWT.getBORDER() | SWT.getV_SCROLL()))
         def commandLine := <widget:Text>(frame, SWT.getSINGLE() | SWT.getBORDER())
         def enterKeyListener {
             to keyPressed (event) {
                 if (event.getKeyCode() == 27) {
                     if (commandLine.getText() =~ `@command @argument`) {
                         commandLine.setText("")
                         switch (command) {
                             match == "save" {chatController.save(<file: argument>)}
                             match == "load" {chatController.load(<file: argument>)}
                             match == "send" {chatController.send(argument)}
                             match _ {commandLine.setText("Error. Try save load, or send")}
                         }
                     } else {commandLine.setText("Error. Try save, load, or send")}
                 }
             }
             match [verb, args] {}
         }
         commandLine.addKeyListener(enterKeyListener)        
         def label := <widget:Label>(frame, SWT.getLEFT())
         label.setText("save, load, send. No quotes for path. Use Escape to start operation. ")
         swtGrid`$frame: $chatArea.X.Y
                         $label.X
                         $commandLine.X`
         frame.open()
     }
     to showMessage(initiator, text) {chatArea.append(`$initiator: $text $\n`)}
 }
 
 def friend
 bind chatController {
     to send(message) {
         when (friend<-receive(message)) -> {
             chatUI.showMessage("self", message)
         } catch prob {chatUI.showMessage("system", "connection lost")}
     }
     to receive(message) {chatUI.showMessage("friend", message)}
     to receiveFriend(friendRcvr) {
         bind friend := friendRcvr        
         chatUI.showMessage("system", "friend has arrived")
     }
     to save(file) {file.setText(makeURIFromObject(chatController))}
     to load(file) {
         bind friend := getObjectFromURI(file.getText())
         friend <- receiveFriend(chatController)
     }
 }
 chatUI.show()
 # In actual code, the following line would not be commented out
 # interp.blockAtTop()</nowiki>

</code>

There are only 5 methods in making chat happen: sending a message to the friend, receiving a message from the friend, being informed that the friend has found you (the ''receivedFriend ''method), saving the URI where your friend can find you, and loading the URI that describes where you can find your friend.

A quick run through can be done by putting the source in a file (file extension ".e-swt", this example uses SWT for user interface) and launching the file twice to get 2 chat windows. In one window, type "save ~/me.minchat" and press the Escape key. This creates a file (in your home directory) that contains a cap: URI describing where your chat session can be found (a chat session that can only be found by someone to whom this file has been given--the location cannot be found or guessed by anyone else). In the other window, type "load ~/me.minchat" and press Escape. In the first window you should see a message pop up that the "friend has arrived". Now in either window type "send hi y'all", and you will see the message appear in both windows.

In minChat , every time you start up the program, it comes to life with a new uri that must be sent to the friend with whom you wish to chat; the old uri from the previous session is useless. MinChat would be far more useful if you could create a minChat session with Bob, and continue that session even after you have rebooted your computer. We look at building such a persistent chat in the chapter on Persistent Secure Distributed Computing--after first looking at minChat here through the eyes of a security reviewer (or a cybercracker) in the chapter on Secure Distributed Computing.

===Patterns of Promises===

====resolveAllVow====

Note: this is available as '''<import:com.skyhunter.e.net.resolveAllVow>'''

Suppose you have a computation that cannot be performed until you have received answers from multiple remote objects. Your first thought would be to use a multi-vow when-catch. But further suppose you do not know beforehand how many vows there are (i.e., suppose you have an arbitrary list of promises). In this case neither the multi-vow when-catch construct nor the nested when-catch pattern will work. For this situation, you can use the resolveAllVow utility described here.

The resolveAllVow utility has another distinction compared to a multi-vow when-catch: whereas the multi-vow activates the catch clause as soon as it hits a broken promise, resolveAllVow sends no resolution until everything in the list is resolved one way or the other. So, even if several of the promises were broken, with resolveAllVow you can be sure that the rest have been fulfilled when the catch clause is activated.

In this example, we use resolveAllVow to sum a list of contributions. We do not know beforehand how many contributions there are (perhaps thousands), and we want to sum them up even if some of the requests for donations return broken promises.

E isBroken(obj) does not work. what does?

<pre>

 <nowiki># E sample
 # Given a List of promises, when all the promises have been resolved,
 # resolve the returned vow, either fulfilling it with the list of fulfillments (if
 # all the promises were fulfilled) or by smashing the returned promise
 # with one of the broken promises (if at least one promise was broken). 
 def resolveAllVow(promises) {
     def [resultVow, resolver] := Ref.promise()
     var count := promises.size()
     var resolution := promises
     var noneBroken := true
     if (count == 0) {resolver.resolve(promises)}
     for promise in promises {
         when (promise) -> {
             # success processed in finally clause
         } catch prob {
             resolution := prob
             noneBroken := false
         } finally {
             count -= 1
             if (count == 0) {
                 if (noneBroken) {
                     resolver.resolve(resolution)
                 } else {resolver.smash(resolution)}
             }
         }
     }
     return resultVow
 }
 #now use the promiseAllResolved to sum contributions, where the
 #number of contributions is unknown prior to execution
 def printTotalContributions(amountsList) {
     var total := 0
     for each in amountsList {
         if (!(Ref.isBroken(each))) {total += each}
     }
     println(`Total contributions are: $total`)
 }
 # scaffold contributors: real contributors would be remote Rcvrs
 def makeContributer(donation) {
     return def contributor { to getDonation() :int {return donation} }
 }
 def contributorRcvrsList := [makeContributer(5), makeContributer(6)]
 def amountsVows := [].diverge()
 for each in contributorRcvrsList {
     amountsVows.push(each <- getDonation())
 }
 when (def amounts := resolveAllVow(amountsVows)) -> {    
     printTotalContributions(amounts)
 } catch prob {
     #due to the nature of resolveAllVows, in this catch clause
     #we are guaranteed everything in amountsVows is resolved to 
     #either an amount or a broken reference
     printTotalContributions(amountsVows)
 }</nowiki>

</pre>

The basic behavior of resolveAllVow is this: when initiated, the function takes a count of how many promises there are, then immediately spins off when-catches for all of them. Each time a promise resolves, the count of outstanding promises is decremented. When the counter drops to zero, implying all the promises have been resolved, the single promise initially returned by resolveAllVow is finally resolved.

====dialogVowMaker====

Java's standard dialog boxes, found in JOptionPane, are seriously flawed for use in a distributed <span class="e">''E''</span> system: those convenience dialogs are fully blocking. As a consequence, if your computer is part of a distributed computation, and you go home for the night, if a Java dialog box pops up, your machine is effectively offline until you return in the morning. Included in the <span class="e">''E''</span> distribution is a dialog box tool, the dialogVowMaker:

<font color="#ff0000">code here</font>

====Recursion for send sequencing====

As discussed earlier, putting a when-catch inside a for loop does not give you any guarantees on which when-catch will activate first: it only guarantees which message is sent first, which is mostly uninteresting. For general-purpose ensured sequencing of resolution, you must use recursion.

Earlier, we had an example in which we told a list of cars to moveTo the same location. Suppose we wished to ensure that the cars arrived in the order in which they appear in the list:

<pre>

 <nowiki># E sample
 def moveAllCarsInSequence(carRcvrs,toX,toY) {
     var carI := 0
     def moveRemainingCars() {
         if (carI < carRcvrs.size()) {
             def nextCarRcvr := carRcvrs[carI]
             carI += 1
             def nameVow := nextCarRcvr <- getName()
             when (nameVow,nextCarRcvr <- moveTo(toX,toY)) -> {
                 println(nameVow + " arrived, next car about to start moving")
             } catch e {
                 println(`car died: $e`)
             } finally {moveRemainingCars()}
         }
     }
     moveRemainingCars()
 }
 # scaffold carmaker
 def makeCarRcvr(name) {
     def carRcvr {
         to getName() :String {return name}
         to moveTo(x, y) {
             #move the car
         }
     }
     return carRcvr
 }
 
 moveAllCarsInSequence([makeCarRcvr("car1"),makeCarRcvr("car2")],3,4) </nowiki>

</pre>

Inside the moveAllCarsInSequence function, we declare the recursive moveRemainingCars function, then simply invoke moveRemainingCars. The function moveRemainingCars tells the next car to move; once that movement has resolved, moveRemainingCars invokes itself once again.

In this version of moveAllCarsInSequence, if a car is lost (i.e., its promise to move is broken), the recursion continues. If we wanted to abort after a problem occurred, we would simply delete the call to moveRemainingCars() from the finally clause, and place it in the when clause.

====sendValve====

As we have seen, it is possible to spin off large numbers of eventual sends in the blink of an eye. In fact, tossing off vast numbers of such sends can sometimes consume a lot of resources for no good purpose. In the eDesk program at the end of the book, it is possible for the user to request the transfer of whole directories full of files. It would be possible for eDesk to initiate transfer of all of those files simultaneously. But the processing bottleneck is probably bandwidth, so initiating them all at once won't get the whole transfer done any faster, and meanwhile each individual file transfer consumes buffer space once initiated. Starting all the transfers at once could actually slow down the transfer if enough buffers thrash virtual memory.

What you want is a ''valve'', which can be opened and closed depending on the situation. The program can still set up all the operations in a single swoop, but the valve will constrain the number of such operations that are actually active at a given moment. The sendValve below performs this function.

<pre>

 <nowiki>#sendValve. If you have numerous eventual sends to initiate,
 # but initiating them all at once would consume vast resources
 # and/or would actually slow down processing, queue the actions 
 # through a valve.
 #An ActionTrio is the list of [obj,"verb",[arguments]]
 # that can be used in an E send()
 #The actions are guaranteed to be initiated in the sequence in
 # which they are placed in the valve, though of course there is
 # no guarantee as to which will terminate first (unless you have
 # special knowledge of such sequencing outside the valve).
 #The numSimultaneousAllowed is the number of actions that can
 #  be run concurrently through this valve.
 
 # E sample
 def makeSendValve (numSimultaneousAllowed) {
     var actionQueue := []
     var numRunning := 0
     def startNext() {
         if (!(actionQueue.size()==0)) {
             def [actionTrio, resolver] := actionQueue[0]
             actionQueue := actionQueue(1, actionQueue.size())
             numRunning += 1
             def completeVow := E.send(actionTrio[0],actionTrio[1],actionTrio[2])
             resolver.resolve(completeVow)
             when (completeVow) -> {
             } catch prob {
             } finally { 
                 numRunning -= 1 
                 startNext()
             }
         }
     }
     def  valve {
         to makeActionVow(actionTrio) {
             def [completeVow, resolver] := Ref.promise()
             actionQueue := actionQueue.with([actionTrio, resolver])
             if (numRunning < numSimultaneousAllowed) {
                 startNext()
             }
             return completeVow
         }
     }
     return valve
 }</nowiki>

</pre>

The sendValve utility, if used with numSimultaneousAllowed==1, can also perform the send sequencing function described earlier. However, the earlier general send sequencing pattern is still useful. For example, if the moveAllCarsInSequence operation should abort upon a encountering a broken resolution, this version of sendValve would not give the correct result.

====Summoning====

====Transparent forwarder====

It is possible to have a network of machines in which the machines do not all have direct access to one another. Suppose in the network with machine A, B, and C, A and C cannot form a direct connection because of network topology, but B can reach everything. If A needs a capability on C, we can put a transparent forwarder on B such that A sends its messages to the forwarder on B:

<pre>

 <nowiki># E sample
 def makeTransparentForwarder(representedRcvr) {
     def forwarder {
         match [verb,args] {E.send(representedRcvr,verb,args)}
     }
     return forwarder
 }</nowiki>

</pre>

Create the forwarder on B, and hand the reference to the forwarder to the appropriate object on A. An example can be found in the eDesk example at the end of the book, in which a forwarder is used if the system detects that two of the file servers are unable to directly connect. Note that the "delegate" keyword does not quite work here: delegate generates immediate calls, not eventual sends. So we had to revert to the more general purpose matching process.

====acceptOnlyMoreRecentVow====

Suppose you are periodically requesting a particular piece of information from a far object. Suppose the far object cannot supply the answer in a single game turn, i.e., it must ask other objects for information to get the answer, and so it sends you a promise rather than a result. It is possible for the resolutions of these promises to occur out-of-order, i.e., the resolution on the second request could get to you after the resolution of the third request has already occurred. In this situation, it will appear that the answer to the second request is newer and more up-to-date than the third request.

<font color="#000000">In this situation, to ensure you are getting only new information, not stale information, use the acceptOnlyMoreRecentVow utility.</font>

<pre>

 def acceptOnlyMoreRecentVow

</pre>

<font color="#ff0000">code here</font>

This situation, while rare, actually arises in the Marketplace example in the Secure Distributed Computing chapter.

<font color="#ff0000">something about time, the race construct, timeout construct</font>

====eDesk overwrite copy jewel====

The following is not really a pattern so much as it is an example of all the more novel (i.e., not-like-Java) elements of <span class="e">''E''</span> playing together. The problem, taken from the eDesk program in the Examples at the end of the book, is this:

example

===Data Lock===

Though <span class="e">''E''</span> is safe from deadlock, in which several objects compete for acquisition to a set of resources, there is a related dilemma that <span class="e">''E''</span> programs can encounter. If two or more "when" clauses wait for each other to resolve before they can resolve themselves, this causes ''data lock''. A complete discussion of data lock and why it is far less of a risk than deadlock is beyond the scope of this book. Briefly, both theory and current experience suggest that data locks are more difficult to create in the wild than deadlocks. Furthermore, when datalocks do occur, they are less likely to freeze up important subsystems on a grand scale: unlike the deadlock, they only choke a few "when" clauses, not a set of "critical sections" which have been named "critical" in thread programming for good reasons. And lastly, datalocks are consistent, reproducible, and therefore fixable, unlike the deadlocks that appear mysteriously when some incredibly arcane race condition occurs.

Here is a simple though foolish example of datalock. It is the E implementation of the English sentence, "This sentence is false."

<pre>

 def sentence := sentence <- not()

</pre>

In this construction, the sentence will forever be an unresolved promise. As noted above, the failure of this to resolve will not impede the progress of the rest of the program in the least. And it is very reliable: the unresolvability of the promise will appear the first time you run the program, and the second, and in every debugging pass thereafter.

At the time of this writing, data lock has been seen only once in the wild. In that instance, a widely used object (call it WUO),contained one piece of mutable state and one piece of immutable state. The object that computed new versions of WUO's mutable state needed the immutable part of the state, and requested the current version of WUO from the owner of the object. Unfortunately, the owner knew that a new version was under construction, and was only sending out promises for WUO until the update was resolved. The solution in this case was to refactor WUO. This resulted in an architecture that was cleaner even by normal object design criteria, without regard to the distributed usage behavior. Insufficient data is yet available to assess how commonly this type of solution will suffice.


Next Section: [[Walnut/Secure_Distributed_Computing|Secure Distributed Computing]]