http://wiki.erights.org/mediawiki/index.php?title=Special:Contributions&feed=atom&target=152.78.239.138Erights - User contributions [en]2024-03-28T20:49:45ZFrom ErightsMediaWiki 1.15.5-7http://wiki.erights.org/wiki/Walnut/introWalnut/intro2010-08-23T10:50:20Z<p>152.78.239.138: privilegedScope.bindings()</p>
<hr />
<div>[[Category:Walnut|1]]<br />
== Introduction==<br />
<br />
=== Why E? ===<br />
<br />
Java, Perl, Python, C++, Tcl, and on and on. Don't we have enough languages already? How could we justify bringing yet another programming language into the world?<br />
<br />
In fact, there is still a compelling reason for developing better programming languages: we keep on writing more complicated programs. And worse, we keep on writing more complicated systems of interdependent programs.<br />
<br />
We have entered the age of globally distributed computing with a vengeance. Anyone who has cobbled together a major system with a hodgepodge of Web servers, Java, JSP, SQL, CGI, CORBA, RMI, XML, and Perl knows that this cannot be the toolset we will use in 20 years. The sooner we move up to the tools of 20 years hence, the better off we will be. Into this situation we introduce <span class="e">''E''</span><nowiki>:</nowiki><br />
<br />
{| style="float: right; width: 2in; border-style: inset; border-width: 3;background-color: #FFFF99; font-size:80%" <br />
|<span class="e">E</span> uses a specific flavor of capability-based security known as ''object capabilities''. With object capabilities, capability-oriented programming has the same flavor as object-oriented programming, except that capability-oriented programming takes the usual OO strictures more seriously. Often when using object capabilities for security, one finds that a more secure program is simply a program that follows OO principles of modularization more closely.<br />
|}<br />
<br />
* <span class="e">''E''</span> has significant advantages compared to other popular programming languages for distributed computing. As one quick example of its power, <span class="e">E</span>'s ''promise-pipelining'' architecture ensures that ''deadlocks cannot occur.''<br />
<br />
* <span class="e">''E''</span> has dramatic advantages for ''secure'' distributed systems. All communication in <span class="e">''E''</span> is strongly encrypted, transparently to the programmer. Capability-based security enables the concise composition of powerful patterns of interoperation, patterns that enable extensive cooperation even in the presence of severely limited trust. Excluding user interface code, a simple but effective peer-to-peer secure chat system has been written in less than 30 lines of code; no more lines of code were required to write a basic digital-money bank server ''despite the severe security issues involved''. When the time comes for a security inspection, capability security allows simple reachability analysis to exclude huge swaths of code because they cannot embody a threat. As a consequence, auditing a system for security becomes cost effective to an extent that is simply unimaginable with other approaches, as documented in [http://www.combex.com/papers/darpa-report/index.html the DarpaBrowser report]. With <span class="e">''E''</span>, it is straightforward to create systems that run across the Internet that are as secure and safe as if the entire system were running on a single computer in your basement. As one of the original developers of Smalltalk observed, upon learning about the object-capability paradigm from <span class="e">E</span>, capability security is "natural security": if you shouldn't use it, you just can't see it.<br />
<br />
* <span class="e">''E''</span> can even enable the fearless yet powerful use of multi-party limited-trust mobile code. "Mobile code" is just about anything executable on your computer that you get from somewhere else. Every time you turn on a word processor, play a game of Solitaire, or double click on an email attachment, you are executing mobile code written by someone you probably don't know and should not trust. Yet we give Solitaire, Barbie Fashion Designer, and Christmas Elf Bowling the full power to read our most private documents, sell them on EBay to the highest bidder, and then delete all your files. Our grandchildren will laugh at how silly this was, yet today there is no choice. If Microsoft used <span class="e">''E''</span> instead of Visual Basic for its application language, Word and Excel would not be vectors for attacks like the original Love Letter virus. If all software were written in <span class="e">''E''</span>, Klez and BackOrifice could never have existed: indeed, the term "virus", so loved by the press because it implies incurability, would never have been applied to computing.<br />
<br />
These qualities cannot be achieved with traditional security approaches. Do not expect the next release of Java, Windows, or Linux to fix the problem: the flaws in these systems lie at the heart of their architectures, unfixable without breaking upward compatibility, as we shall discuss in the chapter on Secure Distributed Programming. Of course, there is nothing to prevent people from advertising that they are releasing a new, upward compatible, totally-secure version of a product. Just don't jump off the Brooklyn Bridge to buy it.<br />
<br />
<span class="e">''E''</span> wraps these strengths in a C/Java syntax to make it quickly comfortable for a large number of software engineers. It is built with objects at the core of its design, making it easy to write modular, readable, maintainable software using the strategies familiar from Java. It has the kind of powerful string handling that will be recognized and seized upon by the Perl programmer. For both better and for worse, <span class="e">''E''</span> is a dynamically typed language like Smalltalk, not a statically typed language like Java. Users of Perl and Python will immediately assert this is an advantage; Java and C++ programmers will not be so sure. A discussion of the merits and demerits of static typing could fill a book the size of this one. Many of the most complex yet most reliable systems in the world today have been developed with dynamically typed languages. If you are a Java programmer, unshakably convinced of the perfect correctness of static typing, all we can do is urge you to try E first and form your conclusions later. We believe you will find the experience both pleasant and productive, as the long heritage of programmers from Scheme to Smalltalk to Perl and Python have found in the past.<br />
<br />
=== Why not E? ===<br />
<br />
<span class="e">''E''</span> is not a panacea. The current implementation of <span class="e">''E''</span> runs on top of the Java Virtual Machine. As such, it is not a good language for low-level machine manipulation. So do not try to use <span class="e">''E''</span> for writing device drivers. And <span class="e">''E''</span>'s performance is quite unfavorable compared to raw C, so do not write the real-time kernel of a missile guidance system with it, either.<br />
<br />
Actually, most arguments over performance miss the most important points about software development. One of the few time-tested truths of software development is, "first get it to work, then get it to work fast". <span class="e">''E''</span> is an excellent language for getting a system to work. Once it is working, you can gain the performance rewards by profiling to see which portions of the code actually affect performance, then rewriting those (typically small) portions in Java, or C, or assembler, depending on your needs. In fact, for many traditional compute-intensive activities, <span class="e">''E''</span> is just as performance-efficient as anything else: when creating windows and buttons on the screen, you use the GUI widgets from the underlying JVM. Since the JVM uses native drawing machinery, there is little distinction among languages when working with windows.<br />
<br />
So perhaps <span class="e">''E''</span> is in fact almost a panacea. But not quite.<br />
<br />
=== Why this Book? Why not this Book? ===<br />
<br />
This book is intended as an introductory text for practical <span class="e">''E''</span> programming. Comparisons to Java are frequent, so some understanding of Java is desirable. If you wish to write software with point-and-click user interfaces, familiarity with either the Java Swing classes or the IBM Standard Widget Toolkit (SWT) is practically required.<br />
<br />
This book is not a language specification. This book is an introduction to get you started and productive. If you encounter some surprising behavior not explained here, please join the e-lang discussion group and ask there: many helpful people can be found. If you need a precise specification, look to the reference materials posted by the language author and keeper, Mark Miller, at the [http://www.erights.org/ ERights web site]. You can join the e-lang discussion group at the same site.<br />
<br />
=== Fireworks In Part II ===<br />
<br />
When a very early version of '''''E''''' and '''''E''''' ''in a Walnut ''were presented to a programmer chat group, this was one of the comments:<br />
<br />
''"I think the key to hooking someone is to make them read the whole walnut cause it just looks like another scripting language until you get into the security and distribution then its like fireworks."''<br />
<br />
Though '''''E''''' is a powerful language with which to write single-CPU programs, the main power of '''''E''''' becomes evident only after you move into distributed programming. It would be tempting to introduce the distributed computing features first...except that you can't really do any meaningful computing without the basic data, flow, function, and object structures. So this book introduces "Ordinary Programming" Part I before getting into the serious distributed computing facilities. However, since '''''E''''' was designed in the C/Java syntax tradition, an experienced programmer can probably glean enough from the [http://www.erights.org/elang/quick-ref.html Quick Reference Card] to skip directly to [[Walnut/Distributed Computing|Part 3 on Distributed Computing]]. If you are short of time and have the requisite background, we recommend that strategy. Go back and read Part I when you are convinced that '''''E''''''s power for distributed programming meets your needs.<br />
<br />
=== [[Rune]] and Trace Log ===<br />
<br />
If you have read this far, you will probably want to [[Downloads|retrieve a current version of '''''E''''']]. Follow the installation directions in the distribution.<br />
<br />
You will find the [[rune]] interactive interpreter by going into the '''''E ''''' directory and typing<br />
<br />
<blockquote><br />
<br />
java -jar e.jar --rune<br />
<br />
</blockquote><br />
<br />
Rune will be your friend for experimentation as you learn''''' E'''''. Indeed, you may find, as others have, that rune is useful even when developing ordinary Java: with this scratchpad you can quickly explore what the API for Java really does when you use it, quickly and easily, before you put Java code into the middle of a program that is difficult to test.<br />
<br />
<div style="color:red"><br />
''Crucial note: At the time of this writing, '''''E''''' is in transition to a return-keyword based syntax for returning values from methods and functions. The following examples will only work if you put the following pragmas at the top of '''''E''''' source files: ''<br />
</div><br />
<pre><br />
<br />
<nowiki># E sample<br />
pragma.syntax("0.9")</nowiki><br />
<br />
</pre><br />
<br />
Using Rune: One-line commands start with a question mark prompt and return the result on the next line in a comment:<br />
<br />
<pre><br />
<br />
<nowiki>? 2 + 2<br />
# value: 4</nowiki><br />
<br />
</pre><br />
<br />
Commands that cannot be completed in a single line will receive close-angle-bracket continuations when you press carriage return at the end of the first line; the whole set of lines from the question mark down to the end of the command (the closing brace) will be evaluated when you close the statement:<br />
<br />
<pre><br />
<br />
? if (3 < 5) {<br />
> println("is less")<br />
> }<br />
is less<br />
<br />
</pre><br />
<br />
Another one of the wonderful little '''''E''''' features that helps enormously in rune is the '''help(obj)''' function. Type "help(interp)" into rune, and you will get a complete listing of all the methods you can call on the interp object. You will be surprised at first glance by the number of methods there; these methods are explained throughout the course of the book, starting at the end of the discussion of '''''E''''' objects.<br />
<br />
You can get a list of all the objects available in the privileged scope using the "bindings" method, and then ask for help on any of them:<br />
<br />
<pre><br />
? privilegedScope.bindings()<br />
...<br />
# stdout<br />
# a TextWriter<br />
# stderr<br />
# a TextWriter<br />
# stdin<br />
# a NearRef<br />
...<br />
? help(stdin)<br />
</pre><br />
<br />
We will present examples both as straight '''''E''''' source and as rune source, depending on how complex the example is.<br />
<br />
The Trace Log will be one of your best friends when you step outside of rune and start writing '''''E''''' programs. When you install '''''E''''', one of the items in the "eprops.txt" file is a specification of the directory where trace information should be written when a program fails during execution. Find your etrace directory (enter <code>interp.getProps()["TraceLog_dir"]</code>), and go there when you need debugging information. This is an important resource--more than one early adopter of '''''E''''' has kicked himself for forgetting that it is there, then spending several hours trying to understand a problem that was simple and obvious if you just looked at the log.<br />
<br />
If you find it annoying to have to keep opening the log file, you can get log messages sent to stderr by setting '''TraceLog_dir''' to '''-'''. You can put configuration settings in '''~/.e/user-eprops.txt'''. For example, this configuration sends trace logging to the console and removes the need to press Return after the program exits dues to an error:<br />
<br />
# ~/.e/user-eprops.txt<br />
DTraceLog_dir=-<br />
e.onErrorExit=report<br />
<br />
Next Part: [[Walnut/Ordinary_Programming|Ordinary Programming]]</div>152.78.239.138http://wiki.erights.org/wiki/Walnut/Distributed_ComputingWalnut/Distributed Computing2010-08-17T13:45:58Z<p>152.78.239.138: /* resolveAllVow */</p>
<hr />
<div>[[Category:Walnut|3]]<br />
<br />
==Distributed Computing==<br />
<br />
Multiple threads form the basis of conventional models of concurrent programming. Remarkably, human beings engage in concurrent distributed computing every day even though people are generally single threaded (there are exceptions: Walter Cronkite routinely listened to one broadcast with one ear, a different broadcast with the other, while simultaneously taking notes on another topic and speaking authoritatively to an audience of millions. But most people, including this author, live single-threaded lives).<br />
<br />
How do we simple creatures pull off the feat of distributed computing without multiple threads? Why do we not see groups of people, deadlocked in frozen tableau around the coffepot, one person holding the sugar waiting for milk, the other holding milk waiting for sugar?<br />
<br />
The answer is, we use a sophisticated computational mechanism known as a ''nonblocking promise''.<br />
<br />
Let us look at a conventional human distributed computation. Alice, the CEO of Evernet Computing, needs a new version of the budget including R&D numbers from the VP of Engineering, Bob. Alice calls Bob: "Could you get me those numbers?"<br />
<br />
Bob jots Alice's request on his to-do list. "Sure thing, Alice, I promise I'll get them for you after I solve this engineering problem."<br />
<br />
Bob has handed Alice a promise for the answer. He has ''not '' handed her the answer. But neither Bob nor Alice sits on their hands, blocked, waiting for the resolution.<br />
<br />
Rather, Bob continues to work his current problem. And Alice goes to Carol, the CFO: "Carol, when Bob gets those numbers, plug 'em into the spreadsheet and give me the new budget,okay?"<br />
<br />
Carol: "No problem." Carol writes Alice's request on her own to-do list, but does not put it either first or last in the list. Rather, she puts it in the conditional part of the list, to be done when the condition is met--in this case, when Bob fulfills his promise.<br />
<br />
Conceptually, Alice has handed to Carol a copy of Bob's promise for numbers, and Carol has handed to Alice a promise for a new integrated spreadsheet. Once again, no one waits around, blocked. Carol ambles down the hall for a contract negotiation, Alice goes back to preparing for the IPO.<br />
<br />
When Bob finishes his calculations, he signals that his promise has been fulfilled; when Carol receives the signal, she uses Bob's fulfilled promise to fulfill her own promise; when Carol fulfills her promise, Alice gets her spreadsheet. A sophisticated distributed computation has been completed so simply that no one realizes an advanced degree in computer science should have been required.<br />
<br />
In this simple example, we see why human beings never get trapped in the thread-based deadlock situations described endlessly in books on concurrent programming. People don't deadlock because they live in a concurrent world managed with a promise-based architecture. And so it is with <span class="e">''E''</span>.<br />
<br />
This chapter on distributed computing is the longest and most challenging part of the book. The Promise Architecture of <span class="e">''E''</span> is very different from the threads, synchronized methods, guarded objects, and RMI of Java. One might expect the Security chapter and its Capability paradigm to be equally new and challenging. But in fact, the implementation of security is embedded so deeply in E's infrastructure that much of the effort merely involves realizing the power inherent in constructs we have already learned. Security with <span class="e">''E''</span> is more a matter of architecture than of coding.<br />
<br />
Distributed computation, however, still requires gritty effort. We start simply, with the eventually operator.<br />
<br />
{{:Walnut/Distributed Computing/Eventually Operator}}<br />
{{:Walnut/Distributed Computing/Promises}}<br />
{{:Walnut/Distributed Computing/Multiway when-catch}}<br />
{{:Walnut/Distributed Computing/Partial Ordering Of Sent Messages}}<br />
{{:Walnut/Distributed Computing/Naming conventions}}<br />
{{:Walnut/Distributed Computing/Testing for Equality in a Distributed System}}<br />
<br />
===Letting When-Catch make a Promise For You===<br />
<br />
As noted in chapter 1 under the Secret Lives of Flow Control Structures, control structures return values. When used as described up to this point, the return value of the when--catch expression has been ignored. In fact, the when-catch expression returns a promise for the value that the done function will return. If the done function does not return a value, then it will always resolve to null. However, if the <font color="#FF0000">done(value) function</font> of the when-catch returns a value, the when-catch returns a promise that will be fulfilled by the last value in the executed clause. Rewriting the calcMilesBeforeEmptyVow using this feature, we get: <span class="warn" style="color:red">not right, not right, fix</span><br />
<br />
<pre><br />
<br />
<nowiki># E sample <br />
def calcMilesBeforeEmptyVow(carRcvr) { <br />
def fuelRemainingVow := carRcvr <- getFuelRemaining() <br />
def mpgVow := carRcvr <- getEfficiency() <br />
return when (fuelRemainingVow, mpgVow) -> { <br />
mpgVow * fuelRemainingVow <br />
} catch prob { <br />
throw(`Car Lost: $prob`)<br />
}<br />
}</nowiki><br />
<br />
</pre><br />
<br />
Here the when-catch creates the promise which is returned by the function, and resolution is automatic at the end of the when-catch execution. <br />
So when the promise that the when-catch depends on is resolved the promise it returned will be resolved to the result of the last seqExp in the when clause but if the promise that the when-catch depends on is smashed the returned promise will be smashed by the throw in the catch clause. Eather way the finally clause has no effect on the resolution of the promise returned by when-catch.<br />
<br />
===Broken Promise Contagion===<br />
<br />
When chaining promises, what happens if one of the promises in the middle or the far end breaks? The problem that breaks the far promise is propagated down all the promises dependent on it. As a consequence, broken promises work their way through the system in a similar way to exceptions being caught in try/catch blocks.<br />
<br />
===State Transitions for References===<br />
<br />
Having worked our way through near, far, broken, and promised references, let us put all the transitions among these together in a single place. The following diagram shows all the kinds of references <span class="e">''E''</span> distinguishes between, and it shows the two possible transitions:<br />
<br />
<font color="rgb(255, 0, 0)"> [[Image:ref-mech-0.gif|state transition diagram]]</font><br />
<br />
The 2 transitions are:<br />
<br />
* A promise may transition (resolve) into a resolved reference, which can be near or far or broken..<br />
* A far ref may transition to a broken ref.<br />
<br />
That is the complete collection of transitions.<br />
<br />
===Guards for distributed systems===<br />
<br />
Very early in the book, we presented a list of <span class="e">''E''</span> types, including any, integer, float64, etc. In that list were "rcvr", "vow", "pbc" and "near". "pbc" means that the type of the value is required to be "pass by construction". Transparent immutables are generally pbc, and can be passed through this type declaration. Strings, integers, floats, booleans, ConstMaps and ConstLists are all pbc. Objects that are pbc are copied when they are sent, so that even if the object originated on a remote computer, when you get it you can use immediate calls on it. Also, since pbc objects do not embody any code, pbc objects passed across a trust boundary can be used with far fewer security concerns: the pbc object may contain bad data, but it cannot play any tricky games in its execution.<br />
<br />
"Near" means that the value must be local. Promises and far references will not pass the near guard. Anything that passes a near guard can be sent messages with immediate calls.<br />
<br />
At this time, "rcvr" and "vow" have the same algorithmic meaning as "any", but can be used as hints to the programmer about how to interact with the object (a vow should be referenced only eventually until it is resolved; a rcvr should always be referenced only eventually).<br />
<br />
===Nested When-Catch constructs===<br />
<br />
As discussed earlier, we can guarantee the ordering of messages sent from one object to another object. But when a program sends messages to several objects, on different machines, the order in which answers will be received cannot be predicted. Suppose you need to perform a computation involving two far answers, and the algorithm depends on the result from the first answer. You can nest when-catch clauses to await the outcome of the dependency:<br />
<br />
<pre><br />
<br />
<nowiki># E syntax<br />
def fileRcvr := fileCacheRcvr <- getFile()<br />
when (def exists := fileRcvr <- exists()) -> {<br />
if (exists) {<br />
when (def text := fileRcvr <- getText()) -> {<br />
println(`File has text: $text`)<br />
} catch p2 {println(`Problem with existing file: $p2`)}<br />
} else {println("File does not exist")}<br />
} catch prob {println(`Problem reaching file: $prob`)}<br />
</nowiki><br />
<br />
</pre><br />
<br />
Here, we want to print the text of a farFile, but we want to be sure the file exists first. So we ask the file if it exists, and only after we determine that it does indeed exist do we proceed to get the text.<br />
<br />
As noted earlier, <font color="#FF0000">the "done(variable)" clause </font>of the when-catch construct is actually the declaration of a function and a series of parameters. The reasons for this are beyond the scope of this book, but the operational consequence is that you must use different function names if you have multiple when-catch clauses in the same scope.<br />
<br />
Nested when-catch clauses put the "done" functions into the same scope. In the example here, we gave the inner when-catch "done" function the name "done2" to prevent name collision. We also gave the error-containing variables in the catch clauses different names, e and e2.<br />
<br />
===Live Refs, Sturdy Refs, and URIs===<br />
<br />
Up to this point we have carefully dodged a critical question. How does a program acquire its very first reference to an object on a different computer? In all our examples up to this point, we have always started out with a reference to at least one remote object, and we retrieved other remote objects by asking that object for other objects: for example, we asked a remote carMakerRcvr for a new (remote) carRcvr, and were able to immediately work with the car just like any other remote object. How did we get the reference to the carMakerRcvr in the first place?<br />
<br />
In <span class="e">''E''</span>, the reference to an object can be encoded as a Universal Resource Identifier string, known as a ''uri'' (the familiar url of the Web is a type of uri). This uri string can be passed around in many fashions; one good secure way of passing a uri is to save it as a text file, encrypt and sign it with PGP, and send it in email. If you wish to run a seriously secure distributed <span class="e">''E''</span> system, encrypting the uris is crucial: indeed, the passing of the uris from machine to machine is the main security issue that <span class="e">''E''</span> cannot address for you (and is a problem shared by all security systems, a problem that will diminish with time as secure systems like PGP email are deployed). Other ways uris have been passed in operational <span class="e">''E''</span> systems have been to send the uri over an ssh connection, and (less securely) by reading the uri off over a telephone! If you are using <span class="e">''E''</span> on a local area network and have no security concerns, but are using <span class="e">''E''</span> simply because it is simpler, safer, and more maintainable for distributed computing, the uris can be stored in files on a shared file system and read directly by the programs on different computers as they start up.<br />
<br />
The functions makeURI(object) and objFromURI(uri) detailed in the <span class="e">''E''</span> Quick Reference Card perform the basic transformations you need to hook up objects on multiple computers. These routines use sturdy refs and liverefs in their computations. A sturdyRef is an object that contains an enduring description of where an object can be found; sturdyRefs and URIs are simple transformations of one another. LiveRefs are actual connections to objects that disappear any time a connection goes down. LiveRefs carry the actual traffic in <span class="e">''E''</span> communication. When you request a remote object from another remote object (as in farCarMaker <- ("mercedes")), what you actually get is a liveRef. If you want to continue to connect with that particular car across multiple sessions, you will need to explicitly get a sturdyRef from the car (and the car will have to have an explicit method for granting the sturdyRef. The sturdyref function is an important capability, not one handed out by default).<br />
<br />
Each program that expects to work with remote objects needs to invoke the command<br />
<br />
<pre><br />
<br />
<nowiki># E sample<br />
introducer.onTheAir() </nowiki><br />
<br />
</pre><br />
<br />
before starting any remote connections, including the making or using of uris. An example of these functions working together can be found in eChat.<br />
<br />
===blockAtTop and continueAtTop===<br />
<br />
E programs never block. They never just sit and wait around for results. However, most programs will toss out some windows, or set up some services, and then desire to wait for someone or some thing to use them. Often, the first version of a program written by a new <span class="e">''E''</span> programmer will set everything up, execute the last line of the program, and then shut down. It is quite disconcerting to watch your windows briefly light up and then disappear, along with the java virtual machine underlying the entire program.<br />
<br />
The command<br />
<br />
<pre><br />
<br />
interp.blockAtTop()<br />
<br />
</pre><br />
<br />
causes the <span class="e">''E''</span> interpreter to wait for requests to come in. The command<br />
<br />
<pre><br />
<br />
interp.continueAtTop()<br />
<br />
</pre><br />
<br />
causes the <span class="e">''E''</span> interpreter to stop waiting. These commands are used in the eChat sample at the end of this chapter: the last line of the program initiates blockAtTop, and continueAtTop is invoked when the user closes his chat window.<br />
<br />
===A bit of Philosophy when using Promises for the first time===<br />
<br />
If a method accepts a promise or far reference as a parameter, the returned value almost certainly cannot be a local object. Why is this? If the computation of the returned object depends on something eventual, the computation will have to wait for an answer, which means the best the object can do for immediately returning something is to return a promise. So a method like getCar(licensePlateVow) has almost certainly violated the naming convention, one way or the other.<br />
<br />
As a consequence of this and other characteristics of promises, beginning E programmers often experience a moment of breathlessness soon after they first try to use them. This moment occurs when the programmer erroneously concludes that, once you let a promise into your system, the program has tumbled down a slippery slope from which there is no recovery. It feels as if you will never be able to have a real value in your hands again; you'll spend the rest of your life making eventual sends, coupled with endless when-catch constructs just to get to the next step.<br />
<br />
The breathlessness can metastasize into a feeling of having lost control of the software: everything is off computing somewhere else and all you have is a bunch of promises. Since <span class="e">''E''</span> never (really, never!) blocks, you can never tell things, "just stop for a moment while I catch my breath and gather up all the values" :-)<br />
<br />
In fact, there are several patterns and tools for getting control of things again. The basic multi-promise when-catch, and the promiseAllResolved and Summoner patterns at the end of this chapter are just a few examples of structures specifically designed to reduce the frequency with which you need when-catch. They allow you to get quickly back into local computation after a distributed action. You really can catch your breath.<br />
<br />
And once the moment of breathlessness passes, you will feel freedom, for you will never have to deal with thread synchronization again; you will never lie awake at night wondering if an unexplored race condition will cause deadlock a few days after the production load level hits your system. <font color="#ff0000">How early can I put this paragraph easily? AHK proposes right after Promises.</font><br />
<br />
===Additional Promise-Related Operations===<br />
<br />
* whenBroken<br />
* E isBroken<br />
* whenResolved Use this when your action will be the same once the promise resolves, regardless of whether it resolves broken or not.<br />
* E isResolved<br />
* E send(object,verb,arguments) This function is identical to the E call() function described earlier, but does an eventual send rather than an immediate call.<br />
* E sendOnly<br />
* E join(a,b) Guarantees sends to result of join are delivered only after messages to a and b have been delivered, sort of a test for equality on-the-fly<br />
<br />
===Under the Covers: Vats and Game Turns===<br />
<br />
<font color="#ff0000">Each vat has a separate event loop, each game turn corresponds to the processing of a single event to conclusion with no interruptions. There are frontend vats, which all share the swing event queue for queuing messages to their event loops, and there are backend queues, each of which has its own separate event queue and runs on a separate thread. Frontend vats have the advantage that they can interact directly with user interface widgets. The initial vat created by default at the start of an E program is a frontend vat. You might want multiple frontend vats if you want to shut down whole subsystems in one fell swoop: you can tell a vat to shut down, and it will terminate all its connections to the outside world, making it available for garbage collection. Have markm review this and tell me what is wrong with this description.</font><br />
<br />
<font color="#ff0000">creating a second frontend vat on a single jvm. creating a backend vat on a jvm</font><br />
<br />
===Example: Alice, Bob, and Carol as Code===<br />
<br />
We started this chapter with a description of a human interaction, with Alice as the CEO of Evernet Computing in search of a new spreadsheet for her IPO preparation. What would that distributed problem look like in code? Though we present only a skeleton of this human operation here, it is perhaps nonetheless informative. It might also be informative, after one has read how to do the example in <span class="e">''E''</span>, to think about how it would be done with threads, in Java or C++. That, however, is left as an exercise for the reader.<br />
<br />
<pre><br />
<br />
<nowiki># E sample<br />
#---- Bob Code<br />
<br />
def bobRcvr {<br />
to handleExplosionInNetworkServerRoom() {<br />
# handle the explosion<br />
}<br />
to getCostNumbers() :pbc {<br />
return ["R&D" => 10000]<br />
}<br />
}<br />
<br />
#------------ Carol Code<br />
<br />
def carolRcvr {<br />
to attendMeeting(room, time) {<br />
# attend the meeting<br />
}<br />
to integrateSpreadsheetVow(RDNumbersVow) {<br />
def sheetVow := when (RDNumbersVow) -> {<br />
["Marketing" => 50000] | RDNumbersVow<br />
} catch prob {<br />
println("No numbers!")<br />
prob<br />
}<br />
return sheetVow<br />
}<br />
}<br />
<br />
#--------- Alice Code<br />
<br />
def aliceRcvr {<br />
to prepareIPOVow() {<br />
def RDNumbersVow := bobRcvr <- getCostNumbers()<br />
def spreadsheet := carolRcvr <- integrateSpreadsheetVow(RDNumbersVow)<br />
return when (spreadsheet) -> {<br />
println(`Appendix: Budget Spreadsheet: $\n$spreadsheet`)<br />
} catch prob {<br />
aliceRcvr <- fireSomebody([bobRcvr, carolRcvr])<br />
}<br />
# do other IPO preparations<br />
}<br />
to fireSomebody(candidateRcvrs) {<br />
# get rid of culprit<br />
}<br />
}<br />
<br />
bobRcvr <- handleExplosionInNetworkServerRoom()<br />
carolRcvr <- attendMeeting(1,2)<br />
aliceRcvr <- prepareIPOVow()</nowiki><br />
<br />
</pre><br />
<br />
===Example: minChat version 1===<br />
<br />
In order to focus on the distributed computation features of this chat tool, we have gone to extreme lengths to make the user interface machinery a very short piece of code. Hence, this should properly be called "minChat", because the user interface is so brutally minimal. Even with such minimization, however, the user interface still requires more lines of code than the actual distributed computation!<br />
<br />
The chatController at the bottom of the example is the heart and soul of the computation.<br />
<br />
<code><br />
<br />
<br />
#?? in new vat minChatVat.e-swt<br />
## E sample<br />
#!/usr/bin/env rune<br />
#eChat with minimalist user interface<br />
pragma.syntax("0.9")<br />
def <widget> := <swt:widgets.*><br />
def SWT := <swt:SWT><br />
introducer.onTheAir()<br />
# return the object represented by the URI<br />
def getObjectFromURI(uri) {return introducer.sturdyFromURI(uri).getRcvr()}<br />
<br />
def makeURIFromObject(obj) :String {<br />
# This implementation assumes a non-persistent single incarnation<br />
def [sr, _, _] := identityMgr.makeKnown(obj)<br />
<span style="color:#FF0000">#XXX not a uri if bracketed, bug, markm?</span><br />
def bracketed := introducer.sturdyToURI(sr)<br />
if (bracketed =~ `<@uri>`) {return uri}<br />
return bracketed<br />
}<br />
<br />
def chatController<br />
def chatArea<br />
def chatUI {<br />
to show() {<br />
def frame := <widget:Shell>(currentDisplay)<br />
frame.setText("eChat"); frame.setBounds(30, 30, 600, 300)<br />
def winDisposeListener {to widgetDisposed(event) {interp.continueAtTop()}} <br />
frame.addDisposeListener(winDisposeListener)<br />
bind chatArea := <widget:Text>(frame, <br />
(SWT.getMULTI() | SWT.getWRAP()) | (SWT.getBORDER() | SWT.getV_SCROLL()))<br />
def commandLine := <widget:Text>(frame, SWT.getSINGLE() | SWT.getBORDER())<br />
def enterKeyListener {<br />
to keyPressed (event) {<br />
if (event.getKeyCode() == 27) {<br />
if (commandLine.getText() =~ `@command @argument`) {<br />
commandLine.setText("")<br />
switch (command) {<br />
match == "save" {chatController.save(<file: argument>)}<br />
match == "load" {chatController.load(<file: argument>)}<br />
match == "send" {chatController.send(argument)}<br />
match _ {commandLine.setText("Error. Try save load, or send")}<br />
}<br />
} else {commandLine.setText("Error. Try save, load, or send")}<br />
}<br />
}<br />
match [verb, args] {}<br />
}<br />
commandLine.addKeyListener(enterKeyListener) <br />
def label := <widget:Label>(frame, SWT.getLEFT())<br />
label.setText("save, load, send. No quotes for path. Use Escape to start operation. ")<br />
swtGrid`$frame: $chatArea.X.Y<br />
$label.X<br />
$commandLine.X`<br />
frame.open()<br />
}<br />
to showMessage(initiator, text) {chatArea.append(`$initiator: $text $\n`)}<br />
}<br />
<br />
def friend<br />
bind chatController {<br />
to send(message) {<br />
when (friend<-receive(message)) -> {<br />
chatUI.showMessage("self", message)<br />
} catch prob {chatUI.showMessage("system", "connection lost")}<br />
}<br />
to receive(message) {chatUI.showMessage("friend", message)}<br />
to receiveFriend(friendRcvr) {<br />
bind friend := friendRcvr <br />
chatUI.showMessage("system", "friend has arrived")<br />
}<br />
to save(file) {file.setText(makeURIFromObject(chatController))}<br />
to load(file) {<br />
bind friend := getObjectFromURI(file.getText())<br />
friend <- receiveFriend(chatController)<br />
}<br />
}<br />
chatUI.show()<br />
# In actual code, the following line would not be commented out<br />
# interp.blockAtTop()</nowiki><br />
<br />
</code><br />
<br />
There are only 5 methods in making chat happen: sending a message to the friend, receiving a message from the friend, being informed that the friend has found you (the ''receivedFriend ''method), saving the URI where your friend can find you, and loading the URI that describes where you can find your friend.<br />
<br />
A quick run through can be done by putting the source in a file (file extension ".e-swt", this example uses SWT for user interface) and launching the file twice to get 2 chat windows. In one window, type "save ~/me.minchat" and press the Escape key. This creates a file (in your home directory) that contains a cap: URI describing where your chat session can be found (a chat session that can only be found by someone to whom this file has been given--the location cannot be found or guessed by anyone else). In the other window, type "load ~/me.minchat" and press Escape. In the first window you should see a message pop up that the "friend has arrived". Now in either window type "send hi y'all", and you will see the message appear in both windows.<br />
<br />
In minChat , every time you start up the program, it comes to life with a new uri that must be sent to the friend with whom you wish to chat; the old uri from the previous session is useless. MinChat would be far more useful if you could create a minChat session with Bob, and continue that session even after you have rebooted your computer. We look at building such a persistent chat in the chapter on Persistent Secure Distributed Computing--after first looking at minChat here through the eyes of a security reviewer (or a cybercracker) in the chapter on Secure Distributed Computing.<br />
<br />
===Patterns of Promises===<br />
<br />
====resolveAllVow====<br />
<br />
Note: this is available as '''<import:com.skyhunter.e.net.resolveAllVow>'''<br />
<br />
Suppose you have a computation that cannot be performed until you have received answers from multiple remote objects. Your first thought would be to use a multi-vow when-catch. But further suppose you do not know beforehand how many vows there are (i.e., suppose you have an arbitrary list of promises). In this case neither the multi-vow when-catch construct nor the nested when-catch pattern will work. For this situation, you can use the resolveAllVow utility described here.<br />
<br />
The resolveAllVow utility has another distinction compared to a multi-vow when-catch: whereas the multi-vow activates the catch clause as soon as it hits a broken promise, resolveAllVow sends no resolution until everything in the list is resolved one way or the other. So, even if several of the promises were broken, with resolveAllVow you can be sure that the rest have been fulfilled when the catch clause is activated.<br />
<br />
In this example, we use resolveAllVow to sum a list of contributions. We do not know beforehand how many contributions there are (perhaps thousands), and we want to sum them up even if some of the requests for donations return broken promises.<br />
<br />
E isBroken(obj) does not work. what does?<br />
<br />
<pre><br />
<br />
<nowiki># E sample<br />
# Given a List of promises, when all the promises have been resolved,<br />
# resolve the returned vow, either fulfilling it with the list of fulfillments (if<br />
# all the promises were fulfilled) or by smashing the returned promise<br />
# with one of the broken promises (if at least one promise was broken). <br />
def resolveAllVow(promises) {<br />
def [resultVow, resolver] := Ref.promise()<br />
var count := promises.size()<br />
var resolution := promises<br />
var noneBroken := true<br />
if (count == 0) {resolver.resolve(promises)}<br />
for promise in promises {<br />
when (promise) -> {<br />
# success processed in finally clause<br />
} catch prob {<br />
resolution := prob<br />
noneBroken := false<br />
} finally {<br />
count -= 1<br />
if (count == 0) {<br />
if (noneBroken) {<br />
resolver.resolve(resolution)<br />
} else {resolver.smash(resolution)}<br />
}<br />
}<br />
}<br />
return resultVow<br />
}<br />
#now use the promiseAllResolved to sum contributions, where the<br />
#number of contributions is unknown prior to execution<br />
def printTotalContributions(amountsList) {<br />
var total := 0<br />
for each in amountsList {<br />
if (!(Ref.isBroken(each))) {total += each}<br />
}<br />
println(`Total contributions are: $total`)<br />
}<br />
# scaffold contributors: real contributors would be remote Rcvrs<br />
def makeContributer(donation) {<br />
return def contributor { to getDonation() :int {return donation} }<br />
}<br />
def contributorRcvrsList := [makeContributer(5), makeContributer(6)]<br />
def amountsVows := [].diverge()<br />
for each in contributorRcvrsList {<br />
amountsVows.push(each <- getDonation())<br />
}<br />
when (def amounts := resolveAllVow(amountsVows)) -> { <br />
printTotalContributions(amounts)<br />
} catch prob {<br />
#due to the nature of resolveAllVows, in this catch clause<br />
#we are guaranteed everything in amountsVows is resolved to <br />
#either an amount or a broken reference<br />
printTotalContributions(amountsVows)<br />
}</nowiki><br />
<br />
</pre><br />
<br />
The basic behavior of resolveAllVow is this: when initiated, the function takes a count of how many promises there are, then immediately spins off when-catches for all of them. Each time a promise resolves, the count of outstanding promises is decremented. When the counter drops to zero, implying all the promises have been resolved, the single promise initially returned by resolveAllVow is finally resolved.<br />
<br />
====dialogVowMaker====<br />
<br />
Java's standard dialog boxes, found in JOptionPane, are seriously flawed for use in a distributed <span class="e">''E''</span> system: those convenience dialogs are fully blocking. As a consequence, if your computer is part of a distributed computation, and you go home for the night, if a Java dialog box pops up, your machine is effectively offline until you return in the morning. Included in the <span class="e">''E''</span> distribution is a dialog box tool, the dialogVowMaker:<br />
<br />
<font color="#ff0000">code here</font><br />
<br />
====Recursion for send sequencing====<br />
<br />
As discussed earlier, putting a when-catch inside a for loop does not give you any guarantees on which when-catch will activate first: it only guarantees which message is sent first, which is mostly uninteresting. For general-purpose ensured sequencing of resolution, you must use recursion.<br />
<br />
Earlier, we had an example in which we told a list of cars to moveTo the same location. Suppose we wished to ensure that the cars arrived in the order in which they appear in the list:<br />
<br />
<pre><br />
<br />
<nowiki># E sample<br />
def moveAllCarsInSequence(carRcvrs,toX,toY) {<br />
var carI := 0<br />
def moveRemainingCars() {<br />
if (carI < carRcvrs.size()) {<br />
def nextCarRcvr := carRcvrs[carI]<br />
carI += 1<br />
def nameVow := nextCarRcvr <- getName()<br />
when (nameVow,nextCarRcvr <- moveTo(toX,toY)) -> {<br />
println(nameVow + " arrived, next car about to start moving")<br />
} catch e {<br />
println(`car died: $e`)<br />
} finally {moveRemainingCars()}<br />
}<br />
}<br />
moveRemainingCars()<br />
}<br />
# scaffold carmaker<br />
def makeCarRcvr(name) {<br />
def carRcvr {<br />
to getName() :String {return name}<br />
to moveTo(x, y) {<br />
#move the car<br />
}<br />
}<br />
return carRcvr<br />
}<br />
<br />
moveAllCarsInSequence([makeCarRcvr("car1"),makeCarRcvr("car2")],3,4) </nowiki><br />
<br />
</pre><br />
<br />
Inside the moveAllCarsInSequence function, we declare the recursive moveRemainingCars function, then simply invoke moveRemainingCars. The function moveRemainingCars tells the next car to move; once that movement has resolved, moveRemainingCars invokes itself once again.<br />
<br />
In this version of moveAllCarsInSequence, if a car is lost (i.e., its promise to move is broken), the recursion continues. If we wanted to abort after a problem occurred, we would simply delete the call to moveRemainingCars() from the finally clause, and place it in the when clause.<br />
<br />
====sendValve====<br />
<br />
As we have seen, it is possible to spin off large numbers of eventual sends in the blink of an eye. In fact, tossing off vast numbers of such sends can sometimes consume a lot of resources for no good purpose. In the eDesk program at the end of the book, it is possible for the user to request the transfer of whole directories full of files. It would be possible for eDesk to initiate transfer of all of those files simultaneously. But the processing bottleneck is probably bandwidth, so initiating them all at once won't get the whole transfer done any faster, and meanwhile each individual file transfer consumes buffer space once initiated. Starting all the transfers at once could actually slow down the transfer if enough buffers thrash virtual memory.<br />
<br />
What you want is a ''valve'', which can be opened and closed depending on the situation. The program can still set up all the operations in a single swoop, but the valve will constrain the number of such operations that are actually active at a given moment. The sendValve below performs this function.<br />
<br />
<pre><br />
<br />
<nowiki>#sendValve. If you have numerous eventual sends to initiate,<br />
# but initiating them all at once would consume vast resources<br />
# and/or would actually slow down processing, queue the actions <br />
# through a valve.<br />
#An ActionTrio is the list of [obj,"verb",[arguments]]<br />
# that can be used in an E send()<br />
#The actions are guaranteed to be initiated in the sequence in<br />
# which they are placed in the valve, though of course there is<br />
# no guarantee as to which will terminate first (unless you have<br />
# special knowledge of such sequencing outside the valve).<br />
#The numSimultaneousAllowed is the number of actions that can<br />
# be run concurrently through this valve.<br />
<br />
# E sample<br />
def makeSendValve (numSimultaneousAllowed) {<br />
var actionQueue := []<br />
var numRunning := 0<br />
def startNext() {<br />
if (!(actionQueue.size()==0)) {<br />
def [actionTrio, resolver] := actionQueue[0]<br />
actionQueue := actionQueue(1, actionQueue.size())<br />
numRunning += 1<br />
def completeVow := E.send(actionTrio[0],actionTrio[1],actionTrio[2])<br />
resolver.resolve(completeVow)<br />
when (completeVow) -> {<br />
} catch prob {<br />
} finally { <br />
numRunning -= 1 <br />
startNext()<br />
}<br />
}<br />
}<br />
def valve {<br />
to makeActionVow(actionTrio) {<br />
def [completeVow, resolver] := Ref.promise()<br />
actionQueue := actionQueue.with([actionTrio, resolver])<br />
if (numRunning < numSimultaneousAllowed) {<br />
startNext()<br />
}<br />
return completeVow<br />
}<br />
}<br />
return valve<br />
}</nowiki><br />
<br />
</pre><br />
<br />
The sendValve utility, if used with numSimultaneousAllowed==1, can also perform the send sequencing function described earlier. However, the earlier general send sequencing pattern is still useful. For example, if the moveAllCarsInSequence operation should abort upon a encountering a broken resolution, this version of sendValve would not give the correct result.<br />
<br />
====Summoning====<br />
<br />
====Transparent forwarder====<br />
<br />
It is possible to have a network of machines in which the machines do not all have direct access to one another. Suppose in the network with machine A, B, and C, A and C cannot form a direct connection because of network topology, but B can reach everything. If A needs a capability on C, we can put a transparent forwarder on B such that A sends its messages to the forwarder on B:<br />
<br />
<pre><br />
<br />
<nowiki># E sample<br />
def makeTransparentForwarder(representedRcvr) {<br />
def forwarder {<br />
match [verb,args] {E.send(representedRcvr,verb,args)}<br />
}<br />
return forwarder<br />
}</nowiki><br />
<br />
</pre><br />
<br />
Create the forwarder on B, and hand the reference to the forwarder to the appropriate object on A. An example can be found in the eDesk example at the end of the book, in which a forwarder is used if the system detects that two of the file servers are unable to directly connect. Note that the "delegate" keyword does not quite work here: delegate generates immediate calls, not eventual sends. So we had to revert to the more general purpose matching process.<br />
<br />
====acceptOnlyMoreRecentVow====<br />
<br />
Suppose you are periodically requesting a particular piece of information from a far object. Suppose the far object cannot supply the answer in a single game turn, i.e., it must ask other objects for information to get the answer, and so it sends you a promise rather than a result. It is possible for the resolutions of these promises to occur out-of-order, i.e., the resolution on the second request could get to you after the resolution of the third request has already occurred. In this situation, it will appear that the answer to the second request is newer and more up-to-date than the third request.<br />
<br />
<font color="#000000">In this situation, to ensure you are getting only new information, not stale information, use the acceptOnlyMoreRecentVow utility.</font><br />
<br />
<pre><br />
<br />
def acceptOnlyMoreRecentVow<br />
<br />
</pre><br />
<br />
<font color="#ff0000">code here</font><br />
<br />
This situation, while rare, actually arises in the Marketplace example in the Secure Distributed Computing chapter.<br />
<br />
<font color="#ff0000">something about time, the race construct, timeout construct</font><br />
<br />
====eDesk overwrite copy jewel====<br />
<br />
The following is not really a pattern so much as it is an example of all the more novel (i.e., not-like-Java) elements of <span class="e">''E''</span> playing together. The problem, taken from the eDesk program in the Examples at the end of the book, is this:<br />
<br />
example<br />
<br />
===Data Lock===<br />
<br />
Though <span class="e">''E''</span> is safe from deadlock, in which several objects compete for acquisition to a set of resources, there is a related dilemma that <span class="e">''E''</span> programs can encounter. If two or more "when" clauses wait for each other to resolve before they can resolve themselves, this causes ''data lock''. A complete discussion of data lock and why it is far less of a risk than deadlock is beyond the scope of this book. Briefly, both theory and current experience suggest that data locks are more difficult to create in the wild than deadlocks. Furthermore, when datalocks do occur, they are less likely to freeze up important subsystems on a grand scale: unlike the deadlock, they only choke a few "when" clauses, not a set of "critical sections" which have been named "critical" in thread programming for good reasons. And lastly, datalocks are consistent, reproducible, and therefore fixable, unlike the deadlocks that appear mysteriously when some incredibly arcane race condition occurs.<br />
<br />
Here is a simple though foolish example of datalock. It is the E implementation of the English sentence, "This sentence is false."<br />
<br />
<pre><br />
<br />
def sentence := sentence <- not()<br />
<br />
</pre><br />
<br />
In this construction, the sentence will forever be an unresolved promise. As noted above, the failure of this to resolve will not impede the progress of the rest of the program in the least. And it is very reliable: the unresolvability of the promise will appear the first time you run the program, and the second, and in every debugging pass thereafter.<br />
<br />
At the time of this writing, data lock has been seen only once in the wild. In that instance, a widely used object (call it WUO),contained one piece of mutable state and one piece of immutable state. The object that computed new versions of WUO's mutable state needed the immutable part of the state, and requested the current version of WUO from the owner of the object. Unfortunately, the owner knew that a new version was under construction, and was only sending out promises for WUO until the update was resolved. The solution in this case was to refactor WUO. This resulted in an architecture that was cleaner even by normal object design criteria, without regard to the distributed usage behavior. Insufficient data is yet available to assess how commonly this type of solution will suffice.<br />
<br />
<br />
Next Section: [[Walnut/Secure_Distributed_Computing|Secure Distributed Computing]]</div>152.78.239.138