[release-branch.r60] doc: document release.r60.3

««« CL 5267049 / eeca0d4a91a3
doc: document release.r60.3

R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/5267049
»»»

R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/5296041

doc/debugging_with_gdb.html

@@ -0,0 +1,479 @@
+<!-- title Debugging Go Code with GDB -->
+
+<p><i>
+This applies to the 6g toolchain. Gccgo has native gdb support. Besides this
+overview you might want to consult the
+<a href="http://sourceware.org/gdb/current/onlinedocs/gdb/">GDB manual</a>.
+</i></p>
+
+<h2 id="Introduction">Introduction</h2>
+
+<p>
+When you compile and link your Go programs with the 6g/6l or 8g/8l toolchains
+on Linux, Mac OSX or FreeBSD, the resulting binaries contain DWARFv3
+debugging information that recent versions (>7.1) of the GDB debugger can
+use to inspect a live process or a core dump.
+</p>
+
+<p>
+Pass the <code>'-s'</code> flag to the linker to omit the debug information.
+</p>
+
+
+<h3 id="Common_Operations">Common Operations</h3>
+
+<ul>
+<li>
+Show file and line number for code 
+and set breakpoints:
+<pre>(gdb) <b>list</b>
+(gdb) <b>list <i>line</i></b>
+(gdb) <b>list <i>file.go</i>:<i>line</i></b>
+(gdb) <b>break <i>line</i></b>
+(gdb) <b>break <i>file.go</i>:<i>line</i></b>
+(gdb) <b>disas</b></pre>
+</li>
+<li>
+Unwind stack frames:
+<pre>(gdb) <b>bt</b>
+(gdb) <b>frame <i>n</i></b></pre>
+</li>
+<li>
+Show the name, type and location on the stack frame of local variables,
+arguments and return values:
+<pre>(gdb) <b>info locals</b>
+(gdb) <b>info args</b>
+(gdb) <b>p variable</b>
+(gdb) <b>whatis variable</b></pre>
+</li>
+<li>
+Show the name, type and location of global variables:
+<pre>(gdb) <b>info variables <i>regexp</i></b></pre>
+</li>
+</ul>
+
+
+<h3 id="Go_Extensions">Go Extensions</h3>
+
+<p>
+A recent extension mechanism to GDB allows it to load extension scripts for a
+given binary. The tool chain uses this to extend GDB with a handful of
+commands to inspect internals of the runtime code (such as goroutines) and to
+pretty print the built-in map, slice and channel types.
+</p>
+
+<ul>
+<li>
+Pretty printing a string, slice, map, channel or interface:
+<pre>(gdb) <b>p <i>var</i></b></pre>
+</li>
+<li>
+A $len() and $cap() function for strings, slices and maps:
+<pre>(gdb) <b>p $len(<i>var</i>)</b></pre>
+</li>
+<li>
+A function to cast interfaces to their dynamic types:
+<pre>(gdb) <b>p $dtype(<i>var</i>)</b>
+(gdb) <b>iface <i>var</i></b></pre>
+<p class="detail"><b>Known issue:</b> GDB can’t automatically find the dynamic
+type of an interface value if its long name differs from its short name
+(annoying when printing stacktraces, the pretty printer falls back to printing
+the short type name and a pointer).</p>
+</li>
+<li>
+Inspecting goroutines:
+<pre>(gdb) <b>info goroutines</b>
+(gdb) <b>goroutine <i>n</i> <i>cmd</i></b>
+(gdb) <b>help goroutine</b></pre>
+For example:
+<pre>(gdb) <b>goroutine 12 bt</b></pre>
+</li>
+</ul>
+
+<p>
+If you'd like to see how this works, or want to extend it, take a look at <a
+href="/src/pkg/runtime/runtime-gdb.py">src/pkg/runtime/runtime-gdb.py</a> in
+the Go source distribution. It depends on some special magic types
+(<code>hash&lt;T,U&gt;</code>) and variables (<code>runtime.m</code> and
+<code>runtime.g</code>) that the linker
+(<a href="/src/cmd/ld/dwarf.c">src/cmd/ld/dwarf.c</a>) ensures are described in
+the DWARF code.
+</ines
+
+<p>
+If you're interested in what the debugging information looks like, run
+'<code>objdump -W 6.out</code>' and browse through the <code>.debug_*</code>
+sections.
+</p>
+
+
+<h3 id="Known_Issues">Known Issues</h3>
+
+<ol>
+<li>String pretty printing only triggers for type string, not for types derived
+from it.</li>
+<li>Type information is missing for the C parts of the runtime library.</li>
+<li>GDB does not understand Go’s name qualifications and treats
+<code>"fmt.Print"</code> as an unstructured literal with a <code>"."</code>
+that needs to be quoted.  It objects even more strongly to method names of
+the form <code>pkg.(*MyType).Meth</code>.
+<li>All global variables are lumped into package <code>"main"</code>.</li>
+</ol>
+
+<h2 id="Tutorial">Tutorial</h2>
+
+<p>
+In this tutorial we will inspect the binary of the
+<a href="/pkg/regexp/">regexp</a> package's unit tests. To build the binary,
+change to <code>$GOROOT/src/pkg/regexp</code> and run <code>gotest</code>.
+This should produce an executable file named <code>6.out</code>.
+</p>
+
+
+<h3 id="Getting_Started">Getting Started</h3>
+
+<p>
+Launch GDB, debugging <code>6.out</code>:
+</p>
+
+<pre>
+$ <b>gdb 6.out</b>
+GNU gdb (GDB) 7.2-gg8
+Copyright (C) 2010 Free Software Foundation, Inc.
+License GPLv  3+: GNU GPL version 3 or later &lt;http://gnu.org/licenses/gpl.html&gt;
+Type "show copying" and "show warranty" for licensing/warranty details.
+This GDB was configured as "x86_64-linux".
+
+Reading symbols from  /home/user/go/src/pkg/regexp/6.out...
+done.
+Loading Go Runtime support.
+(gdb) 
+</pre>
+
+<p>
+The message <code>"Loading Go Runtime support"</code> means that GDB loaded the
+extension from <code>$GOROOT/src/pkg/runtime/runtime-gdb.py</code>.
+</p>
+
+<p>
+To help GDB find the Go runtime sources and the accompanying support script,
+pass your <code>$GOROOT</code> with the <code>'-d'</code> flag:
+</p>
+
+<pre>
+$ <b>gdb 6.out -d $GOROOT</b>
+</pre>
+
+<p>
+If for some reason GDB still can't find that directory or that script, you can load
+it by hand by telling gdb (assuming you have the go sources in
+<code>~/go/</code>):
+<p>
+
+<pre>
+(gdb) <b>source ~/go/src/pkg/runtime/runtime-gdb.py</b>
+Loading Go Runtime support.
+</pre>
+
+<h3 id="Inspecting_the_source">Inspecting the source</h3>
+
+<p>
+Use the <code>"l"</code> or <code>"list"</code> command to inspect source code.
+</p>
+
+<pre>
+(gdb) <b>l</b>
+</pre>
+
+<p>
+List a specific part of the source parametrizing <code>"list"</code> with a
+function name (it must be qualified with its package name).
+</p>
+
+<pre>
+(gdb) <b>l main.main</b>
+</pre>
+
+<p>
+List a specific file and line number:
+</p>
+
+<pre>
+(gdb) <b>l regexp.go:1</b>
+(gdb) <i># Hit enter to repeat last command. Here, this lists next 10 lines.</i>
+</pre>
+
+
+<h3 id="Naming">Naming</h3>
+
+<p>
+Variable and function names must be qualified with the name of the packages
+they belong to. The <code>Compile</code> function from the <code>regexp</code>
+package is known to GDB as <code>'regexp.Compile'</code>. 
+</p>
+
+<p>
+Methods must be qualified with the name of their receiver types. For example,
+the <code>*Regexp</code> type’s <code>doParse</code> method is known as
+<code>'regexp.*Regexp.doParse'</code>. (Note that the second dot is a "middot,"
+an artifact of Go’s internal representation of methods.)
+</p>
+
+<p>
+Variables that shadow other variables are magically suffixed with a number in the debug info.
+Variables referenced by closures will appear as pointers magically prefixed with '&amp'.
+</p>
+
+<h3 id="Setting_breakpoints">Setting breakpoints</h3>
+
+<p>
+Set a breakpoint at the <code>TestFind</code> function:
+</p>
+
+<pre>
+(gdb) <b>b 'regexp.TestFind'</b>
+Breakpoint 1 at 0x424908: file /home/user/go/src/pkg/regexp/find_test.go, line 148.
+</pre>
+
+<p>
+Run the program:
+</p>
+
+<pre>
+(gdb) <b>run</b>
+Starting program: /home/lvd/g/src/pkg/regexp/6.out 
+
+Breakpoint 1, regexp.TestFind (t=0xf8404a89c0) at /home/user/go/src/pkg/regexp/find_test.go:148
+148	func TestFind(t *testing.T) {
+</pre>
+
+<p>
+Execution has paused at the breakpoint.
+See which goroutines are running, and what they're doing:
+</p>
+
+<pre>
+(gdb) <b>info goroutines</b>
+  1  waiting runtime.gosched
+* 13  running runtime.goexit
+</pre>
+
+<p>
+the one marked with the <code>*</code> is the current goroutine.
+</p>
+
+<h3 id="Inspecting_the_stack">Inspecting the stack</h3>
+
+<p>
+Look at the stack trace for where we’ve paused the program:
+</p>
+
+<pre>
+(gdb) <b>bt</b>  <i># backtrace</i>
+#0  regexp.TestFind (t=0xf8404a89c0) at /home/user/go/src/pkg/regexp/find_test.go:148
+#1  0x000000000042f60b in testing.tRunner (t=0xf8404a89c0, test=0x573720) at /home/user/go/src/pkg/testing/testing.go:156
+#2  0x000000000040df64 in runtime.initdone () at /home/user/go/src/pkg/runtime/proc.c:242
+#3  0x000000f8404a89c0 in ?? ()
+#4  0x0000000000573720 in ?? ()
+#5  0x0000000000000000 in ?? ()
+</pre>
+
+<p>
+The other goroutine, number 1, is stuck in <code>runtime.gosched</code>, blocked on a channel receive:
+</p>
+
+<pre>
+(gdb) <b>goroutine 1 bt</b>
+#0  0x000000000040facb in runtime.gosched () at /home/lvd/g/src/pkg/runtime/proc.c:873
+#1  0x00000000004031c9 in runtime.chanrecv (c=void, ep=void, selected=void, received=void)
+ at  /home/lvd/g/src/pkg/runtime/chan.c:342
+#2  0x0000000000403299 in runtime.chanrecv1 (t=void, c=void) at/home/lvd/g/src/pkg/runtime/chan.c:423
+#3  0x000000000043075b in testing.RunTests (matchString={void (struct string, struct string, bool *, os.Error *)} 0x7ffff7f9ef60, tests=  []testing.InternalTest = {...}) at /home/lvd/g/src/pkg/testing/testing.go:201
+#4  0x00000000004302b1 in testing.Main (matchString={void (struct string, struct string, bool *, os.Error *)} 0x7ffff7f9ef80, tests= []testing.InternalTest = {...}, benchmarks= []testing.InternalBenchmark = {...})
+    at /home/lvd/g/src/pkg/testing/testing.go:168
+#5  0x0000000000400dc1 in main.main () at /home/lvd/g/src/pkg/regexp/_testmain.go:98
+#6  0x00000000004022e7 in runtime.mainstart () at /home/lvd/g/src/pkg/runtime/amd64/asm.s:78
+#7  0x000000000040ea6f in runtime.initdone () at /home/lvd/g/src/pkg/runtime/proc.c:243
+#8  0x0000000000000000 in ?? ()
+</pre>
+
+<p>
+The stack frame shows we’re currently executing the <code>regexp.TestFind</code> function, as expected.
+</p>
+
+<pre>
+(gdb) <b>info frame</b>
+Stack level 0, frame at 0x7ffff7f9ff88:
+ rip = 0x425530 in regexp.TestFind (/home/lvd/g/src/pkg/regexp/find_test.go:148); 
+    saved rip 0x430233
+ called by frame at 0x7ffff7f9ffa8
+ source language minimal.
+ Arglist at 0x7ffff7f9ff78, args: t=0xf840688b60
+ Locals at 0x7ffff7f9ff78, Previous frame's sp is 0x7ffff7f9ff88
+ Saved registers:
+  rip at 0x7ffff7f9ff80
+</pre>
+
+<p>
+The command <code>info locals</code> lists all variables local to the function and their values, but is a bit
+dangerous to use, since it will also try to print uninitialized variables. Uninitialized slices may cause gdb to try
+to print arbitrary large arrays.
+</p>
+
+<p>
+The function’s arguments:
+</p>
+
+<pre>
+(gdb) <b>info args</b>
+t = 0xf840688b60
+</pre>
+
+<p>
+When printing the argument, notice that it’s a pointer to a
+<code>Regexp</code> value. Note that GDB has incorrectly put the <code>*</code>
+on the right-hand side of the type name and made up a 'struct' keyword, in traditional C style.
+</p>
+
+<pre>
+(gdb) <b>p re</b>
+(gdb) p t
+$1 = (struct testing.T *) 0xf840688b60
+(gdb) p t
+$1 = (struct testing.T *) 0xf840688b60
+(gdb) p *t
+$2 = {errors = "", failed = false, ch = 0xf8406f5690}
+(gdb) p *t->ch
+$3 = struct hchan<*testing.T>
+</pre>
+
+<p>
+That <code>struct hchan<*testing.T></code> is the runtime-internal represntation of a channel.  It is currently empty, or gdb would have pretty-printed it's contents.
+</p>
+
+<p>
+Stepping forward:
+</p>
+
+<pre>
+(gdb) <b>n</b>  <i># execute next line</i>
+149             for _, test := range findTests {
+(gdb)    <i># enter is repeat</i>
+150                     re := MustCompile(test.pat)
+(gdb) <b>p test.pat</b>
+$4 = ""
+(gdb) <b>p re</b>
+$5 = (struct regexp.Regexp *) 0xf84068d070
+(gdb) <b>p *re</b>
+$6 = {expr = "", prog = 0xf840688b80, prefix = "", prefixBytes =  []uint8, prefixComplete = true, 
+  prefixRune = 0, cond = 0 '\000', numSubexp = 0, longest = false, mu = {state = 0, sema = 0}, 
+  machine =  []*regexp.machine}
+(gdb) <b>p *re->prog</b>
+$7 = {Inst =  []regexp/syntax.Inst = {{Op = 5 '\005', Out = 0, Arg = 0, Rune =  []int}, {Op = 
+    6 '\006', Out = 2, Arg = 0, Rune =  []int}, {Op = 4 '\004', Out = 0, Arg = 0, Rune =  []int}}, 
+  Start = 1, NumCap = 2}
+</pre>
+
+
+<p>
+We can step into the <code>String</code>function call with <code>"s"</code>:
+</p>
+
+<pre>
+(gdb) <b>s</b>
+regexp.(*Regexp).String (re=0xf84068d070, noname=void) at /home/lvd/g/src/pkg/regexp/regexp.go:97
+97      func (re *Regexp) String() string {
+</pre>
+
+<p>
+Get a stack trace to see where we are:
+</p>
+
+<pre>
+(gdb) <b>bt</b>
+(gdb) bt
+#0  regexp.(*Regexp).String (re=0xf84068d070, noname=void)
+    at /home/lvd/g/src/pkg/regexp/regexp.go:97
+#1  0x0000000000425615 in regexp.TestFind (t=0xf840688b60)
+    at /home/lvd/g/src/pkg/regexp/find_test.go:151
+#2  0x0000000000430233 in testing.tRunner (t=0xf840688b60, test=0x5747b8)
+    at /home/lvd/g/src/pkg/testing/testing.go:156
+#3  0x000000000040ea6f in runtime.initdone () at /home/lvd/g/src/pkg/runtime/proc.c:243
+....
+</pre>
+
+<p>
+Look at the source code:
+</p>
+
+<pre>
+(gdb) <b>l</b>
+92              mu      sync.Mutex
+93              machine []*machine
+94      }
+95
+96      // String returns the source text used to compile the regular expression.
+97      func (re *Regexp) String() string {
+98              return re.expr
+99      }
+100
+101     // Compile parses a regular expression and returns, if successful,
+</pre>
+
+<h3 id="Pretty_Printing">Pretty Printing</h3>
+
+<p>
+GDB's pretty printing mechanism is triggered by regexp matches on type names.  An example for slices:
+</p>
+
+<pre>
+(gdb) <b>p utf</b>
+$22 =  []uint8 = {0 '\000', 0 '\000', 0 '\000', 0 '\000'}
+</pre>
+
+<p>
+Since slices, arrays and strings are not C pointers, GDB can't interpret the subscripting operation for you, but
+you can look inside the runtime representation to do that (tab completion helps here):
+</p>
+<pre>
+
+(gdb) <b>p slc</b>
+$11 =  []int = {0, 0}
+(gdb) <b>p slc-&gt</b><i>&ltTAB&gt</i>
+array  slc    len    
+(gdb) <b>p slc->array</b>
+$12 = (int *) 0xf84057af00
+(gdb) <b>p slc->array[1]</b>
+$13 = 0</pre>
+
+
+
+<p>
+The extension functions $len and $cap work on strings, arrays and slices:
+</p>
+
+<pre>
+(gdb) <b>p $len(utf)</b>
+$23 = 4
+(gdb) <b>p $cap(utf)</b>
+$24 = 4
+</pre>
+
+<p>
+Channels and maps are 'reference' types, which gdb shows as pointers to C++-like types <code>hash&ltint,string&gt*</code>.  Dereferencing will trigger prettyprinting
+</p>
+
+<p>
+Interfaces are represented in the runtime as a pointer to a type descriptor and a pointer to a value.  The Go GDB runtime extension decodes this and automatically triggers pretty printing for the runtime type.  The extension function <code>$dtype</code> decodes the dynamic type for you (examples are taken from a breakpoint at <code>regexp.go</code> line 293.)
+</p>
+
+<pre>
+(gdb) <b>p i</b>
+$4 = {str = "cbb"}
+(gdb) <b>whatis i</b>
+type = regexp.input
+(gdb) <b>p $dtype(i)</b>
+$26 = (struct regexp.inputBytes *) 0xf8400b4930
+(gdb) <b>iface i</b>
+regexp.input: struct regexp.inputBytes *
+</pre>

doc/devel/release.html

@@ -14,6 +14,107 @@ hg pull
 hg update release.r<i>NN</i>
 </pre>
 
+<h2 id="r60">r60 (released 2011/09/07)</h2>
+
+<p>
+The r60 release corresponds to 
+<code><a href="weekly.html#2011-08-17">weekly.2011-08-17</a></code>.
+This section highlights the most significant changes in this release.
+For a more detailed summary, see the
+<a href="weekly.html#2011-08-17">weekly release notes</a>.
+For complete information, see the
+<a href="http://code.google.com/p/go/source/list?r=release-branch.r60">Mercurial change list</a>.
+</p>
+
+<h3 id="r60.lang">Language</h3>
+
+<p>
+An "else" block is now required to have braces except if the body of the "else"
+is another "if". Since gofmt always puts those braces in anyway,
+gofmt-formatted programs will not be affected.
+To fix other programs, run gofmt.
+</p>
+
+<h3 id="r60.pkg">Packages</h3>
+
+<p>
+<a href="/pkg/http/">Package http</a>'s URL parsing and query escaping code
+(such as <code>ParseURL</code> and <code>URLEscape</code>) has been moved to
+the new <a href="/pkg/url/">url package</a>, with several simplifications to
+the names. Client code can be updated automatically with gofix.
+</p>
+
+<p>
+<a href="/pkg/image/">Package image</a> has had significant changes made to the
+<code>Pix</code> field of struct types such as
+<a href="/pkg/image/#RGBA">image.RGBA</a> and
+<a href="/pkg/image/#NRGBA">image.NRGBA</a>.
+The <a href="/pkg/image/#Image">image.Image</a> interface type has not changed,
+though, and you should not need to change your code if you don't explicitly
+refer to <code>Pix</code> fields. For example, if you decode a number of images
+using the <a href="/pkg/image/jpeg/">image/jpeg</a> package, compose them using
+<a href="/pkg/image/draw/">image/draw</a>, and then encode the result using
+<a href="/pkg/img/png">image/png</a>, then your code should still work as
+before.
+If your code <i>does</i> refer to <code>Pix</code> fields see the 
+<a href="/doc/devel/weekly.html#2011-07-19">weekly.2011-07-19</a>
+snapshot notes for how to update your code.
+</p>
+
+<p>
+<a href="/pkg/template/">Package template</a> has been replaced with a new
+templating package (formerly <code>exp/template</code>). The original template
+package is still available as <a href="/pkg/old/template/">old/template</a>.
+The <code>old/template</code> package is deprecated and will be removed.
+The Go tree has been updated to use the new template package. We encourage
+users of the old template package to switch to the new one. Code that uses
+<code>template</code> or <code>exp/template</code> will need to change its
+import lines to <code>"old/template"</code> or <code>"template"</code>,
+respectively.
+</p>
+
+<h3 id="r60.cmd">Tools</h3>
+
+<p>
+<a href="/cmd/goinstall/">Goinstall</a> now uses a new tag selection scheme.
+When downloading or updating, goinstall looks for a tag or branch with the
+<code>"go."</code> prefix that corresponds to the local Go version. For Go
+<code>release.r58</code> it looks for <code>go.r58</code>. For
+<code>weekly.2011-06-03</code> it looks for <code>go.weekly.2011-06-03</code>.
+If the specific <code>go.X</code> tag or branch is not found, it chooses the
+closest earlier version. If an appropriate tag or branch is found, goinstall
+uses that version of the code. Otherwise it uses the default version selected
+by the version control system. Library authors are encouraged to use the
+appropriate tag or branch names in their repositories to make their libraries
+more accessible.
+</p>
+
+<h3 id="r60.minor">Minor revisions</h3>
+
+<p>
+r60.1 includes a 
+<a href="http://code.google.com/p/go/source/detail?r=1824581bf62d">linker
+fix</a>, a pair of
+<a href="http://code.google.com/p/go/source/detail?r=9ef4429c2c64">goplay</a>
+<a href="http://code.google.com/p/go/source/detail?r=d42ed8c3098e">fixes</a>,
+and a <code>json</code> package
+<a href="http://code.google.com/p/go/source/detail?r=d5e97874fe84">fix</a> and
+a new
+<a href="http://code.google.com/p/go/source/detail?r=4f0e6269213f">struct tag
+option</a>.
+</p>
+
+<p>
+r60.2
+<a href="http://code.google.com/p/go/source/detail?r=ff19536042ac">fixes</a>
+a memory leak involving maps.
+</p>
+
+<p>
+r60.3 fixes a
+<a href="http://code.google.com/p/go/source/detail?r=01fa62f5e4e5">reflect bug</a>.
+</p>
+
 <h2 id="r59">r59 (released 2011/08/01)</h2>
 
 <p>
@@ -323,7 +424,7 @@ Remember that gofix will handle the bulk of the rewrites
 necessary for these changes to package APIs.
 </p>
 
-<h3 id="r57.tool">Tools</h3>
+<h3 id="r57.cmd">Tools</h3>
 
 <p><a href="/cmd/gofix/">Gofix</a>, a new command, is described above.</p>
 

doc/devel/weekly.html

@@ -14,7 +14,195 @@ hg pull
 hg update weekly.<i>YYYY-MM-DD</i>
 </pre>
 
-<h2 id="2011-08-17">2011-08-17</h2>
+<h2 id="2011-09-07">2011-09-07</h2>
+
+<pre>
+This weekly snapshot consists of improvements and bug fixes, including fixes
+for issues introduced by escape analysis changes in the gc compiler.
+
+* build: clear execute bit from Go files (thanks Mike Rosset),
+	error out if problem with sudo.bash /usr/local/bin (thanks Mike Rosset).
+* exp/norm: add Reader and Writer,
+	performance improvements of quickSpan.
+* exp/regexp: bug fixes and RE2 tests.
+* exp/template/html: string replacement refactoring,
+	tweaks to js{,_test}.go.
+* gc: add -p flag to catch import cycles earlier,
+	fix label recursion bugs,
+	fix zero-length struct eval,
+	zero stack-allocated slice backing arrays,
+* gc, ld: fix Windows file paths (thanks Hector Chu).
+* go/parser: accept corner cases of signature syntax.
+* gobuilder: ignore _test.go files when looking for docs, more logging.
+* godoc: minor tweaks for App Engine use.
+* gofix: do not convert url in field names (thanks Gustavo Niemeyer).
+* gofmt: indent multi-line signatures.
+* gopprof: regexp fixes (thanks Hector Chu).
+* image/png: check zlib checksum during Decode.
+* libmach: fix incorrect use of memset (thanks Dave Cheney).
+* misc/goplay: fix template output.
+* net: ParseCIDR returns IPNet instead of IPMask (thanks Mikio Hara),
+	sync CIDRMask code, doc.
+* os: use GetFileAttributesEx to implement Stat on windows (thanks Alex Brainman).
+* runtime: fix openbsd 386 raisesigpipe,
+	implement exception handling on windows/amd64 (thanks Hector Chu),
+	test for concurrent channel consumers (thanks Christopher Wedgwood).
+* sort: use heapsort to bail out quicksort (thanks Ziad Hatahet).
+* sync/atomic: add LoadUintptr, add Store functions.
+* syscall: update routing message attributes handling (thanks Mikio Hara).
+* template: fix deadlock,
+	indirect or dereference function arguments if necessary,
+	slightly simplify the test for assignability of arguments.
+* url: handle ; in ParseQuery.
+* websocket: fix incorrect prints found by govet (thanks Robert Hencke).
+</pre>
+
+<h2 id="2011-09-01">2011-09-01</h2>
+
+<pre>
+This weekly contains performance improvements and bug fixes.
+
+The gc compiler now does escape analysis, which improves program performance
+by placing variables on the call stack instead of the heap when it is safe to
+do so.
+
+The container/vector package is deprecated and will be removed at some point
+in the future.
+
+Other changes:
+* archive/tar: support symlinks. (thanks Mike Rosset)
+* big: fix nat.scan bug. (thanks Evan Shaw)
+* bufio: handle a "\r\n" that straddles the buffer.
+	add openbsd.
+	avoid redundant bss declarations.
+	fix unused parameters.
+	fix windows/amd64 build with newest mingw-w64. (thanks Hector Chu)
+* bytes: clarify that NewBuffer is not for beginners.
+* cgo: explain how to free something.
+	fix GoBytes. (thanks Gustavo Niemeyer)
+	fixes callback for windows amd64. (thanks Wei Guangjing)
+	note that CString result must be freed. (thanks Gustavo Niemeyer)
+* cov: remove tautological #defines. (thanks Lucio De Re)
+* dashboard: yet another utf-8 fix.
+* doc/codelab/wiki: fix Makefile.
+* doc/progs: fix windows/amd64. (thanks Jaroslavas Počepko)
+* doc/tmpltohtml: update to new template package.
+* doc: emphasize that environment variables are optional.
+* effective_go: convert to use tmpltohtml.
+* exp/norm: reduced the size of the byte buffer used by reorderBuffer by half by reusing space when combining.
+	a few minor fixes to support the implementation of norm.
+	added implementation for []byte versions of methods.
+* exp/template/html: add some tests for ">" attributes.
+	added handling for URL attributes.
+	differentiate URL-valued attributes (such as href).
+	reworked escapeText to recognize attr boundaries.
+* exp/wingui: made compatible with windows/amd64. (thanks Jaroslavas Počepko)
+* flag: add Parsed, restore Usage.
+* gc: add openbsd.
+	escape analysis.
+	fix build on Plan 9. (thanks Lucio De Re)
+	fix div bug.
+	fix pc/line table. (thanks Julian Phillips)
+	fix some spurious leaks.
+	make static initialization more static.
+	remove JCXZ; add JCXZW, JCXZL, and JCXZQ instructions. (thanks Jaroslavas Počepko)
+	shuffle #includes.
+	simplify escape analysis recursion.
+	tweak and enable escape analysis.
+* go/ast cleanup: base File/PackageExports on FilterFile/FilterPackage code.
+	adjustments to filter function.
+	fix ast.MergePackageFiles to collect infos about imports. (thanks Sebastien Binet)
+	generalize ast.FilterFile.
+* go/build: add test support & use in gotest.
+	separate test imports out when scanning. (thanks Gustavo Niemeyer)
+* go/parser: fix type switch scoping.
+	fix type switch scoping.
+* gob: explain that Debug isn't useful unless it's compiled in.
+* gobuilder: increase log limit.
+* godashboard: fix utf-8 in user names.
+* godoc: first step towards reducing index size.
+	add dummy playground.js to silence godoc warning at start-up.
+	added systematic throttling to indexing goroutine.
+	fix bug in zip.go.
+	support for reading/writing (splitted) index files.
+	use virtual file system when generating package synopses.
+* gofix: forgot to rename the URL type.
+	osopen: fixed=true when changing O_CREAT. (thanks Tarmigan Casebolt)
+* goinstall: error out with paths that end with '/'. (thanks Tarmigan Casebolt)
+	report lack of $GOPATH on errors. (thanks Gustavo Niemeyer)
+	select the tag that is closest to runtime.Version.
+* gotry: add missing $. (thanks Tarmigan Casebolt)
+* http: add MaxBytesReader to limit request body size.
+	add file protocol transport.
+	adjust test threshold for larger suse buffers.
+	delete error kludge.
+	on invalid request, send 400 response.
+	return 413 instead of 400 when the request body is too large. (thanks Dave Cheney)
+	support setting Transport's TLS client config.
+* image/tiff: add a decode benchmark. (thanks Benny Siegert)
+	decoder optimization. (thanks Benny Siegert)
+* image: add PalettedImage interface, and make image/png recognize it. (thanks Jaroslavas Počepko)
+* io: add TeeReader. (thanks Hector Chu)
+* json: add struct tag option to wrap literals in strings.
+	calculate Offset for Indent correctly. (thanks Jeff Hodges)
+	fix decode bug with struct tag names with ,opts being ignored.
+* ld: handle Plan 9 ar format. (thanks Lucio De Re)
+	remove duplicate bss definitions.
+* libmach: support reading symbols from Windows .exe for nm. (thanks Mateusz Czapliński)
+* math: fix Pow10 loop. (thanks Volker Dobler)
+* mime: ParseMediaType returns os.Error now, not a nil map.
+	media type formatter. (thanks Pascal S. de Kloe)
+	text charset defaults. (thanks Pascal S. de Kloe)
+* misc/dashboard: remove limit for json package list.
+* misc/emacs: refine label detection.
+* net: add ParseMAC function. (thanks Paul Borman)
+	change the internal form of IPMask for IPv4. (thanks Mikio Hara)
+	disable "tcp" test on openbsd.
+	fix windows build. (thanks Alex Brainman)
+	join and leave a IPv6 group address, on a specific interface. (thanks Mikio Hara)
+	make use of IPv4len, IPv6len. (thanks Mikio Hara)
+	move internal string manipulation routines to parse.go. (thanks Mikio Hara)
+* os: disable Hostname test on OpenBSD.
+	fix WNOHANG Waitmsg. (thanks Gustavo Niemeyer)
+* reflect: add Value.Bytes, Value.SetBytes methods.
+* rpc: add benchmark for async rpc calls.
+* runtime: add openbsd 386 defs.h.
+	add runtime support for openbsd 386.
+	add runtime· prefix to showframe.
+	ctrlhandler for windows amd64. (thanks Wei Guangjing)
+	fix stack cleanup on windows/amd64. (thanks Hector Chu)
+	fix void warnings.
+	go interface to cdecl calbacks. (thanks Jaroslavas Počepko)
+	handle string + char literals in goc2c.
+	make arm work on Ubuntu Natty qemu.
+	openbsd thread tweaks.
+	simplify stack traces.
+	speed up cgo calls. (thanks Alex Brainman)
+	use cgo runtime functions to call windows syscalls. (thanks Alex Brainman)
+	windows/amd64 callbacks fixed and syscall fixed to allow using it in callbacks. (thanks Jaroslavas Počepko)
+* strconv: put decimal on stack.
+* spec: update section on Implementation Differences.
+* syscall: SOMAXCONN should be 0x7fffffff at winsock2. (thanks Yasuhiro Matsumoto)
+	add openbsd 386.
+	handle RTM_NEWROUTE in ParseNetlinkRouteAttr on Linux. (thanks Albert Strasheim)
+	handle routing entry in ParseRoutingSockaddr on BSD variants. (thanks Mikio Hara)
+	openbsd amd64 syscall support.
+	use the vdso page on linux x86 for faster syscalls instead of int $0x80. (thanks Yuval Pavel Zholkover)
+* template/parse: give if, range, and with a common representation.
+* template: grammar fix for template documentation. (thanks Bill Neubauer)
+	range over channel.
+	remove else and end nodes from public view.
+* test: put GOROOT/bin before all others in run.
+* time: fix Plan 9 build. (thanks Fazlul Shahriar)
+	fix zone during windows test.
+* type switches: test for pathological case.
+* version.bash: update VERSION on -save if already present. (thanks Gustavo Niemeyer)
+* websocket: implements new version of WebSocket protocol. (thanks Fumitoshi Ukai)
+* windows/386: clean stack after syscall. (thanks Jaroslavas Počepko)
+* xml: marshal "parent>child" tags correctly. (thanks Ross Light)
+</pre>
+
+<h2 id="2011-08-17">2011-08-17 (<a href="release.html#r60">base for r60</a>)</h2>
 
 <pre>
 This weekly contains some package re-shuffling. Users of the http and

doc/docs.html

@@ -16,6 +16,16 @@ Once you've learned a little about the language,
 idioms of programming in Go.
 </p>
 
+<h3 id="go_tour"><a href="http://go-tour.appspot.com/">A Tour of Go</a></h3>
+<p>
+An interactive introduction to Go in three sections.
+The first section covers basic syntax and data structures; the second discusses
+methods and interfaces; and the third introduces Go's concurrency primitives.
+Each section concludes with a few exercises so you can practice what you've
+learned. You can <a href="http://go-tour.appspot.com/">take the tour online</a> or
+<a href="http://code.google.com/p/go-tour/">install it locally</a>.
+</p>
+
 <h3 id="orig_tutorial"><a href="go_tutorial.html">A Tutorial for the Go Programming Language</a></h3>
 <p>
 The first tutorial. An introductory text that touches upon several core
@@ -29,9 +39,9 @@ Slides from a 3-day course about the Go programming language.
 A more thorough introduction than the tutorial.
 </p>
 <ul>
-<li><a href="GoCourseDay1.pdf">Day 1: Basics</a> <small>[270KB PDF]</small>
-<li><a href="GoCourseDay2.pdf">Day 2: Types, Methods, Interfaces</a> <small>[270KB PDF]</small>
-<li><a href="GoCourseDay3.pdf">Day 3: Concurrency and Communication</a> <small>[180KB PDF]</small>
+<li><a href="GoCourseDay1.pdf">Day 1: Basics</a> <small>[270KB PDF]</small></li>
+<li><a href="GoCourseDay2.pdf">Day 2: Types, Methods, Interfaces</a> <small>[270KB PDF]</small></li>
+<li><a href="GoCourseDay3.pdf">Day 3: Concurrency and Communication</a> <small>[180KB PDF]</small></li>
 </ul>
 
 <h3 id="effective_go"><a href="effective_go.html">Effective Go</a></h3>
@@ -62,12 +72,53 @@ and closures.
 <p>
 Guided tours of Go programs. 
 </p>
+<ul>
+<li><a href="/doc/codewalk/functions">First-Class Functions in Go</a></li>
+<li><a href="/doc/codewalk/markov">Generating arbitrary text: a Markov chain algorithm</a></li>
+<li><a href="/doc/codewalk/sharemem">Share Memory by Communicating</a></li>
+</ul>
 
 <h3 id="go_for_cpp_programmers"><a href="go_for_cpp_programmers.html">Go for C++ Programmers</a></h3>
 <p>
 An introduction to Go for C++ programmers.
 </p>
 
+<h2 id="articles">Go Articles</h2>
+
+<p>
+Notable articles from the <a href="http://blog.golang.org/">Go Blog</a>.
+</p>
+
+<h3>Language</h3>
+
+<ul>
+<li><a href="http://blog.golang.org/2010/04/json-rpc-tale-of-interfaces.html">JSON-RPC: a tale of interfaces</a></li>
+<li><a href="http://blog.golang.org/2010/07/gos-declaration-syntax.html">Go's Declaration Syntax</a></li>
+<li><a href="http://blog.golang.org/2010/08/defer-panic-and-recover.html">Defer, Panic, and Recover</a></li>
+<li><a href="http://blog.golang.org/2010/09/go-concurrency-patterns-timing-out-and.html">Go Concurrency Patterns: Timing out, moving on</a></li>
+<li><a href="http://blog.golang.org/2011/01/go-slices-usage-and-internals.html">Go Slices: usage and internals</a></li>
+<li><a href="http://blog.golang.org/2011/05/gif-decoder-exercise-in-go-interfaces.html">A GIF decoder: an exercise in Go interfaces</a></li>
+<li><a href="http://blog.golang.org/2011/07/error-handling-and-go.html">Error Handling and Go</a></li>
+</ul>
+
+<h3>Packages</h3>
+
+<ul>
+<li><a href="http://blog.golang.org/2011/01/json-and-go.html">JSON and Go</a> - using the <a href="/pkg/json/">json</a> package.</li>
+<li><a href="http://blog.golang.org/2011/03/gobs-of-data.html">Gobs of data</a> - the design and use of the <a href="/pkg/gob/">gob</a> package.</li>
+<li><a href="http://blog.golang.org/2011/09/laws-of-reflection.html">The Laws of Reflection</a> - the fundamentals of the <a href="/pkg/reflect/">reflect</a> package.</li>
+<li><a href="http://blog.golang.org/2011/09/go-image-package.html">The Go image package</a> - the fundamentals of the <a href="/pkg/image/">image</a> package.</li>
+<li><a href="http://blog.golang.org/2011/09/go-imagedraw-package.html">The Go image/draw package</a> - the fundamentals of the <a href="/pkg/image/draw/">image/draw</a> package.</li>
+</ul>
+
+<h3>Tools</h3>
+
+<ul>
+<li><a href="http://blog.golang.org/2011/03/c-go-cgo.html">C? Go? Cgo!</a> - linking against C code with <a href="/cmd/cgo/">cgo</a>.</li>
+<li><a href="http://blog.golang.org/2011/03/godoc-documenting-go-code.html">Godoc: documenting Go code</a> - writing good documentation for <a href="/cmd/godoc/">godoc</a>.</li>
+<li><a href="http://blog.golang.org/2011/06/profiling-go-programs.html">Profiling Go Programs</a></li>
+</ul>
+
 <h2 id="tutorials_nonenglish">Non-English Documentation</h2>
 
 <h3 id="docs_be">Belarusian &mdash; Беларуская</h3>
@@ -140,6 +191,11 @@ one goroutine can be guaranteed to observe values produced by writes to the
 same variable in a different goroutine.
 </p>
 
+<h3 id="debugging_with_gdb"><a href="debugging_with_gdb.html">Debugging Go Code with GDB</a></h3>
+<p>
+Using GDB to debug Go programs.
+</p>
+
 <h2 id="videos_talks">Videos and Talks</h2>
 
 <h3 id="writing_web_apps"><a href="http://www.youtube.com/watch?v=-i0hat7pdpk">Writing Web Apps in Go</a></h3>

doc/go_tutorial.html

@@ -95,7 +95,7 @@ Here's how to compile and run our program.  With <code>6g</code>, say,
 <pre>
 $ 6g helloworld.go  # compile; object goes into helloworld.6
 $ 6l helloworld.6   # link; output goes into 6.out
-$ 6.out
+$ ./6.out
 Hello, world; or Καλημέρα κόσμε; or こんにちは 世界
 $
 </pre>
@@ -544,13 +544,12 @@ composite literal, as is done here in the <code>return</code> statement from <co
 <p>
 We can use the factory to construct some familiar, exported variables of type <code>*File</code>:
 <p>
-<pre><!--{{code "progs/file.go" `/var/` `/^.$/`}}
+<pre><!--{{code "progs/file.go" `/var/` `/^\)/`}}
 -->var (
     Stdin  = newFile(syscall.Stdin, &#34;/dev/stdin&#34;)
     Stdout = newFile(syscall.Stdout, &#34;/dev/stdout&#34;)
     Stderr = newFile(syscall.Stderr, &#34;/dev/stderr&#34;)
 )
-
 </pre>
 <p>
 The <code>newFile</code> function was not exported because it's internal. The proper,

doc/go_tutorial.tmpl

@@ -87,7 +87,7 @@ Here's how to compile and run our program.  With <code>6g</code>, say,
 <pre>
 $ 6g helloworld.go  # compile; object goes into helloworld.6
 $ 6l helloworld.6   # link; output goes into 6.out
-$ 6.out
+$ ./6.out
 Hello, world; or Καλημέρα κόσμε; or こんにちは 世界
 $
 </pre>
@@ -470,7 +470,7 @@ composite literal, as is done here in the <code>return</code> statement from <co
 <p>
 We can use the factory to construct some familiar, exported variables of type <code>*File</code>:
 <p>
-{{code "progs/file.go" `/var/` `/^.$/`}}
+{{code "progs/file.go" `/var/` `/^\)/`}}
 <p>
 The <code>newFile</code> function was not exported because it's internal. The proper,
 exported factory to use is <code>OpenFile</code> (we'll explain that name in a moment):

doc/install.html

@@ -257,7 +257,8 @@ the process of building and testing Go programs.
 <h2 id="next">What's next</h2>
 
 <p>
-Start by reading the <a href="go_tutorial.html">Go Tutorial</a>.
+Start by taking <a href="http://code.google.com/p/go-tour/">A Tour of Go</a>
+or reading the <a href="go_tutorial.html">Go Tutorial</a>.
 </p>
 
 <p>

doc/root.html

@@ -49,10 +49,13 @@ google.setOnLoadCallback(loadFeed);
         It's a fast, statically typed, compiled language that feels like a 
         dynamically typed, interpreted language.
       </p>
-      <h2>Check it out!</h2>
+      <h2>
+        Get started now with
+	<a target="_blank" href="http://go-tour.appspot.com/">A Tour of Go</a>.
+      </h2>
       <p>
       <div class="how">[<a href="/doc/playground.html">How does this work?</a>]</div>
-      <a href="/doc/install.html">Install Go now</a>, or try it right here in your browser:</p>
+      Or try it right here in your browser:</p>
       <div id="playground" class="small"></div>
       <script src="/doc/play/playground.js"></script>
     </div>

misc/goplay/goplay.go

@@ -145,7 +145,7 @@ func run(cmd ...string) ([]byte, os.Error) {
 var frontPage = template.Must(template.New("frontPage").Parse(frontPageText)) // HTML template
 var output = template.Must(template.New("output").Parse(outputText))          // HTML template
 
-var outputText = `<pre>{{html .}}</pre>`
+var outputText = `<pre>{{printf "%s" . |html}}</pre>`
 
 var frontPageText = `<!doctype html>
 <html>
@@ -256,7 +256,7 @@ function compileUpdate() {
 </head>
 <body>
 <table width="100%"><tr><td width="60%" valign="top">
-<textarea autofocus="true" id="edit" spellcheck="false" onkeydown="keyHandler(event);" onkeyup="autocompile();">{{html .}}</textarea>
+<textarea autofocus="true" id="edit" spellcheck="false" onkeydown="keyHandler(event);" onkeyup="autocompile();">{{printf "%s" . |html}}</textarea>
 <div class="hints">
 (Shift-Enter to compile and run.)&nbsp;&nbsp;&nbsp;&nbsp;
 <input type="checkbox" id="autocompile" value="checked" /> Compile and run after each keystroke

src/cmd/cgo/doc.go

@@ -92,5 +92,8 @@ process of using cgo.  See $GOROOT/misc/cgo/stdio and
 $GOROOT/misc/cgo/gmp for examples.
 
 Cgo does not yet work with gccgo.
+
+See "C? Go? Cgo!" for an introduction to using cgo:
+http://blog.golang.org/2011/03/c-go-cgo.html
 */
 package documentation

src/cmd/gc/obj.c

@@ -156,13 +156,14 @@ outhist(Biobuf *b)
 					outzfile(b, p+1);
 				} else {
 					// relative name, like dir/file.go
-					if(h->offset == 0 && pathname && pathname[0] == '/') {
+					if(h->offset >= 0 && pathname && pathname[0] == '/') {
 						zfile(b, "/", 1);	// leading "/"
 						outzfile(b, pathname+1);
 					}
 					outzfile(b, p);
 				}
 			}
+		
 		}
 		zhist(b, h->line, h->offset);
 	}

src/cmd/godoc/doc.go

@@ -126,5 +126,8 @@ one may run godoc as follows:
 
 	godoc -http=:6060 -zip=go.zip -goroot=$HOME/go
 
+
+See "Godoc: documenting Go code" for how to write good comments for godoc:
+http://blog.golang.org/2011/03/godoc-documenting-go-code.html
 */
 package documentation

src/cmd/gofix/osopen.go

@@ -31,6 +31,7 @@ func osopen(f *ast.File) bool {
 		// Rename O_CREAT to O_CREATE.
 		if expr, ok := n.(ast.Expr); ok && isPkgDot(expr, "os", "O_CREAT") {
 			expr.(*ast.SelectorExpr).Sel.Name = "O_CREATE"
+			fixed = true
 			return
 		}
 

src/cmd/gofix/osopen_test.go

@@ -54,6 +54,29 @@ func f() {
 	os.OpenFile(a, os.O_SURPRISE|os.O_CREATE, 0666)
 	_ = os.O_CREATE
 }
+`,
+	},
+	{
+		Name: "osopen.1",
+		In: `package main
+
+import (
+	"os"
+)
+
+func f() {
+	_ = os.O_CREAT
+}
+`,
+		Out: `package main
+
+import (
+	"os"
+)
+
+func f() {
+	_ = os.O_CREATE
+}
 `,
 	},
 }

src/cmd/gofix/url.go

@@ -27,6 +27,7 @@ func init() {
 }
 
 var urlRenames = []struct{ in, out string }{
+	{"URL", "URL"},
 	{"ParseURL", "Parse"},
 	{"ParseURLReference", "ParseWithReference"},
 	{"ParseQuery", "ParseQuery"},
@@ -45,7 +46,12 @@ func url(f *ast.File) bool {
 	fixed := false
 
 	// Update URL code.
+	var skip interface{}
 	urlWalk := func(n interface{}) {
+		if n == skip {
+			skip = nil
+			return
+		}
 		// Is it an identifier?
 		if ident, ok := n.(*ast.Ident); ok && ident.Name == "url" {
 			ident.Name = "url_"
@@ -56,6 +62,12 @@ func url(f *ast.File) bool {
 			fixed = urlDoFields(fn.Params) || fixed
 			fixed = urlDoFields(fn.Results) || fixed
 		}
+		// U{url: ...} is likely a struct field.
+		if kv, ok := n.(*ast.KeyValueExpr); ok {
+			if ident, ok := kv.Key.(*ast.Ident); ok && ident.Name == "url" {
+				skip = ident
+			}
+		}
 	}
 
 	// Fix up URL code and add import, at most once.
@@ -63,7 +75,7 @@ func url(f *ast.File) bool {
 		if fixed {
 			return
 		}
-		walk(f, urlWalk)
+		walkBeforeAfter(f, urlWalk, nop)
 		addImport(f, "url")
 		fixed = true
 	}

src/cmd/gofix/url_test.go

@@ -18,6 +18,7 @@ import (
 )
 
 func f() {
+	var _ http.URL
 	http.ParseURL(a)
 	http.ParseURLReference(a)
 	http.ParseQuery(a)
@@ -33,6 +34,7 @@ func f() {
 import "url"
 
 func f() {
+	var _ url.URL
 	url.Parse(a)
 	url.ParseWithReference(a)
 	url.ParseQuery(a)
@@ -78,10 +80,15 @@ import (
 	"http"
 )
 
+type U struct{ url int }
+type M map[int]int
+
 func f() {
 	http.ParseURL(a)
 	var url = 23
 	url, x := 45, y
+	_ = U{url: url}
+	_ = M{url + 1: url}
 }
 
 func g(url string) string {
@@ -96,10 +103,15 @@ func h() (url string) {
 
 import "url"
 
+type U struct{ url int }
+type M map[int]int
+
 func f() {
 	url.Parse(a)
 	var url_ = 23
 	url_, x := 45, y
+	_ = U{url: url_}
+	_ = M{url_ + 1: url_}
 }
 
 func g(url_ string) string {

src/cmd/goinstall/Makefile

@@ -11,3 +11,9 @@ GOFILES=\
 	make.go\
 
 include ../../Make.cmd
+
+test:
+	gotest
+
+testshort:
+	gotest -test.short

src/cmd/goinstall/doc.go

@@ -94,8 +94,11 @@ attempt to fetch updates.  The -u flag changes this behavior,
 causing goinstall to update all remote packages encountered during
 the installation.
 
-When downloading or updating, goinstall first looks for a tag or branch
-named "release".  If there is one, it uses that version of the code.
+When downloading or updating, goinstall looks for a tag with the "go." prefix
+that corresponds to the local Go version. For Go "release.r58" it looks for a
+tag named "go.r58". For "weekly.2011-06-03" it looks for "go.weekly.2011-06-03".
+If the specific "go.X" tag is not found, it chooses the closest earlier version.
+If an appropriate tag is found, goinstall uses that version of the code.
 Otherwise it uses the default version selected by the version control
 system, typically HEAD for git, tip for Mercurial.
 

src/cmd/goinstall/download.go

@@ -7,12 +7,15 @@
 package main
 
 import (
+	"bytes"
 	"exec"
 	"fmt"
 	"http"
 	"os"
 	"path/filepath"
 	"regexp"
+	"runtime"
+	"strconv"
 	"strings"
 )
 
@@ -36,22 +39,21 @@ func maybeReportToDashboard(path string) {
 // a vcs represents a version control system
 // like Mercurial, Git, or Subversion.
 type vcs struct {
-	name              string
-	cmd               string
-	metadir           string
-	checkout          string
-	clone             string
-	update            string
-	updateReleaseFlag string
-	pull              string
-	pullForceFlag     string
-	log               string
-	logLimitFlag      string
-	logReleaseFlag    string
-	check             string
-	protocols         []string
-	suffix            string
-	defaultHosts      []host
+	name          string
+	cmd           string
+	metadir       string
+	checkout      string
+	clone         string
+	update        string
+	updateRevFlag string
+	pull          string
+	pullForceFlag string
+	tagList       string
+	tagListRe     *regexp.Regexp
+	check         string
+	protocols     []string
+	suffix        string
+	defaultHosts  []host
 }
 
 type host struct {
@@ -61,20 +63,18 @@ type host struct {
 }
 
 var hg = vcs{
-	name:              "Mercurial",
-	cmd:               "hg",
-	metadir:           ".hg",
-	checkout:          "checkout",
-	clone:             "clone",
-	update:            "update",
-	updateReleaseFlag: "release",
-	pull:              "pull",
-	log:               "log",
-	logLimitFlag:      "-l1",
-	logReleaseFlag:    "-rrelease",
-	check:             "identify",
-	protocols:         []string{"https", "http"},
-	suffix:            ".hg",
+	name:      "Mercurial",
+	cmd:       "hg",
+	metadir:   ".hg",
+	checkout:  "checkout",
+	clone:     "clone",
+	update:    "update",
+	pull:      "pull",
+	tagList:   "tags",
+	tagListRe: regexp.MustCompile("([^ ]+)[^\n]+\n"),
+	check:     "identify",
+	protocols: []string{"https", "http"},
+	suffix:    ".hg",
 	defaultHosts: []host{
 		{regexp.MustCompile(`^([a-z0-9\-]+\.googlecode\.com/hg)(/[a-z0-9A-Z_.\-/]*)?$`), "https", ""},
 		{regexp.MustCompile(`^(bitbucket\.org/[a-z0-9A-Z_.\-]+/[a-z0-9A-Z_.\-]+)(/[a-z0-9A-Z_.\-/]*)?$`), "http", ""},
@@ -82,20 +82,18 @@ var hg = vcs{
 }
 
 var git = vcs{
-	name:              "Git",
-	cmd:               "git",
-	metadir:           ".git",
-	checkout:          "checkout",
-	clone:             "clone",
-	update:            "pull",
-	updateReleaseFlag: "release",
-	pull:              "fetch",
-	log:               "show-ref",
-	logLimitFlag:      "",
-	logReleaseFlag:    "release",
-	check:             "ls-remote",
-	protocols:         []string{"git", "https", "http"},
-	suffix:            ".git",
+	name:      "Git",
+	cmd:       "git",
+	metadir:   ".git",
+	checkout:  "checkout",
+	clone:     "clone",
+	update:    "pull",
+	pull:      "fetch",
+	tagList:   "tag",
+	tagListRe: regexp.MustCompile("([^\n]+)\n"),
+	check:     "ls-remote",
+	protocols: []string{"git", "https", "http"},
+	suffix:    ".git",
 	defaultHosts: []host{
 		{regexp.MustCompile(`^([a-z0-9\-]+\.googlecode\.com/git)(/[a-z0-9A-Z_.\-/]*)?$`), "https", ""},
 		{regexp.MustCompile(`^(github\.com/[a-z0-9A-Z_.\-]+/[a-z0-9A-Z_.\-]+)(/[a-z0-9A-Z_.\-/]*)?$`), "http", ".git"},
@@ -103,40 +101,35 @@ var git = vcs{
 }
 
 var svn = vcs{
-	name:              "Subversion",
-	cmd:               "svn",
-	metadir:           ".svn",
-	checkout:          "checkout",
-	clone:             "checkout",
-	update:            "update",
-	updateReleaseFlag: "release",
-	log:               "log",
-	logLimitFlag:      "-l1",
-	logReleaseFlag:    "release",
-	check:             "info",
-	protocols:         []string{"https", "http", "svn"},
-	suffix:            ".svn",
+	name:      "Subversion",
+	cmd:       "svn",
+	metadir:   ".svn",
+	checkout:  "checkout",
+	clone:     "checkout",
+	update:    "update",
+	check:     "info",
+	protocols: []string{"https", "http", "svn"},
+	suffix:    ".svn",
 	defaultHosts: []host{
 		{regexp.MustCompile(`^([a-z0-9\-]+\.googlecode\.com/svn)(/[a-z0-9A-Z_.\-/]*)?$`), "https", ""},
 	},
 }
 
 var bzr = vcs{
-	name:              "Bazaar",
-	cmd:               "bzr",
-	metadir:           ".bzr",
-	checkout:          "update",
-	clone:             "branch",
-	update:            "update",
-	updateReleaseFlag: "-rrelease",
-	pull:              "pull",
-	pullForceFlag:     "--overwrite",
-	log:               "log",
-	logLimitFlag:      "-l1",
-	logReleaseFlag:    "-rrelease",
-	check:             "info",
-	protocols:         []string{"https", "http", "bzr"},
-	suffix:            ".bzr",
+	name:          "Bazaar",
+	cmd:           "bzr",
+	metadir:       ".bzr",
+	checkout:      "update",
+	clone:         "branch",
+	update:        "update",
+	updateRevFlag: "-r",
+	pull:          "pull",
+	pullForceFlag: "--overwrite",
+	tagList:       "tags",
+	tagListRe:     regexp.MustCompile("([^ ]+)[^\n]+\n"),
+	check:         "info",
+	protocols:     []string{"https", "http", "bzr"},
+	suffix:        ".bzr",
 	defaultHosts: []host{
 		{regexp.MustCompile(`^(launchpad\.net/([a-z0-9A-Z_.\-]+(/[a-z0-9A-Z_.\-]+)?|~[a-z0-9A-Z_.\-]+/(\+junk|[a-z0-9A-Z_.\-]+)/[a-z0-9A-Z_.\-]+))(/[a-z0-9A-Z_.\-/]+)?$`), "https", ""},
 	},
@@ -240,20 +233,84 @@ func download(pkg, srcDir string) (public bool, err os.Error) {
 	return
 }
 
-// Try to detect if a "release" tag exists.  If it does, update
-// to the tagged version, otherwise just update the current branch.
-// NOTE(_nil): svn will always fail because it is trying to get
-// the revision history of a file named "release" instead of
-// looking for a commit with a release tag
+// updateRepo gets a list of tags in the repository and
+// checks out the tag closest to the current runtime.Version.
+// If no matching tag is found, it just updates to tip.
 func (v *vcs) updateRepo(dst string) os.Error {
-	if err := quietRun(dst, nil, v.cmd, v.log, v.logLimitFlag, v.logReleaseFlag); err == nil {
-		if err := run(dst, nil, v.cmd, v.checkout, v.updateReleaseFlag); err != nil {
-			return err
-		}
-	} else if err := run(dst, nil, v.cmd, v.update); err != nil {
+	if v.tagList == "" || v.tagListRe == nil {
+		// TODO(adg): fix for svn
+		return run(dst, nil, v.cmd, v.update)
+	}
+
+	// Get tag list.
+	stderr := new(bytes.Buffer)
+	cmd := exec.Command(v.cmd, v.tagList)
+	cmd.Dir = dst
+	cmd.Stderr = stderr
+	b, err := cmd.Output()
+	if err != nil {
+		errorf("%s %s: %s\n", v.cmd, v.tagList, stderr)
 		return err
 	}
-	return nil
+	var tags []string
+	for _, m := range v.tagListRe.FindAllStringSubmatch(string(b), -1) {
+		tags = append(tags, m[1])
+	}
+
+	// Only use the tag component of runtime.Version.
+	ver := strings.Split(runtime.Version(), " ")[0]
+
+	// Select tag.
+	if tag := selectTag(ver, tags); tag != "" {
+		printf("selecting revision %q\n", tag)
+		return run(dst, nil, v.cmd, v.checkout, v.updateRevFlag+tag)
+	}
+
+	// No matching tag found, make default selection.
+	printf("selecting tip\n")
+	return run(dst, nil, v.cmd, v.update)
+}
+
+// selectTag returns the closest matching tag for a given version.
+// Closest means the latest one that is not after the current release.
+// Version "release.rN" matches tags of the form "go.rN" (N being a decimal).
+// Version "weekly.YYYY-MM-DD" matches tags like "go.weekly.YYYY-MM-DD".
+func selectTag(goVersion string, tags []string) (match string) {
+	const rPrefix = "release.r"
+	if strings.HasPrefix(goVersion, rPrefix) {
+		p := "go.r"
+		v, err := strconv.Atof64(goVersion[len(rPrefix):])
+		if err != nil {
+			return ""
+		}
+		var matchf float64
+		for _, t := range tags {
+			if !strings.HasPrefix(t, p) {
+				continue
+			}
+			tf, err := strconv.Atof64(t[len(p):])
+			if err != nil {
+				continue
+			}
+			if matchf < tf && tf <= v {
+				match, matchf = t, tf
+			}
+		}
+	}
+	const wPrefix = "weekly."
+	if strings.HasPrefix(goVersion, wPrefix) {
+		p := "go.weekly."
+		v := goVersion[len(wPrefix):]
+		for _, t := range tags {
+			if !strings.HasPrefix(t, p) {
+				continue
+			}
+			if match < t && t[len(p):] <= v {
+				match = t
+			}
+		}
+	}
+	return match
 }
 
 // checkoutRepo checks out repo into dst using vcs.

src/cmd/goinstall/main.go

@@ -71,6 +71,13 @@ func errorf(format string, args ...interface{}) {
 	logf(format, args...)
 }
 
+func terrorf(tree *build.Tree, format string, args ...interface{}) {
+	if tree != nil && tree.Goroot && os.Getenv("GOPATH") == "" {
+		format = strings.TrimRight(format, "\n") + " ($GOPATH not set)\n"
+	}
+	errorf(format, args...)
+}
+
 func main() {
 	flag.Usage = usage
 	flag.Parse()
@@ -156,7 +163,7 @@ func logPackage(pkg string, tree *build.Tree) (logged bool) {
 	name := filepath.Join(tree.Path, logfile)
 	fout, err := os.OpenFile(name, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0666)
 	if err != nil {
-		logf("%s\n", err)
+		terrorf(tree, "package log: %s\n", err)
 		return false
 	}
 	fmt.Fprintf(fout, "%s\n", pkg)
@@ -182,6 +189,12 @@ func install(pkg, parent string) {
 		visit[pkg] = done
 	}()
 
+	// Don't allow trailing '/'
+	if _, f := filepath.Split(pkg); f == "" {
+		errorf("%s should not have trailing '/'\n", pkg)
+		return
+	}
+
 	// Check whether package is local or remote.
 	// If remote, download or update it.
 	tree, pkg, err := build.FindTree(pkg)
@@ -209,7 +222,7 @@ func install(pkg, parent string) {
 		}
 	}
 	if err != nil {
-		errorf("%s: %v\n", pkg, err)
+		terrorf(tree, "%s: %v\n", pkg, err)
 		return
 	}
 	dir := filepath.Join(tree.SrcDir(), pkg)
@@ -217,11 +230,11 @@ func install(pkg, parent string) {
 	// Install prerequisites.
 	dirInfo, err := build.ScanDir(dir, parent == "")
 	if err != nil {
-		errorf("%s: %v\n", pkg, err)
+		terrorf(tree, "%s: %v\n", pkg, err)
 		return
 	}
 	if len(dirInfo.GoFiles)+len(dirInfo.CgoFiles) == 0 {
-		errorf("%s: package has no files\n", pkg)
+		terrorf(tree, "%s: package has no files\n", pkg)
 		return
 	}
 	for _, p := range dirInfo.Imports {
@@ -237,13 +250,13 @@ func install(pkg, parent string) {
 	if *useMake {
 		err := domake(dir, pkg, tree, dirInfo.IsCommand())
 		if err != nil {
-			errorf("%s: install: %v\n", pkg, err)
+			terrorf(tree, "%s: install: %v\n", pkg, err)
 			return
 		}
 	} else {
 		script, err := build.Build(tree, pkg, dirInfo)
 		if err != nil {
-			errorf("%s: install: %v\n", pkg, err)
+			terrorf(tree, "%s: install: %v\n", pkg, err)
 			return
 		}
 		if *nuke {
@@ -257,7 +270,7 @@ func install(pkg, parent string) {
 			if script.Stale() {
 				printf("%s: install\n", pkg)
 				if err := script.Run(); err != nil {
-					errorf("%s: install: %v\n", pkg, err)
+					terrorf(tree, "%s: install: %v\n", pkg, err)
 					return
 				}
 			} else {

src/cmd/goinstall/tag_test.go

@@ -0,0 +1,73 @@
+// Copyright 2011 The Go Authors.  All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package main
+
+import "testing"
+
+var selectTagTestTags = []string{
+	"go.r58",
+	"go.r58.1",
+	"go.r59",
+	"go.r59.1",
+	"go.r61",
+	"go.r61.1",
+	"go.weekly.2010-01-02",
+	"go.weekly.2011-10-12",
+	"go.weekly.2011-10-12.1",
+	"go.weekly.2011-10-14",
+	"go.weekly.2011-11-01",
+	// these should be ignored:
+	"release.r59",
+	"release.r59.1",
+	"release",
+	"weekly.2011-10-12",
+	"weekly.2011-10-12.1",
+	"weekly",
+	"foo",
+	"bar",
+	"go.f00",
+	"go!r60",
+	"go.1999-01-01",
+}
+
+var selectTagTests = []struct {
+	version  string
+	selected string
+}{
+	{"release.r57", ""},
+	{"release.r58.2", "go.r58.1"},
+	{"release.r59", "go.r59"},
+	{"release.r59.1", "go.r59.1"},
+	{"release.r60", "go.r59.1"},
+	{"release.r60.1", "go.r59.1"},
+	{"release.r61", "go.r61"},
+	{"release.r66", "go.r61.1"},
+	{"weekly.2010-01-01", ""},
+	{"weekly.2010-01-02", "go.weekly.2010-01-02"},
+	{"weekly.2010-01-02.1", "go.weekly.2010-01-02"},
+	{"weekly.2010-01-03", "go.weekly.2010-01-02"},
+	{"weekly.2011-10-12", "go.weekly.2011-10-12"},
+	{"weekly.2011-10-12.1", "go.weekly.2011-10-12.1"},
+	{"weekly.2011-10-13", "go.weekly.2011-10-12.1"},
+	{"weekly.2011-10-14", "go.weekly.2011-10-14"},
+	{"weekly.2011-10-14.1", "go.weekly.2011-10-14"},
+	{"weekly.2011-11-01", "go.weekly.2011-11-01"},
+	{"weekly.2014-01-01", "go.weekly.2011-11-01"},
+	{"weekly.3000-01-01", "go.weekly.2011-11-01"},
+	// faulty versions:
+	{"release.f00", ""},
+	{"weekly.1999-01-01", ""},
+	{"junk", ""},
+	{"", ""},
+}
+
+func TestSelectTag(t *testing.T) {
+	for _, c := range selectTagTests {
+		selected := selectTag(c.version, selectTagTestTags)
+		if selected != c.selected {
+			t.Errorf("selectTag(%q) = %q, want %q", c.version, selected, c.selected)
+		}
+	}
+}

src/cmd/ld/dwarf.c

@@ -1356,7 +1356,7 @@ synthesizemaptypes(DWDie *die)
 				getattr(keytype, DW_AT_name)->data,
 				getattr(valtype, DW_AT_name)->data));
 		copychildren(dwhs, hash_subtable);
-		substitutetype(dwhs, "end", defptrto(dwhe));
+		substitutetype(dwhs, "last", defptrto(dwhe));
 		substitutetype(dwhs, "entry", dwhe);  // todo: []hash_entry with dynamic size
 		newattr(dwhs, DW_AT_byte_size, DW_CLS_CONSTANT,
 			getattr(hash_subtable, DW_AT_byte_size)->value, nil);
@@ -1578,13 +1578,16 @@ addhistfile(char *zentry)
 		histfile[histfilesize++] = "<eof>";
 
 	fname = decodez(zentry);
+//	print("addhistfile %d: %s\n", histfilesize, fname);
 	if (fname == 0)
 		return -1;
+
 	// Don't fill with duplicates (check only top one).
 	if (strcmp(fname, histfile[histfilesize-1]) == 0) {
 		free(fname);
 		return histfilesize - 1;
 	}
+
 	histfile[histfilesize++] = fname;
 	return histfilesize - 1;
 }
@@ -1608,11 +1611,13 @@ finddebugruntimepath(void)
 }
 
 // Go's runtime C sources are sane, and Go sources nest only 1 level,
-// so 16 should be plenty.
+// so a handful would be plenty, if it weren't for the fact that line
+// directives can push an unlimited number of them.
 static struct {
 	int file;
 	vlong line;
-} includestack[16];
+} *includestack;
+static int includestacksize;
 static int includetop;
 static vlong absline;
 
@@ -1629,17 +1634,15 @@ static Linehist *linehist;
 static void
 checknesting(void)
 {
-	int i;
-
 	if (includetop < 0) {
 		diag("dwarf: corrupt z stack");
 		errorexit();
 	}
-	if (includetop >= nelem(includestack)) {
-		diag("dwarf: nesting too deep");
-		for (i = 0; i < nelem(includestack); i++)
-			diag("\t%s", histfile[includestack[i].file]);
-		errorexit();
+	if (includetop >= includestacksize) {
+		includestacksize += 1;
+		includestacksize <<= 2;
+//		print("checknesting: growing to %d\n", includestacksize);
+		includestack = realloc(includestack, includestacksize * sizeof *includestack);	       
 	}
 }
 
@@ -1669,6 +1672,7 @@ inithist(Auto *a)
 	// Clear the history.
 	clearhistfile();
 	includetop = 0;
+	checknesting();
 	includestack[includetop].file = 0;
 	includestack[includetop].line = -1;
 	absline = 0;
@@ -1682,10 +1686,10 @@ inithist(Auto *a)
 	for (; a; a = a->link) {
 		if (a->type == D_FILE) {  // 'z'
 			int f = addhistfile(a->asym->name);
-			if (f < 0) {	   // pop file
+			if (f < 0) {	// pop file
 				includetop--;
 				checknesting();
-			} else if(f != includestack[includetop].file) { // pushed a new file
+			} else {	// pushed a file (potentially same)
 				includestack[includetop].line += a->aoffset - absline;
 				includetop++;
 				checknesting();

src/pkg/Makefile

@@ -77,7 +77,6 @@ DIRS=\
 	encoding/hex\
 	encoding/pem\
 	exec\
-	exp/datafmt\
 	exp/gui\
 	exp/gui/x11\
 	exp/norm\
@@ -240,7 +239,6 @@ NOTEST+=\
 	../cmd/cgo\
 	../cmd/ebnflint\
 	../cmd/godoc\
-	../cmd/goinstall\
 	../cmd/gotest\
 	../cmd/goyacc\
 	../cmd/hgpatch\

src/pkg/exp/datafmt/Makefile

@@ -1,12 +0,0 @@
-# Copyright 2009 The Go Authors. All rights reserved.
-# Use of this source code is governed by a BSD-style
-# license that can be found in the LICENSE file.
-
-include ../../../Make.inc
-
-TARG=exp/datafmt
-GOFILES=\
-	datafmt.go\
-	parser.go\
-
-include ../../../Make.pkg

src/pkg/exp/datafmt/datafmt.go

@@ -1,710 +0,0 @@
-// Copyright 2009 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-/*	Package datafmt implements syntax-directed, type-driven formatting
-	of arbitrary data structures. Formatting a data structure consists of
-	two phases: first, a parser reads a format specification and builds a
-	"compiled" format. Then, the format can be applied repeatedly to
-	arbitrary values. Applying a format to a value evaluates to a []byte
-	containing the formatted value bytes, or nil.
-
-	A format specification is a set of package declarations and format rules:
-
-		Format      = [ Entry { ";" Entry } [ ";" ] ] .
-		Entry       = PackageDecl | FormatRule .
-
-	(The syntax of a format specification is presented in the same EBNF
-	notation as used in the Go language specification. The syntax of white
-	space, comments, identifiers, and string literals is the same as in Go.)
-
-	A package declaration binds a package name (such as 'ast') to a
-	package import path (such as '"go/ast"'). Each package used (in
-	a type name, see below) must be declared once before use.
-
-		PackageDecl = PackageName ImportPath .
-		PackageName = identifier .
-		ImportPath  = string .
-
-	A format rule binds a rule name to a format expression. A rule name
-	may be a type name or one of the special names 'default' or '/'.
-	A type name may be the name of a predeclared type (for example, 'int',
-	'float32', etc.), the package-qualified name of a user-defined type
-	(for example, 'ast.MapType'), or an identifier indicating the structure
-	of unnamed composite types ('array', 'chan', 'func', 'interface', 'map',
-	or 'ptr'). Each rule must have a unique name; rules can be declared in
-	any order.
-
-		FormatRule  = RuleName "=" Expression .
-		RuleName    = TypeName | "default" | "/" .
-		TypeName    = [ PackageName "." ] identifier .
-
-	To format a value, the value's type name is used to select the format rule
-	(there is an override mechanism, see below). The format expression of the
-	selected rule specifies how the value is formatted. Each format expression,
-	when applied to a value, evaluates to a byte sequence or nil.
-
-	In its most general form, a format expression is a list of alternatives,
-	each of which is a sequence of operands:
-
-		Expression  = [ Sequence ] { "|" [ Sequence ] } .
-		Sequence    = Operand { Operand } .
-
-	The formatted result produced by an expression is the result of the first
-	alternative sequence that evaluates to a non-nil result; if there is no
-	such alternative, the expression evaluates to nil. The result produced by
-	an operand sequence is the concatenation of the results of its operands.
-	If any operand in the sequence evaluates to nil, the entire sequence
-	evaluates to nil.
-
-	There are five kinds of operands:
-
-		Operand     = Literal | Field | Group | Option | Repetition .
-
-	Literals evaluate to themselves, with two substitutions. First,
-	%-formats expand in the manner of fmt.Printf, with the current value
-	passed as the parameter. Second, the current indentation (see below)
-	is inserted after every newline or form feed character.
-
-		Literal     = string .
-
-	This table shows string literals applied to the value 42 and the
-	corresponding formatted result:
-
-		"foo"       foo
-		"%x"        2a
-		"x = %d"    x = 42
-		"%#x = %d"  0x2a = 42
-
-	A field operand is a field name optionally followed by an alternate
-	rule name. The field name may be an identifier or one of the special
-	names @ or *.
-
-		Field       = FieldName [ ":" RuleName ] .
-		FieldName   = identifier | "@" | "*" .
-
-	If the field name is an identifier, the current value must be a struct,
-	and there must be a field with that name in the struct. The same lookup
-	rules apply as in the Go language (for instance, the name of an anonymous
-	field is the unqualified type name). The field name denotes the field
-	value in the struct. If the field is not found, formatting is aborted
-	and an error message is returned. (TODO consider changing the semantics
-	such that if a field is not found, it evaluates to nil).
-
-	The special name '@' denotes the current value.
-
-	The meaning of the special name '*' depends on the type of the current
-	value:
-
-		array, slice types   array, slice element (inside {} only, see below)
-		interfaces           value stored in interface
-		pointers             value pointed to by pointer
-
-	(Implementation restriction: channel, function and map types are not
-	supported due to missing reflection support).
-
-	Fields are evaluated as follows: If the field value is nil, or an array
-	or slice element does not exist, the result is nil (see below for details
-	on array/slice elements). If the value is not nil the field value is
-	formatted (recursively) using the rule corresponding to its type name,
-	or the alternate rule name, if given.
-
-	The following example shows a complete format specification for a
-	struct 'myPackage.Point'. Assume the package
-
-		package myPackage  // in directory myDir/myPackage
-		type Point struct {
-			name string;
-			x, y int;
-		}
-
-	Applying the format specification
-
-		myPackage "myDir/myPackage";
-		int = "%d";
-		hexInt = "0x%x";
-		string = "---%s---";
-		myPackage.Point = name "{" x ", " y:hexInt "}";
-
-	to the value myPackage.Point{"foo", 3, 15} results in
-
-		---foo---{3, 0xf}
-
-	Finally, an operand may be a grouped, optional, or repeated expression.
-	A grouped expression ("group") groups a more complex expression (body)
-	so that it can be used in place of a single operand:
-
-		Group       = "(" [ Indentation ">>" ] Body ")" .
-		Indentation = Expression .
-		Body        = Expression .
-
-	A group body may be prefixed by an indentation expression followed by '>>'.
-	The indentation expression is applied to the current value like any other
-	expression and the result, if not nil, is appended to the current indentation
-	during the evaluation of the body (see also formatting state, below).
-
-	An optional expression ("option") is enclosed in '[]' brackets.
-
-		Option      = "[" Body "]" .
-
-	An option evaluates to its body, except that if the body evaluates to nil,
-	the option expression evaluates to an empty []byte. Thus an option's purpose
-	is to protect the expression containing the option from a nil operand.
-
-	A repeated expression ("repetition") is enclosed in '{}' braces.
-
-		Repetition  = "{" Body [ "/" Separator ] "}" .
-		Separator   = Expression .
-
-	A repeated expression is evaluated as follows: The body is evaluated
-	repeatedly and its results are concatenated until the body evaluates
-	to nil. The result of the repetition is the (possibly empty) concatenation,
-	but it is never nil. An implicit index is supplied for the evaluation of
-	the body: that index is used to address elements of arrays or slices. If
-	the corresponding elements do not exist, the field denoting the element
-	evaluates to nil (which in turn may terminate the repetition).
-
-	The body of a repetition may be followed by a '/' and a "separator"
-	expression. If the separator is present, it is invoked between repetitions
-	of the body.
-
-	The following example shows a complete format specification for formatting
-	a slice of unnamed type. Applying the specification
-
-		int = "%b";
-		array = { * / ", " };  // array is the type name for an unnamed slice
-
-	to the value '[]int{2, 3, 5, 7}' results in
-
-		10, 11, 101, 111
-
-	Default rule: If a format rule named 'default' is present, it is used for
-	formatting a value if no other rule was found. A common default rule is
-
-		default = "%v"
-
-	to provide default formatting for basic types without having to specify
-	a specific rule for each basic type.
-
-	Global separator rule: If a format rule named '/' is present, it is
-	invoked with the current value between literals. If the separator
-	expression evaluates to nil, it is ignored.
-
-	For instance, a global separator rule may be used to punctuate a sequence
-	of values with commas. The rules:
-
-		default = "%v";
-		/ = ", ";
-
-	will format an argument list by printing each one in its default format,
-	separated by a comma and a space.
-*/
-package datafmt
-
-import (
-	"bytes"
-	"fmt"
-	"go/token"
-	"io"
-	"os"
-	"reflect"
-	"runtime"
-)
-
-// ----------------------------------------------------------------------------
-// Format representation
-
-// Custom formatters implement the Formatter function type.
-// A formatter is invoked with the current formatting state, the
-// value to format, and the rule name under which the formatter
-// was installed (the same formatter function may be installed
-// under different names). The formatter may access the current state
-// to guide formatting and use State.Write to append to the state's
-// output.
-//
-// A formatter must return a boolean value indicating if it evaluated
-// to a non-nil value (true), or a nil value (false).
-//
-type Formatter func(state *State, value interface{}, ruleName string) bool
-
-// A FormatterMap is a set of custom formatters.
-// It maps a rule name to a formatter function.
-//
-type FormatterMap map[string]Formatter
-
-// A parsed format expression is built from the following nodes.
-//
-type (
-	expr interface{}
-
-	alternatives []expr // x | y | z
-
-	sequence []expr // x y z
-
-	literal [][]byte // a list of string segments, possibly starting with '%'
-
-	field struct {
-		fieldName string // including "@", "*"
-		ruleName  string // "" if no rule name specified
-	}
-
-	group struct {
-		indent, body expr // (indent >> body)
-	}
-
-	option struct {
-		body expr // [body]
-	}
-
-	repetition struct {
-		body, separator expr // {body / separator}
-	}
-
-	custom struct {
-		ruleName string
-		fun      Formatter
-	}
-)
-
-// A Format is the result of parsing a format specification.
-// The format may be applied repeatedly to format values.
-//
-type Format map[string]expr
-
-// ----------------------------------------------------------------------------
-// Formatting
-
-// An application-specific environment may be provided to Format.Apply;
-// the environment is available inside custom formatters via State.Env().
-// Environments must implement copying; the Copy method must return an
-// complete copy of the receiver. This is necessary so that the formatter
-// can save and restore an environment (in case of an absent expression).
-//
-// If the Environment doesn't change during formatting (this is under
-// control of the custom formatters), the Copy function can simply return
-// the receiver, and thus can be very light-weight.
-//
-type Environment interface {
-	Copy() Environment
-}
-
-// State represents the current formatting state.
-// It is provided as argument to custom formatters.
-//
-type State struct {
-	fmt       Format         // format in use
-	env       Environment    // user-supplied environment
-	errors    chan os.Error  // not chan *Error (errors <- nil would be wrong!)
-	hasOutput bool           // true after the first literal has been written
-	indent    bytes.Buffer   // current indentation
-	output    bytes.Buffer   // format output
-	linePos   token.Position // position of line beginning (Column == 0)
-	default_  expr           // possibly nil
-	separator expr           // possibly nil
-}
-
-func newState(fmt Format, env Environment, errors chan os.Error) *State {
-	s := new(State)
-	s.fmt = fmt
-	s.env = env
-	s.errors = errors
-	s.linePos = token.Position{Line: 1}
-
-	// if we have a default rule, cache its expression for fast access
-	if x, found := fmt["default"]; found {
-		s.default_ = x
-	}
-
-	// if we have a global separator rule, cache its expression for fast access
-	if x, found := fmt["/"]; found {
-		s.separator = x
-	}
-
-	return s
-}
-
-// Env returns the environment passed to Format.Apply.
-func (s *State) Env() interface{} { return s.env }
-
-// LinePos returns the position of the current line beginning
-// in the state's output buffer. Line numbers start at 1.
-//
-func (s *State) LinePos() token.Position { return s.linePos }
-
-// Pos returns the position of the next byte to be written to the
-// output buffer. Line numbers start at 1.
-//
-func (s *State) Pos() token.Position {
-	offs := s.output.Len()
-	return token.Position{Line: s.linePos.Line, Column: offs - s.linePos.Offset, Offset: offs}
-}
-
-// Write writes data to the output buffer, inserting the indentation
-// string after each newline or form feed character. It cannot return an error.
-//
-func (s *State) Write(data []byte) (int, os.Error) {
-	n := 0
-	i0 := 0
-	for i, ch := range data {
-		if ch == '\n' || ch == '\f' {
-			// write text segment and indentation
-			n1, _ := s.output.Write(data[i0 : i+1])
-			n2, _ := s.output.Write(s.indent.Bytes())
-			n += n1 + n2
-			i0 = i + 1
-			s.linePos.Offset = s.output.Len()
-			s.linePos.Line++
-		}
-	}
-	n3, _ := s.output.Write(data[i0:])
-	return n + n3, nil
-}
-
-type checkpoint struct {
-	env       Environment
-	hasOutput bool
-	outputLen int
-	linePos   token.Position
-}
-
-func (s *State) save() checkpoint {
-	saved := checkpoint{nil, s.hasOutput, s.output.Len(), s.linePos}
-	if s.env != nil {
-		saved.env = s.env.Copy()
-	}
-	return saved
-}
-
-func (s *State) restore(m checkpoint) {
-	s.env = m.env
-	s.output.Truncate(m.outputLen)
-}
-
-func (s *State) error(msg string) {
-	s.errors <- os.NewError(msg)
-	runtime.Goexit()
-}
-
-// TODO At the moment, unnamed types are simply mapped to the default
-//      names below. For instance, all unnamed arrays are mapped to
-//      'array' which is not really sufficient. Eventually one may want
-//      to be able to specify rules for say an unnamed slice of T.
-//
-
-func typename(typ reflect.Type) string {
-	switch typ.Kind() {
-	case reflect.Array:
-		return "array"
-	case reflect.Slice:
-		return "array"
-	case reflect.Chan:
-		return "chan"
-	case reflect.Func:
-		return "func"
-	case reflect.Interface:
-		return "interface"
-	case reflect.Map:
-		return "map"
-	case reflect.Ptr:
-		return "ptr"
-	}
-	return typ.String()
-}
-
-func (s *State) getFormat(name string) expr {
-	if fexpr, found := s.fmt[name]; found {
-		return fexpr
-	}
-
-	if s.default_ != nil {
-		return s.default_
-	}
-
-	s.error(fmt.Sprintf("no format rule for type: '%s'", name))
-	return nil
-}
-
-// eval applies a format expression fexpr to a value. If the expression
-// evaluates internally to a non-nil []byte, that slice is appended to
-// the state's output buffer and eval returns true. Otherwise, eval
-// returns false and the state remains unchanged.
-//
-func (s *State) eval(fexpr expr, value reflect.Value, index int) bool {
-	// an empty format expression always evaluates
-	// to a non-nil (but empty) []byte
-	if fexpr == nil {
-		return true
-	}
-
-	switch t := fexpr.(type) {
-	case alternatives:
-		// append the result of the first alternative that evaluates to
-		// a non-nil []byte to the state's output
-		mark := s.save()
-		for _, x := range t {
-			if s.eval(x, value, index) {
-				return true
-			}
-			s.restore(mark)
-		}
-		return false
-
-	case sequence:
-		// append the result of all operands to the state's output
-		// unless a nil result is encountered
-		mark := s.save()
-		for _, x := range t {
-			if !s.eval(x, value, index) {
-				s.restore(mark)
-				return false
-			}
-		}
-		return true
-
-	case literal:
-		// write separator, if any
-		if s.hasOutput {
-			// not the first literal
-			if s.separator != nil {
-				sep := s.separator // save current separator
-				s.separator = nil  // and disable it (avoid recursion)
-				mark := s.save()
-				if !s.eval(sep, value, index) {
-					s.restore(mark)
-				}
-				s.separator = sep // enable it again
-			}
-		}
-		s.hasOutput = true
-		// write literal segments
-		for _, lit := range t {
-			if len(lit) > 1 && lit[0] == '%' {
-				// segment contains a %-format at the beginning
-				if lit[1] == '%' {
-					// "%%" is printed as a single "%"
-					s.Write(lit[1:])
-				} else {
-					// use s instead of s.output to get indentation right
-					fmt.Fprintf(s, string(lit), value.Interface())
-				}
-			} else {
-				// segment contains no %-formats
-				s.Write(lit)
-			}
-		}
-		return true // a literal never evaluates to nil
-
-	case *field:
-		// determine field value
-		switch t.fieldName {
-		case "@":
-			// field value is current value
-
-		case "*":
-			// indirection: operation is type-specific
-			switch v := value; v.Kind() {
-			case reflect.Array:
-				if v.Len() <= index {
-					return false
-				}
-				value = v.Index(index)
-
-			case reflect.Slice:
-				if v.IsNil() || v.Len() <= index {
-					return false
-				}
-				value = v.Index(index)
-
-			case reflect.Map:
-				s.error("reflection support for maps incomplete")
-
-			case reflect.Ptr:
-				if v.IsNil() {
-					return false
-				}
-				value = v.Elem()
-
-			case reflect.Interface:
-				if v.IsNil() {
-					return false
-				}
-				value = v.Elem()
-
-			case reflect.Chan:
-				s.error("reflection support for chans incomplete")
-
-			case reflect.Func:
-				s.error("reflection support for funcs incomplete")
-
-			default:
-				s.error(fmt.Sprintf("error: * does not apply to `%s`", value.Type()))
-			}
-
-		default:
-			// value is value of named field
-			var field reflect.Value
-			if sval := value; sval.Kind() == reflect.Struct {
-				field = sval.FieldByName(t.fieldName)
-				if !field.IsValid() {
-					// TODO consider just returning false in this case
-					s.error(fmt.Sprintf("error: no field `%s` in `%s`", t.fieldName, value.Type()))
-				}
-			}
-			value = field
-		}
-
-		// determine rule
-		ruleName := t.ruleName
-		if ruleName == "" {
-			// no alternate rule name, value type determines rule
-			ruleName = typename(value.Type())
-		}
-		fexpr = s.getFormat(ruleName)
-
-		mark := s.save()
-		if !s.eval(fexpr, value, index) {
-			s.restore(mark)
-			return false
-		}
-		return true
-
-	case *group:
-		// remember current indentation
-		indentLen := s.indent.Len()
-
-		// update current indentation
-		mark := s.save()
-		s.eval(t.indent, value, index)
-		// if the indentation evaluates to nil, the state's output buffer
-		// didn't change - either way it's ok to append the difference to
-		// the current indentation
-		s.indent.Write(s.output.Bytes()[mark.outputLen:s.output.Len()])
-		s.restore(mark)
-
-		// format group body
-		mark = s.save()
-		b := true
-		if !s.eval(t.body, value, index) {
-			s.restore(mark)
-			b = false
-		}
-
-		// reset indentation
-		s.indent.Truncate(indentLen)
-		return b
-
-	case *option:
-		// evaluate the body and append the result to the state's output
-		// buffer unless the result is nil
-		mark := s.save()
-		if !s.eval(t.body, value, 0) { // TODO is 0 index correct?
-			s.restore(mark)
-		}
-		return true // an option never evaluates to nil
-
-	case *repetition:
-		// evaluate the body and append the result to the state's output
-		// buffer until a result is nil
-		for i := 0; ; i++ {
-			mark := s.save()
-			// write separator, if any
-			if i > 0 && t.separator != nil {
-				// nil result from separator is ignored
-				mark := s.save()
-				if !s.eval(t.separator, value, i) {
-					s.restore(mark)
-				}
-			}
-			if !s.eval(t.body, value, i) {
-				s.restore(mark)
-				break
-			}
-		}
-		return true // a repetition never evaluates to nil
-
-	case *custom:
-		// invoke the custom formatter to obtain the result
-		mark := s.save()
-		if !t.fun(s, value.Interface(), t.ruleName) {
-			s.restore(mark)
-			return false
-		}
-		return true
-	}
-
-	panic("unreachable")
-	return false
-}
-
-// Eval formats each argument according to the format
-// f and returns the resulting []byte and os.Error. If
-// an error occurred, the []byte contains the partially
-// formatted result. An environment env may be passed
-// in which is available in custom formatters through
-// the state parameter.
-//
-func (f Format) Eval(env Environment, args ...interface{}) ([]byte, os.Error) {
-	if f == nil {
-		return nil, os.NewError("format is nil")
-	}
-
-	errors := make(chan os.Error)
-	s := newState(f, env, errors)
-
-	go func() {
-		for _, v := range args {
-			fld := reflect.ValueOf(v)
-			if !fld.IsValid() {
-				errors <- os.NewError("nil argument")
-				return
-			}
-			mark := s.save()
-			if !s.eval(s.getFormat(typename(fld.Type())), fld, 0) { // TODO is 0 index correct?
-				s.restore(mark)
-			}
-		}
-		errors <- nil // no errors
-	}()
-
-	err := <-errors
-	return s.output.Bytes(), err
-}
-
-// ----------------------------------------------------------------------------
-// Convenience functions
-
-// Fprint formats each argument according to the format f
-// and writes to w. The result is the total number of bytes
-// written and an os.Error, if any.
-//
-func (f Format) Fprint(w io.Writer, env Environment, args ...interface{}) (int, os.Error) {
-	data, err := f.Eval(env, args...)
-	if err != nil {
-		// TODO should we print partial result in case of error?
-		return 0, err
-	}
-	return w.Write(data)
-}
-
-// Print formats each argument according to the format f
-// and writes to standard output. The result is the total
-// number of bytes written and an os.Error, if any.
-//
-func (f Format) Print(args ...interface{}) (int, os.Error) {
-	return f.Fprint(os.Stdout, nil, args...)
-}
-
-// Sprint formats each argument according to the format f
-// and returns the resulting string. If an error occurs
-// during formatting, the result string contains the
-// partially formatted result followed by an error message.
-//
-func (f Format) Sprint(args ...interface{}) string {
-	var buf bytes.Buffer
-	_, err := f.Fprint(&buf, nil, args...)
-	if err != nil {
-		var i interface{} = args
-		fmt.Fprintf(&buf, "--- Sprint(%s) failed: %v", fmt.Sprint(i), err)
-	}
-	return buf.String()
-}

src/pkg/exp/datafmt/datafmt_test.go

@@ -1,330 +0,0 @@
-// Copyright 2009 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package datafmt
-
-import (
-	"fmt"
-	"testing"
-	"go/token"
-)
-
-var fset = token.NewFileSet()
-
-func parse(t *testing.T, form string, fmap FormatterMap) Format {
-	f, err := Parse(fset, "", []byte(form), fmap)
-	if err != nil {
-		t.Errorf("Parse(%s): %v", form, err)
-		return nil
-	}
-	return f
-}
-
-func verify(t *testing.T, f Format, expected string, args ...interface{}) {
-	if f == nil {
-		return // allow other tests to run
-	}
-	result := f.Sprint(args...)
-	if result != expected {
-		t.Errorf(
-			"result  : `%s`\nexpected: `%s`\n\n",
-			result, expected)
-	}
-}
-
-func formatter(s *State, value interface{}, rule_name string) bool {
-	switch rule_name {
-	case "/":
-		fmt.Fprintf(s, "%d %d %d", s.Pos().Line, s.LinePos().Column, s.Pos().Column)
-		return true
-	case "blank":
-		s.Write([]byte{' '})
-		return true
-	case "int":
-		if value.(int)&1 == 0 {
-			fmt.Fprint(s, "even ")
-		} else {
-			fmt.Fprint(s, "odd ")
-		}
-		return true
-	case "nil":
-		return false
-	case "testing.T":
-		s.Write([]byte("testing.T"))
-		return true
-	}
-	panic("unreachable")
-	return false
-}
-
-func TestCustomFormatters(t *testing.T) {
-	fmap0 := FormatterMap{"/": formatter}
-	fmap1 := FormatterMap{"int": formatter, "blank": formatter, "nil": formatter}
-	fmap2 := FormatterMap{"testing.T": formatter}
-
-	f := parse(t, `int=`, fmap0)
-	verify(t, f, ``, 1, 2, 3)
-
-	f = parse(t, `int="#"`, nil)
-	verify(t, f, `###`, 1, 2, 3)
-
-	f = parse(t, `int="#";string="%s"`, fmap0)
-	verify(t, f, "#1 0 1#1 0 7#1 0 13\n2 0 0foo2 0 8\n", 1, 2, 3, "\n", "foo", "\n")
-
-	f = parse(t, ``, fmap1)
-	verify(t, f, `even odd even odd `, 0, 1, 2, 3)
-
-	f = parse(t, `/ =@:blank; float64="#"`, fmap1)
-	verify(t, f, `# # #`, 0.0, 1.0, 2.0)
-
-	f = parse(t, `float64=@:nil`, fmap1)
-	verify(t, f, ``, 0.0, 1.0, 2.0)
-
-	f = parse(t, `testing "testing"; ptr=*`, fmap2)
-	verify(t, f, `testing.T`, t)
-
-	// TODO needs more tests
-}
-
-// ----------------------------------------------------------------------------
-// Formatting of basic and simple composite types
-
-func check(t *testing.T, form, expected string, args ...interface{}) {
-	f := parse(t, form, nil)
-	if f == nil {
-		return // allow other tests to run
-	}
-	result := f.Sprint(args...)
-	if result != expected {
-		t.Errorf(
-			"format  : %s\nresult  : `%s`\nexpected: `%s`\n\n",
-			form, result, expected)
-	}
-}
-
-func TestBasicTypes(t *testing.T) {
-	check(t, ``, ``)
-	check(t, `bool=":%v"`, `:true:false`, true, false)
-	check(t, `int="%b %d %o 0x%x"`, `101010 42 52 0x2a`, 42)
-
-	check(t, `int="%"`, `%`, 42)
-	check(t, `int="%%"`, `%`, 42)
-	check(t, `int="**%%**"`, `**%**`, 42)
-	check(t, `int="%%%%%%"`, `%%%`, 42)
-	check(t, `int="%%%d%%"`, `%42%`, 42)
-
-	const i = -42
-	const is = `-42`
-	check(t, `int  ="%d"`, is, i)
-	check(t, `int8 ="%d"`, is, int8(i))
-	check(t, `int16="%d"`, is, int16(i))
-	check(t, `int32="%d"`, is, int32(i))
-	check(t, `int64="%d"`, is, int64(i))
-
-	const u = 42
-	const us = `42`
-	check(t, `uint  ="%d"`, us, uint(u))
-	check(t, `uint8 ="%d"`, us, uint8(u))
-	check(t, `uint16="%d"`, us, uint16(u))
-	check(t, `uint32="%d"`, us, uint32(u))
-	check(t, `uint64="%d"`, us, uint64(u))
-
-	const f = 3.141592
-	const fs = `3.141592`
-	check(t, `float64="%g"`, fs, f)
-	check(t, `float32="%g"`, fs, float32(f))
-	check(t, `float64="%g"`, fs, float64(f))
-}
-
-func TestArrayTypes(t *testing.T) {
-	var a0 [10]int
-	check(t, `array="array";`, `array`, a0)
-
-	a1 := [...]int{1, 2, 3}
-	check(t, `array="array";`, `array`, a1)
-	check(t, `array={*}; int="%d";`, `123`, a1)
-	check(t, `array={* / ", "}; int="%d";`, `1, 2, 3`, a1)
-	check(t, `array={* / *}; int="%d";`, `12233`, a1)
-
-	a2 := []interface{}{42, "foo", 3.14}
-	check(t, `array={* / ", "}; interface=*; string="bar"; default="%v";`, `42, bar, 3.14`, a2)
-}
-
-func TestChanTypes(t *testing.T) {
-	var c0 chan int
-	check(t, `chan="chan"`, `chan`, c0)
-
-	c1 := make(chan int)
-	go func() { c1 <- 42 }()
-	check(t, `chan="chan"`, `chan`, c1)
-	// check(t, `chan=*`, `42`, c1);  // reflection support for chans incomplete
-}
-
-func TestFuncTypes(t *testing.T) {
-	var f0 func() int
-	check(t, `func="func"`, `func`, f0)
-
-	f1 := func() int { return 42 }
-	check(t, `func="func"`, `func`, f1)
-	// check(t, `func=*`, `42`, f1);  // reflection support for funcs incomplete
-}
-
-func TestMapTypes(t *testing.T) {
-	var m0 map[string]int
-	check(t, `map="map"`, `map`, m0)
-
-	m1 := map[string]int{}
-	check(t, `map="map"`, `map`, m1)
-	// check(t, `map=*`, ``, m1);  // reflection support for maps incomplete
-}
-
-func TestPointerTypes(t *testing.T) {
-	var p0 *int
-	check(t, `ptr="ptr"`, `ptr`, p0)
-	check(t, `ptr=*`, ``, p0)
-	check(t, `ptr=*|"nil"`, `nil`, p0)
-
-	x := 99991
-	p1 := &x
-	check(t, `ptr="ptr"`, `ptr`, p1)
-	check(t, `ptr=*; int="%d"`, `99991`, p1)
-}
-
-func TestDefaultRule(t *testing.T) {
-	check(t, `default="%v"`, `42foo3.14`, 42, "foo", 3.14)
-	check(t, `default="%v"; int="%x"`, `abcdef`, 10, 11, 12, 13, 14, 15)
-	check(t, `default="%v"; int="%x"`, `ab**ef`, 10, 11, "**", 14, 15)
-	check(t, `default="%x"; int=@:default`, `abcdef`, 10, 11, 12, 13, 14, 15)
-}
-
-func TestGlobalSeparatorRule(t *testing.T) {
-	check(t, `int="%d"; / ="-"`, `1-2-3-4`, 1, 2, 3, 4)
-	check(t, `int="%x%x"; / ="*"`, `aa*aa`, 10, 10)
-}
-
-// ----------------------------------------------------------------------------
-// Formatting of a struct
-
-type T1 struct {
-	a int
-}
-
-const F1 = `datafmt "datafmt";` +
-	`int = "%d";` +
-	`datafmt.T1 = "<" a ">";`
-
-func TestStruct1(t *testing.T) { check(t, F1, "<42>", T1{42}) }
-
-// ----------------------------------------------------------------------------
-// Formatting of a struct with an optional field (ptr)
-
-type T2 struct {
-	s string
-	p *T1
-}
-
-const F2a = F1 +
-	`string = "%s";` +
-	`ptr = *;` +
-	`datafmt.T2 = s ["-" p "-"];`
-
-const F2b = F1 +
-	`string = "%s";` +
-	`ptr = *;` +
-	`datafmt.T2 = s ("-" p "-" | "empty");`
-
-func TestStruct2(t *testing.T) {
-	check(t, F2a, "foo", T2{"foo", nil})
-	check(t, F2a, "bar-<17>-", T2{"bar", &T1{17}})
-	check(t, F2b, "fooempty", T2{"foo", nil})
-}
-
-// ----------------------------------------------------------------------------
-// Formatting of a struct with a repetitive field (slice)
-
-type T3 struct {
-	s string
-	a []int
-}
-
-const F3a = `datafmt "datafmt";` +
-	`default = "%v";` +
-	`array = *;` +
-	`datafmt.T3 = s  {" " a a / ","};`
-
-const F3b = `datafmt "datafmt";` +
-	`int = "%d";` +
-	`string = "%s";` +
-	`array = *;` +
-	`nil = ;` +
-	`empty = *:nil;` +
-	`datafmt.T3 = s [a:empty ": " {a / "-"}]`
-
-func TestStruct3(t *testing.T) {
-	check(t, F3a, "foo", T3{"foo", nil})
-	check(t, F3a, "foo 00, 11, 22", T3{"foo", []int{0, 1, 2}})
-	check(t, F3b, "bar", T3{"bar", nil})
-	check(t, F3b, "bal: 2-3-5", T3{"bal", []int{2, 3, 5}})
-}
-
-// ----------------------------------------------------------------------------
-// Formatting of a struct with alternative field
-
-type T4 struct {
-	x *int
-	a []int
-}
-
-const F4a = `datafmt "datafmt";` +
-	`int = "%d";` +
-	`ptr = *;` +
-	`array = *;` +
-	`nil = ;` +
-	`empty = *:nil;` +
-	`datafmt.T4 = "<" (x:empty x | "-") ">" `
-
-const F4b = `datafmt "datafmt";` +
-	`int = "%d";` +
-	`ptr = *;` +
-	`array = *;` +
-	`nil = ;` +
-	`empty = *:nil;` +
-	`datafmt.T4 = "<" (a:empty {a / ", "} | "-") ">" `
-
-func TestStruct4(t *testing.T) {
-	x := 7
-	check(t, F4a, "<->", T4{nil, nil})
-	check(t, F4a, "<7>", T4{&x, nil})
-	check(t, F4b, "<->", T4{nil, nil})
-	check(t, F4b, "<2, 3, 7>", T4{nil, []int{2, 3, 7}})
-}
-
-// ----------------------------------------------------------------------------
-// Formatting a struct (documentation example)
-
-type Point struct {
-	name string
-	x, y int
-}
-
-const FPoint = `datafmt "datafmt";` +
-	`int = "%d";` +
-	`hexInt = "0x%x";` +
-	`string = "---%s---";` +
-	`datafmt.Point = name "{" x ", " y:hexInt "}";`
-
-func TestStructPoint(t *testing.T) {
-	p := Point{"foo", 3, 15}
-	check(t, FPoint, "---foo---{3, 0xf}", p)
-}
-
-// ----------------------------------------------------------------------------
-// Formatting a slice (documentation example)
-
-const FSlice = `int = "%b";` +
-	`array = { * / ", " }`
-
-func TestSlice(t *testing.T) { check(t, FSlice, "10, 11, 101, 111", []int{2, 3, 5, 7}) }
-
-// TODO add more tests

src/pkg/exp/datafmt/parser.go

@@ -1,368 +0,0 @@
-// Copyright 2009 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package datafmt
-
-import (
-	"go/scanner"
-	"go/token"
-	"os"
-	"strconv"
-	"strings"
-)
-
-// ----------------------------------------------------------------------------
-// Parsing
-
-type parser struct {
-	scanner.ErrorVector
-	scanner scanner.Scanner
-	file    *token.File
-	pos     token.Pos   // token position
-	tok     token.Token // one token look-ahead
-	lit     string      // token literal
-
-	packs map[string]string // PackageName -> ImportPath
-	rules map[string]expr   // RuleName -> Expression
-}
-
-func (p *parser) next() {
-	p.pos, p.tok, p.lit = p.scanner.Scan()
-	switch p.tok {
-	case token.CHAN, token.FUNC, token.INTERFACE, token.MAP, token.STRUCT:
-		// Go keywords for composite types are type names
-		// returned by reflect. Accept them as identifiers.
-		p.tok = token.IDENT // p.lit is already set correctly
-	}
-}
-
-func (p *parser) init(fset *token.FileSet, filename string, src []byte) {
-	p.ErrorVector.Reset()
-	p.file = fset.AddFile(filename, fset.Base(), len(src))
-	p.scanner.Init(p.file, src, p, scanner.AllowIllegalChars) // return '@' as token.ILLEGAL w/o error message
-	p.next()                                                  // initializes pos, tok, lit
-	p.packs = make(map[string]string)
-	p.rules = make(map[string]expr)
-}
-
-func (p *parser) error(pos token.Pos, msg string) {
-	p.Error(p.file.Position(pos), msg)
-}
-
-func (p *parser) errorExpected(pos token.Pos, msg string) {
-	msg = "expected " + msg
-	if pos == p.pos {
-		// the error happened at the current position;
-		// make the error message more specific
-		msg += ", found '" + p.tok.String() + "'"
-		if p.tok.IsLiteral() {
-			msg += " " + p.lit
-		}
-	}
-	p.error(pos, msg)
-}
-
-func (p *parser) expect(tok token.Token) token.Pos {
-	pos := p.pos
-	if p.tok != tok {
-		p.errorExpected(pos, "'"+tok.String()+"'")
-	}
-	p.next() // make progress in any case
-	return pos
-}
-
-func (p *parser) parseIdentifier() string {
-	name := p.lit
-	p.expect(token.IDENT)
-	return name
-}
-
-func (p *parser) parseTypeName() (string, bool) {
-	pos := p.pos
-	name, isIdent := p.parseIdentifier(), true
-	if p.tok == token.PERIOD {
-		// got a package name, lookup package
-		if importPath, found := p.packs[name]; found {
-			name = importPath
-		} else {
-			p.error(pos, "package not declared: "+name)
-		}
-		p.next()
-		name, isIdent = name+"."+p.parseIdentifier(), false
-	}
-	return name, isIdent
-}
-
-// Parses a rule name and returns it. If the rule name is
-// a package-qualified type name, the package name is resolved.
-// The 2nd result value is true iff the rule name consists of a
-// single identifier only (and thus could be a package name).
-//
-func (p *parser) parseRuleName() (string, bool) {
-	name, isIdent := "", false
-	switch p.tok {
-	case token.IDENT:
-		name, isIdent = p.parseTypeName()
-	case token.DEFAULT:
-		name = "default"
-		p.next()
-	case token.QUO:
-		name = "/"
-		p.next()
-	default:
-		p.errorExpected(p.pos, "rule name")
-		p.next() // make progress in any case
-	}
-	return name, isIdent
-}
-
-func (p *parser) parseString() string {
-	s := ""
-	if p.tok == token.STRING {
-		s, _ = strconv.Unquote(p.lit)
-		// Unquote may fail with an error, but only if the scanner found
-		// an illegal string in the first place. In this case the error
-		// has already been reported.
-		p.next()
-		return s
-	} else {
-		p.expect(token.STRING)
-	}
-	return s
-}
-
-func (p *parser) parseLiteral() literal {
-	s := []byte(p.parseString())
-
-	// A string literal may contain %-format specifiers. To simplify
-	// and speed up printing of the literal, split it into segments
-	// that start with "%" possibly followed by a last segment that
-	// starts with some other character.
-	var list []interface{}
-	i0 := 0
-	for i := 0; i < len(s); i++ {
-		if s[i] == '%' && i+1 < len(s) {
-			// the next segment starts with a % format
-			if i0 < i {
-				// the current segment is not empty, split it off
-				list = append(list, s[i0:i])
-				i0 = i
-			}
-			i++ // skip %; let loop skip over char after %
-		}
-	}
-	// the final segment may start with any character
-	// (it is empty iff the string is empty)
-	list = append(list, s[i0:])
-
-	// convert list into a literal
-	lit := make(literal, len(list))
-	for i := 0; i < len(list); i++ {
-		lit[i] = list[i].([]byte)
-	}
-
-	return lit
-}
-
-func (p *parser) parseField() expr {
-	var fname string
-	switch p.tok {
-	case token.ILLEGAL:
-		if p.lit != "@" {
-			return nil
-		}
-		fname = "@"
-		p.next()
-	case token.MUL:
-		fname = "*"
-		p.next()
-	case token.IDENT:
-		fname = p.parseIdentifier()
-	default:
-		return nil
-	}
-
-	var ruleName string
-	if p.tok == token.COLON {
-		p.next()
-		ruleName, _ = p.parseRuleName()
-	}
-
-	return &field{fname, ruleName}
-}
-
-func (p *parser) parseOperand() (x expr) {
-	switch p.tok {
-	case token.STRING:
-		x = p.parseLiteral()
-
-	case token.LPAREN:
-		p.next()
-		x = p.parseExpression()
-		if p.tok == token.SHR {
-			p.next()
-			x = &group{x, p.parseExpression()}
-		}
-		p.expect(token.RPAREN)
-
-	case token.LBRACK:
-		p.next()
-		x = &option{p.parseExpression()}
-		p.expect(token.RBRACK)
-
-	case token.LBRACE:
-		p.next()
-		x = p.parseExpression()
-		var div expr
-		if p.tok == token.QUO {
-			p.next()
-			div = p.parseExpression()
-		}
-		x = &repetition{x, div}
-		p.expect(token.RBRACE)
-
-	default:
-		x = p.parseField() // may be nil
-	}
-
-	return x
-}
-
-func (p *parser) parseSequence() expr {
-	var list []interface{}
-
-	for x := p.parseOperand(); x != nil; x = p.parseOperand() {
-		list = append(list, x)
-	}
-
-	// no need for a sequence if list.Len() < 2
-	switch len(list) {
-	case 0:
-		return nil
-	case 1:
-		return list[0].(expr)
-	}
-
-	// convert list into a sequence
-	seq := make(sequence, len(list))
-	for i := 0; i < len(list); i++ {
-		seq[i] = list[i].(expr)
-	}
-	return seq
-}
-
-func (p *parser) parseExpression() expr {
-	var list []interface{}
-
-	for {
-		x := p.parseSequence()
-		if x != nil {
-			list = append(list, x)
-		}
-		if p.tok != token.OR {
-			break
-		}
-		p.next()
-	}
-
-	// no need for an alternatives if list.Len() < 2
-	switch len(list) {
-	case 0:
-		return nil
-	case 1:
-		return list[0].(expr)
-	}
-
-	// convert list into a alternatives
-	alt := make(alternatives, len(list))
-	for i := 0; i < len(list); i++ {
-		alt[i] = list[i].(expr)
-	}
-	return alt
-}
-
-func (p *parser) parseFormat() {
-	for p.tok != token.EOF {
-		pos := p.pos
-
-		name, isIdent := p.parseRuleName()
-		switch p.tok {
-		case token.STRING:
-			// package declaration
-			importPath := p.parseString()
-
-			// add package declaration
-			if !isIdent {
-				p.error(pos, "illegal package name: "+name)
-			} else if _, found := p.packs[name]; !found {
-				p.packs[name] = importPath
-			} else {
-				p.error(pos, "package already declared: "+name)
-			}
-
-		case token.ASSIGN:
-			// format rule
-			p.next()
-			x := p.parseExpression()
-
-			// add rule
-			if _, found := p.rules[name]; !found {
-				p.rules[name] = x
-			} else {
-				p.error(pos, "format rule already declared: "+name)
-			}
-
-		default:
-			p.errorExpected(p.pos, "package declaration or format rule")
-			p.next() // make progress in any case
-		}
-
-		if p.tok == token.SEMICOLON {
-			p.next()
-		} else {
-			break
-		}
-	}
-	p.expect(token.EOF)
-}
-
-func remap(p *parser, name string) string {
-	i := strings.Index(name, ".")
-	if i >= 0 {
-		packageName, suffix := name[0:i], name[i:]
-		// lookup package
-		if importPath, found := p.packs[packageName]; found {
-			name = importPath + suffix
-		} else {
-			var invalidPos token.Position
-			p.Error(invalidPos, "package not declared: "+packageName)
-		}
-	}
-	return name
-}
-
-// Parse parses a set of format productions from source src. Custom
-// formatters may be provided via a map of formatter functions. If
-// there are no errors, the result is a Format and the error is nil.
-// Otherwise the format is nil and a non-empty ErrorList is returned.
-//
-func Parse(fset *token.FileSet, filename string, src []byte, fmap FormatterMap) (Format, os.Error) {
-	// parse source
-	var p parser
-	p.init(fset, filename, src)
-	p.parseFormat()
-
-	// add custom formatters, if any
-	for name, form := range fmap {
-		name = remap(&p, name)
-		if _, found := p.rules[name]; !found {
-			p.rules[name] = &custom{name, form}
-		} else {
-			var invalidPos token.Position
-			p.Error(invalidPos, "formatter already declared: "+name)
-		}
-	}
-
-	return p.rules, p.GetError(scanner.NoMultiples)
-}

src/pkg/fmt/fmt_test.go

@@ -61,7 +61,7 @@ type I int
 func (i I) String() string { return Sprintf("<%d>", int(i)) }
 
 type B struct {
-	i I
+	I I
 	j int
 }
 
@@ -83,8 +83,8 @@ func (g G) GoString() string {
 }
 
 type S struct {
-	f F // a struct field that Formats
-	g G // a struct field that GoStrings
+	F F // a struct field that Formats
+	G G // a struct field that GoStrings
 }
 
 // A type with a String method with pointer receiver for testing %p
@@ -332,8 +332,8 @@ var fmttests = []struct {
 	{"%+v", A{1, 2, "a", []int{1, 2}}, `{i:1 j:2 s:a x:[1 2]}`},
 
 	// +v on structs with Stringable items
-	{"%+v", B{1, 2}, `{i:<1> j:2}`},
-	{"%+v", C{1, B{2, 3}}, `{i:1 B:{i:<2> j:3}}`},
+	{"%+v", B{1, 2}, `{I:<1> j:2}`},
+	{"%+v", C{1, B{2, 3}}, `{i:1 B:{I:<2> j:3}}`},
 
 	// q on Stringable items
 	{"%s", I(23), `<23>`},
@@ -349,7 +349,7 @@ var fmttests = []struct {
 	{"%#v", uint64(1<<64 - 1), "0xffffffffffffffff"},
 	{"%#v", 1000000000, "1000000000"},
 	{"%#v", map[string]int{"a": 1, "b": 2}, `map[string] int{"a":1, "b":2}`},
-	{"%#v", map[string]B{"a": {1, 2}, "b": {3, 4}}, `map[string] fmt_test.B{"a":fmt_test.B{i:1, j:2}, "b":fmt_test.B{i:3, j:4}}`},
+	{"%#v", map[string]B{"a": {1, 2}, "b": {3, 4}}, `map[string] fmt_test.B{"a":fmt_test.B{I:1, j:2}, "b":fmt_test.B{I:3, j:4}}`},
 	{"%#v", []string{"a", "b"}, `[]string{"a", "b"}`},
 
 	// slices with other formats
@@ -384,11 +384,11 @@ var fmttests = []struct {
 	// Formatter
 	{"%x", F(1), "<x=F(1)>"},
 	{"%x", G(2), "2"},
-	{"%+v", S{F(4), G(5)}, "{f:<v=F(4)> g:5}"},
+	{"%+v", S{F(4), G(5)}, "{F:<v=F(4)> G:5}"},
 
 	// GoStringer
 	{"%#v", G(6), "GoString(6)"},
-	{"%#v", S{F(7), G(8)}, "fmt_test.S{f:<v=F(7)>, g:GoString(8)}"},
+	{"%#v", S{F(7), G(8)}, "fmt_test.S{F:<v=F(7)>, G:GoString(8)}"},
 
 	// %T
 	{"%T", (4 - 3i), "complex128"},

src/pkg/fmt/print.go

@@ -258,10 +258,8 @@ func Sprintln(a ...interface{}) string {
 // the thing inside the interface, not the interface itself.
 func getField(v reflect.Value, i int) reflect.Value {
 	val := v.Field(i)
-	if i := val; i.Kind() == reflect.Interface {
-		if inter := i.Interface(); inter != nil {
-			return reflect.ValueOf(inter)
-		}
+	if val.Kind() == reflect.Interface && !val.IsNil() {
+		val = val.Elem()
 	}
 	return val
 }
@@ -288,27 +286,32 @@ func (p *pp) unknownType(v interface{}) {
 	p.buf.WriteByte('?')
 }
 
-func (p *pp) badVerb(verb int, val interface{}) {
+func (p *pp) badVerb(verb int, val interface{}, val1 reflect.Value) {
 	p.add('%')
 	p.add('!')
 	p.add(verb)
 	p.add('(')
-	if val == nil {
-		p.buf.Write(nilAngleBytes)
-	} else {
+	switch {
+	case val != nil:
 		p.buf.WriteString(reflect.TypeOf(val).String())
 		p.add('=')
 		p.printField(val, 'v', false, false, 0)
+	case val1.IsValid():
+		p.buf.WriteString(val1.Type().String())
+		p.add('=')
+		p.printValue(val1, 'v', false, false, 0)
+	default:
+		p.buf.Write(nilAngleBytes)
 	}
 	p.add(')')
 }
 
-func (p *pp) fmtBool(v bool, verb int, value interface{}) {
+func (p *pp) fmtBool(v bool, verb int, value interface{}, value1 reflect.Value) {
 	switch verb {
 	case 't', 'v':
 		p.fmt.fmt_boolean(v)
 	default:
-		p.badVerb(verb, value)
+		p.badVerb(verb, value, value1)
 	}
 }
 
@@ -322,7 +325,7 @@ func (p *pp) fmtC(c int64) {
 	p.fmt.pad(p.runeBuf[0:w])
 }
 
-func (p *pp) fmtInt64(v int64, verb int, value interface{}) {
+func (p *pp) fmtInt64(v int64, verb int, value interface{}, value1 reflect.Value) {
 	switch verb {
 	case 'b':
 		p.fmt.integer(v, 2, signed, ldigits)
@@ -336,7 +339,7 @@ func (p *pp) fmtInt64(v int64, verb int, value interface{}) {
 		if 0 <= v && v <= unicode.MaxRune {
 			p.fmt.fmt_qc(v)
 		} else {
-			p.badVerb(verb, value)
+			p.badVerb(verb, value, value1)
 		}
 	case 'x':
 		p.fmt.integer(v, 16, signed, ldigits)
@@ -345,7 +348,7 @@ func (p *pp) fmtInt64(v int64, verb int, value interface{}) {
 	case 'X':
 		p.fmt.integer(v, 16, signed, udigits)
 	default:
-		p.badVerb(verb, value)
+		p.badVerb(verb, value, value1)
 	}
 }
 
@@ -380,7 +383,7 @@ func (p *pp) fmtUnicode(v int64) {
 	p.fmt.sharp = sharp
 }
 
-func (p *pp) fmtUint64(v uint64, verb int, goSyntax bool, value interface{}) {
+func (p *pp) fmtUint64(v uint64, verb int, goSyntax bool, value interface{}, value1 reflect.Value) {
 	switch verb {
 	case 'b':
 		p.fmt.integer(int64(v), 2, unsigned, ldigits)
@@ -400,7 +403,7 @@ func (p *pp) fmtUint64(v uint64, verb int, goSyntax bool, value interface{}) {
 		if 0 <= v && v <= unicode.MaxRune {
 			p.fmt.fmt_qc(int64(v))
 		} else {
-			p.badVerb(verb, value)
+			p.badVerb(verb, value, value1)
 		}
 	case 'x':
 		p.fmt.integer(int64(v), 16, unsigned, ldigits)
@@ -409,11 +412,11 @@ func (p *pp) fmtUint64(v uint64, verb int, goSyntax bool, value interface{}) {
 	case 'U':
 		p.fmtUnicode(int64(v))
 	default:
-		p.badVerb(verb, value)
+		p.badVerb(verb, value, value1)
 	}
 }
 
-func (p *pp) fmtFloat32(v float32, verb int, value interface{}) {
+func (p *pp) fmtFloat32(v float32, verb int, value interface{}, value1 reflect.Value) {
 	switch verb {
 	case 'b':
 		p.fmt.fmt_fb32(v)
@@ -428,11 +431,11 @@ func (p *pp) fmtFloat32(v float32, verb int, value interface{}) {
 	case 'G':
 		p.fmt.fmt_G32(v)
 	default:
-		p.badVerb(verb, value)
+		p.badVerb(verb, value, value1)
 	}
 }
 
-func (p *pp) fmtFloat64(v float64, verb int, value interface{}) {
+func (p *pp) fmtFloat64(v float64, verb int, value interface{}, value1 reflect.Value) {
 	switch verb {
 	case 'b':
 		p.fmt.fmt_fb64(v)
@@ -447,33 +450,33 @@ func (p *pp) fmtFloat64(v float64, verb int, value interface{}) {
 	case 'G':
 		p.fmt.fmt_G64(v)
 	default:
-		p.badVerb(verb, value)
+		p.badVerb(verb, value, value1)
 	}
 }
 
-func (p *pp) fmtComplex64(v complex64, verb int, value interface{}) {
+func (p *pp) fmtComplex64(v complex64, verb int, value interface{}, value1 reflect.Value) {
 	switch verb {
 	case 'e', 'E', 'f', 'F', 'g', 'G':
 		p.fmt.fmt_c64(v, verb)
 	case 'v':
 		p.fmt.fmt_c64(v, 'g')
 	default:
-		p.badVerb(verb, value)
+		p.badVerb(verb, value, value1)
 	}
 }
 
-func (p *pp) fmtComplex128(v complex128, verb int, value interface{}) {
+func (p *pp) fmtComplex128(v complex128, verb int, value interface{}, value1 reflect.Value) {
 	switch verb {
 	case 'e', 'E', 'f', 'F', 'g', 'G':
 		p.fmt.fmt_c128(v, verb)
 	case 'v':
 		p.fmt.fmt_c128(v, 'g')
 	default:
-		p.badVerb(verb, value)
+		p.badVerb(verb, value, value1)
 	}
 }
 
-func (p *pp) fmtString(v string, verb int, goSyntax bool, value interface{}) {
+func (p *pp) fmtString(v string, verb int, goSyntax bool, value interface{}, value1 reflect.Value) {
 	switch verb {
 	case 'v':
 		if goSyntax {
@@ -490,11 +493,11 @@ func (p *pp) fmtString(v string, verb int, goSyntax bool, value interface{}) {
 	case 'q':
 		p.fmt.fmt_q(v)
 	default:
-		p.badVerb(verb, value)
+		p.badVerb(verb, value, value1)
 	}
 }
 
-func (p *pp) fmtBytes(v []byte, verb int, goSyntax bool, depth int, value interface{}) {
+func (p *pp) fmtBytes(v []byte, verb int, goSyntax bool, depth int, value interface{}, value1 reflect.Value) {
 	if verb == 'v' || verb == 'd' {
 		if goSyntax {
 			p.buf.Write(bytesBytes)
@@ -529,7 +532,7 @@ func (p *pp) fmtBytes(v []byte, verb int, goSyntax bool, depth int, value interf
 	case 'q':
 		p.fmt.fmt_q(s)
 	default:
-		p.badVerb(verb, value)
+		p.badVerb(verb, value, value1)
 	}
 }
 
@@ -539,12 +542,12 @@ func (p *pp) fmtPointer(field interface{}, value reflect.Value, verb int, goSynt
 	case reflect.Chan, reflect.Func, reflect.Map, reflect.Ptr, reflect.Slice, reflect.UnsafePointer:
 		u = value.Pointer()
 	default:
-		p.badVerb(verb, field)
+		p.badVerb(verb, field, value)
 		return
 	}
 	if goSyntax {
 		p.add('(')
-		p.buf.WriteString(reflect.TypeOf(field).String())
+		p.buf.WriteString(value.Type().String())
 		p.add(')')
 		p.add('(')
 		if u == 0 {
@@ -590,12 +593,51 @@ func (p *pp) catchPanic(val interface{}, verb int) {
 	}
 }
 
+func (p *pp) handleMethods(field interface{}, verb int, plus, goSyntax bool, depth int) (wasString, handled bool) {
+	// Is it a Formatter?
+	if formatter, ok := field.(Formatter); ok {
+		handled = true
+		wasString = false
+		defer p.catchPanic(field, verb)
+		formatter.Format(p, verb)
+		return
+	}
+	// Must not touch flags before Formatter looks at them.
+	if plus {
+		p.fmt.plus = false
+	}
+
+	// If we're doing Go syntax and the field knows how to supply it, take care of it now.
+	if goSyntax {
+		p.fmt.sharp = false
+		if stringer, ok := field.(GoStringer); ok {
+			wasString = false
+			handled = true
+			defer p.catchPanic(field, verb)
+			// Print the result of GoString unadorned.
+			p.fmtString(stringer.GoString(), 's', false, field, reflect.Value{})
+			return
+		}
+	} else {
+		// Is it a Stringer?
+		if stringer, ok := field.(Stringer); ok {
+			wasString = false
+			handled = true
+			defer p.catchPanic(field, verb)
+			p.printField(stringer.String(), verb, plus, false, depth)
+			return
+		}
+	}
+	handled = false
+	return
+}
+
 func (p *pp) printField(field interface{}, verb int, plus, goSyntax bool, depth int) (wasString bool) {
 	if field == nil {
 		if verb == 'T' || verb == 'v' {
 			p.buf.Write(nilAngleBytes)
 		} else {
-			p.badVerb(verb, field)
+			p.badVerb(verb, field, reflect.Value{})
 		}
 		return false
 	}
@@ -610,118 +652,133 @@ func (p *pp) printField(field interface{}, verb int, plus, goSyntax bool, depth
 		p.fmtPointer(field, reflect.ValueOf(field), verb, goSyntax)
 		return false
 	}
-	// Is it a Formatter?
-	if formatter, ok := field.(Formatter); ok {
-		defer p.catchPanic(field, verb)
-		formatter.Format(p, verb)
-		return false // this value is not a string
 
-	}
-	// Must not touch flags before Formatter looks at them.
-	if plus {
-		p.fmt.plus = false
-	}
-	// If we're doing Go syntax and the field knows how to supply it, take care of it now.
-	if goSyntax {
-		p.fmt.sharp = false
-		if stringer, ok := field.(GoStringer); ok {
-			defer p.catchPanic(field, verb)
-			// Print the result of GoString unadorned.
-			p.fmtString(stringer.GoString(), 's', false, field)
-			return false // this value is not a string
-		}
-	} else {
-		// Is it a Stringer?
-		if stringer, ok := field.(Stringer); ok {
-			defer p.catchPanic(field, verb)
-			p.printField(stringer.String(), verb, plus, false, depth)
-			return false // this value is not a string
-		}
+	if wasString, handled := p.handleMethods(field, verb, plus, goSyntax, depth); handled {
+		return wasString
 	}
 
 	// Some types can be done without reflection.
 	switch f := field.(type) {
 	case bool:
-		p.fmtBool(f, verb, field)
+		p.fmtBool(f, verb, field, reflect.Value{})
 		return false
 	case float32:
-		p.fmtFloat32(f, verb, field)
+		p.fmtFloat32(f, verb, field, reflect.Value{})
 		return false
 	case float64:
-		p.fmtFloat64(f, verb, field)
+		p.fmtFloat64(f, verb, field, reflect.Value{})
 		return false
 	case complex64:
-		p.fmtComplex64(complex64(f), verb, field)
+		p.fmtComplex64(complex64(f), verb, field, reflect.Value{})
 		return false
 	case complex128:
-		p.fmtComplex128(f, verb, field)
+		p.fmtComplex128(f, verb, field, reflect.Value{})
 		return false
 	case int:
-		p.fmtInt64(int64(f), verb, field)
+		p.fmtInt64(int64(f), verb, field, reflect.Value{})
 		return false
 	case int8:
-		p.fmtInt64(int64(f), verb, field)
+		p.fmtInt64(int64(f), verb, field, reflect.Value{})
 		return false
 	case int16:
-		p.fmtInt64(int64(f), verb, field)
+		p.fmtInt64(int64(f), verb, field, reflect.Value{})
 		return false
 	case int32:
-		p.fmtInt64(int64(f), verb, field)
+		p.fmtInt64(int64(f), verb, field, reflect.Value{})
 		return false
 	case int64:
-		p.fmtInt64(f, verb, field)
+		p.fmtInt64(f, verb, field, reflect.Value{})
 		return false
 	case uint:
-		p.fmtUint64(uint64(f), verb, goSyntax, field)
+		p.fmtUint64(uint64(f), verb, goSyntax, field, reflect.Value{})
 		return false
 	case uint8:
-		p.fmtUint64(uint64(f), verb, goSyntax, field)
+		p.fmtUint64(uint64(f), verb, goSyntax, field, reflect.Value{})
 		return false
 	case uint16:
-		p.fmtUint64(uint64(f), verb, goSyntax, field)
+		p.fmtUint64(uint64(f), verb, goSyntax, field, reflect.Value{})
 		return false
 	case uint32:
-		p.fmtUint64(uint64(f), verb, goSyntax, field)
+		p.fmtUint64(uint64(f), verb, goSyntax, field, reflect.Value{})
 		return false
 	case uint64:
-		p.fmtUint64(f, verb, goSyntax, field)
+		p.fmtUint64(f, verb, goSyntax, field, reflect.Value{})
 		return false
 	case uintptr:
-		p.fmtUint64(uint64(f), verb, goSyntax, field)
+		p.fmtUint64(uint64(f), verb, goSyntax, field, reflect.Value{})
 		return false
 	case string:
-		p.fmtString(f, verb, goSyntax, field)
+		p.fmtString(f, verb, goSyntax, field, reflect.Value{})
 		return verb == 's' || verb == 'v'
 	case []byte:
-		p.fmtBytes(f, verb, goSyntax, depth, field)
+		p.fmtBytes(f, verb, goSyntax, depth, field, reflect.Value{})
 		return verb == 's'
 	}
 
 	// Need to use reflection
-	value := reflect.ValueOf(field)
+	return p.printReflectValue(reflect.ValueOf(field), verb, plus, goSyntax, depth)
+}
 
+// printValue is like printField but starts with a reflect value, not an interface{} value.
+func (p *pp) printValue(value reflect.Value, verb int, plus, goSyntax bool, depth int) (wasString bool) {
+	if !value.IsValid() {
+		if verb == 'T' || verb == 'v' {
+			p.buf.Write(nilAngleBytes)
+		} else {
+			p.badVerb(verb, nil, value)
+		}
+		return false
+	}
+
+	// Special processing considerations.
+	// %T (the value's type) and %p (its address) are special; we always do them first.
+	switch verb {
+	case 'T':
+		p.printField(value.Type().String(), 's', false, false, 0)
+		return false
+	case 'p':
+		p.fmtPointer(nil, value, verb, goSyntax)
+		return false
+	}
+
+	// Handle values with special methods.
+	// Call always, even when field == nil, because handleMethods clears p.fmt.plus for us.
+	var field interface{}
+	if value.CanInterface() {
+		field = value.Interface()
+	}
+	if wasString, handled := p.handleMethods(field, verb, plus, goSyntax, depth); handled {
+		return wasString
+	}
+
+	return p.printReflectValue(value, verb, plus, goSyntax, depth)
+}
+
+// printReflectValue is the fallback for both printField and printValue.
+// It uses reflect to print the value.
+func (p *pp) printReflectValue(value reflect.Value, verb int, plus, goSyntax bool, depth int) (wasString bool) {
 BigSwitch:
 	switch f := value; f.Kind() {
 	case reflect.Bool:
-		p.fmtBool(f.Bool(), verb, field)
+		p.fmtBool(f.Bool(), verb, nil, value)
 	case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
-		p.fmtInt64(f.Int(), verb, field)
+		p.fmtInt64(f.Int(), verb, nil, value)
 	case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
-		p.fmtUint64(uint64(f.Uint()), verb, goSyntax, field)
+		p.fmtUint64(uint64(f.Uint()), verb, goSyntax, nil, value)
 	case reflect.Float32, reflect.Float64:
 		if f.Type().Size() == 4 {
-			p.fmtFloat32(float32(f.Float()), verb, field)
+			p.fmtFloat32(float32(f.Float()), verb, nil, value)
 		} else {
-			p.fmtFloat64(float64(f.Float()), verb, field)
+			p.fmtFloat64(float64(f.Float()), verb, nil, value)
 		}
 	case reflect.Complex64, reflect.Complex128:
 		if f.Type().Size() == 8 {
-			p.fmtComplex64(complex64(f.Complex()), verb, field)
+			p.fmtComplex64(complex64(f.Complex()), verb, nil, value)
 		} else {
-			p.fmtComplex128(complex128(f.Complex()), verb, field)
+			p.fmtComplex128(complex128(f.Complex()), verb, nil, value)
 		}
 	case reflect.String:
-		p.fmtString(f.String(), verb, goSyntax, field)
+		p.fmtString(f.String(), verb, goSyntax, nil, value)
 	case reflect.Map:
 		if goSyntax {
 			p.buf.WriteString(f.Type().String())
@@ -738,9 +795,9 @@ BigSwitch:
 					p.buf.WriteByte(' ')
 				}
 			}
-			p.printField(key.Interface(), verb, plus, goSyntax, depth+1)
+			p.printValue(key, verb, plus, goSyntax, depth+1)
 			p.buf.WriteByte(':')
-			p.printField(f.MapIndex(key).Interface(), verb, plus, goSyntax, depth+1)
+			p.printValue(f.MapIndex(key), verb, plus, goSyntax, depth+1)
 		}
 		if goSyntax {
 			p.buf.WriteByte('}')
@@ -749,7 +806,7 @@ BigSwitch:
 		}
 	case reflect.Struct:
 		if goSyntax {
-			p.buf.WriteString(reflect.TypeOf(field).String())
+			p.buf.WriteString(value.Type().String())
 		}
 		p.add('{')
 		v := f
@@ -768,20 +825,20 @@ BigSwitch:
 					p.buf.WriteByte(':')
 				}
 			}
-			p.printField(getField(v, i).Interface(), verb, plus, goSyntax, depth+1)
+			p.printValue(getField(v, i), verb, plus, goSyntax, depth+1)
 		}
 		p.buf.WriteByte('}')
 	case reflect.Interface:
 		value := f.Elem()
 		if !value.IsValid() {
 			if goSyntax {
-				p.buf.WriteString(reflect.TypeOf(field).String())
+				p.buf.WriteString(value.Type().String())
 				p.buf.Write(nilParenBytes)
 			} else {
 				p.buf.Write(nilAngleBytes)
 			}
 		} else {
-			return p.printField(value.Interface(), verb, plus, goSyntax, depth+1)
+			return p.printValue(value, verb, plus, goSyntax, depth+1)
 		}
 	case reflect.Array, reflect.Slice:
 		// Byte slices are special.
@@ -797,11 +854,11 @@ BigSwitch:
 			for i := range bytes {
 				bytes[i] = byte(f.Index(i).Uint())
 			}
-			p.fmtBytes(bytes, verb, goSyntax, depth, field)
+			p.fmtBytes(bytes, verb, goSyntax, depth, nil, value)
 			return verb == 's'
 		}
 		if goSyntax {
-			p.buf.WriteString(reflect.TypeOf(field).String())
+			p.buf.WriteString(value.Type().String())
 			p.buf.WriteByte('{')
 		} else {
 			p.buf.WriteByte('[')
@@ -814,7 +871,7 @@ BigSwitch:
 					p.buf.WriteByte(' ')
 				}
 			}
-			p.printField(f.Index(i).Interface(), verb, plus, goSyntax, depth+1)
+			p.printValue(f.Index(i), verb, plus, goSyntax, depth+1)
 		}
 		if goSyntax {
 			p.buf.WriteByte('}')
@@ -829,17 +886,17 @@ BigSwitch:
 			switch a := f.Elem(); a.Kind() {
 			case reflect.Array, reflect.Slice:
 				p.buf.WriteByte('&')
-				p.printField(a.Interface(), verb, plus, goSyntax, depth+1)
+				p.printValue(a, verb, plus, goSyntax, depth+1)
 				break BigSwitch
 			case reflect.Struct:
 				p.buf.WriteByte('&')
-				p.printField(a.Interface(), verb, plus, goSyntax, depth+1)
+				p.printValue(a, verb, plus, goSyntax, depth+1)
 				break BigSwitch
 			}
 		}
 		if goSyntax {
 			p.buf.WriteByte('(')
-			p.buf.WriteString(reflect.TypeOf(field).String())
+			p.buf.WriteString(value.Type().String())
 			p.buf.WriteByte(')')
 			p.buf.WriteByte('(')
 			if v == 0 {
@@ -856,7 +913,7 @@ BigSwitch:
 		}
 		p.fmt0x64(uint64(v), true)
 	case reflect.Chan, reflect.Func, reflect.UnsafePointer:
-		p.fmtPointer(field, value, verb, goSyntax)
+		p.fmtPointer(nil, value, verb, goSyntax)
 	default:
 		p.unknownType(f)
 	}

src/pkg/gob/doc.go

@@ -221,6 +221,9 @@ In summary, a gob stream looks like
 
 where * signifies zero or more repetitions and the type id of a value must
 be predefined or be defined before the value in the stream.
+
+See "Gobs of data" for a design discussion of the gob wire format:
+http://blog.golang.org/2011/03/gobs-of-data.html
 */
 package gob
 

src/pkg/image/draw/draw.go

@@ -2,10 +2,10 @@
 // Use of this source code is governed by a BSD-style
 // license that can be found in the LICENSE file.
 
-// Package draw provides image composition functions
-// in the style of the Plan 9 graphics library
-// (see http://plan9.bell-labs.com/magic/man2html/2/draw)
-// and the X Render extension.
+// Package draw provides image composition functions.
+//
+// See "The Go image/draw package" for an introduction to this package:
+// http://blog.golang.org/2011/09/go-imagedraw-package.html
 package draw
 
 import (

src/pkg/image/image.go

@@ -3,6 +3,9 @@
 // license that can be found in the LICENSE file.
 
 // Package image implements a basic 2-D image library.
+//
+// See "The Go image package" for an introduction to this package:
+// http://blog.golang.org/2011/09/go-image-package.html
 package image
 
 // Config holds an image's color model and dimensions.

src/pkg/json/Makefile

@@ -11,5 +11,6 @@ GOFILES=\
 	indent.go\
 	scanner.go\
 	stream.go\
+	tags.go\
 
 include ../../Make.pkg

src/pkg/json/decode.go

@@ -140,6 +140,7 @@ type decodeState struct {
 	scan       scanner
 	nextscan   scanner // for calls to nextValue
 	savedError os.Error
+	tempstr    string // scratch space to avoid some allocations
 }
 
 // errPhase is used for errors that should not happen unless
@@ -470,6 +471,8 @@ func (d *decodeState) object(v reflect.Value) {
 
 		// Figure out field corresponding to key.
 		var subv reflect.Value
+		destring := false // whether the value is wrapped in a string to be decoded first
+
 		if mv.IsValid() {
 			elemType := mv.Type().Elem()
 			if !mapElem.IsValid() {
@@ -486,7 +489,8 @@ func (d *decodeState) object(v reflect.Value) {
 			if isValidTag(key) {
 				for i := 0; i < sv.NumField(); i++ {
 					f = st.Field(i)
-					if f.Tag.Get("json") == key {
+					tagName, _ := parseTag(f.Tag.Get("json"))
+					if tagName == key {
 						ok = true
 						break
 					}
@@ -508,6 +512,8 @@ func (d *decodeState) object(v reflect.Value) {
 				} else {
 					subv = sv.FieldByIndex(f.Index)
 				}
+				_, opts := parseTag(f.Tag.Get("json"))
+				destring = opts.Contains("string")
 			}
 		}
 
@@ -520,8 +526,12 @@ func (d *decodeState) object(v reflect.Value) {
 		}
 
 		// Read value.
-		d.value(subv)
-
+		if destring {
+			d.value(reflect.ValueOf(&d.tempstr))
+			d.literalStore([]byte(d.tempstr), subv)
+		} else {
+			d.value(subv)
+		}
 		// Write value back to map;
 		// if using struct, subv points into struct already.
 		if mv.IsValid() {
@@ -550,8 +560,12 @@ func (d *decodeState) literal(v reflect.Value) {
 	// Scan read one byte too far; back up.
 	d.off--
 	d.scan.undo(op)
-	item := d.data[start:d.off]
 
+	d.literalStore(d.data[start:d.off], v)
+}
+
+// literalStore decodes a literal stored in item into v.
+func (d *decodeState) literalStore(item []byte, v reflect.Value) {
 	// Check for unmarshaler.
 	wantptr := item[0] == 'n' // null
 	unmarshaler, pv := d.indirect(v, wantptr)

src/pkg/json/decode_test.go

@@ -262,7 +262,10 @@ type All struct {
 	Float32 float32
 	Float64 float64
 
-	Foo string `json:"bar"`
+	Foo  string `json:"bar"`
+	Foo2 string `json:"bar2,dummyopt"`
+
+	IntStr int64 `json:",string"`
 
 	PBool    *bool
 	PInt     *int
@@ -331,6 +334,8 @@ var allValue = All{
 	Float32: 14.1,
 	Float64: 15.1,
 	Foo:     "foo",
+	Foo2:    "foo2",
+	IntStr:  42,
 	String:  "16",
 	Map: map[string]Small{
 		"17": {Tag: "tag17"},
@@ -391,6 +396,8 @@ var allValueIndent = `{
 	"Float32": 14.1,
 	"Float64": 15.1,
 	"bar": "foo",
+	"bar2": "foo2",
+	"IntStr": "42",
 	"PBool": null,
 	"PInt": null,
 	"PInt8": null,
@@ -481,6 +488,8 @@ var pallValueIndent = `{
 	"Float32": 0,
 	"Float64": 0,
 	"bar": "",
+	"bar2": "",
+        "IntStr": "0",
 	"PBool": true,
 	"PInt": 2,
 	"PInt8": 3,

src/pkg/json/encode.go

@@ -4,6 +4,9 @@
 
 // Package json implements encoding and decoding of JSON objects as defined in
 // RFC 4627.
+//
+// See "JSON and Go" for an introduction to this package:
+// http://blog.golang.org/2011/01/json-and-go.html
 package json
 
 import (
@@ -14,7 +17,6 @@ import (
 	"runtime"
 	"sort"
 	"strconv"
-	"strings"
 	"unicode"
 	"utf8"
 )
@@ -59,6 +61,12 @@ import (
 //   // Note the leading comma.
 //   Field int `json:",omitempty"`
 //
+// The "string" option signals that a field is stored as JSON inside a
+// JSON-encoded string.  This extra level of encoding is sometimes
+// used when communicating with JavaScript programs:
+//
+//    Int64String int64 `json:",string"`
+//
 // The key name will be used if it's a non-empty string consisting of
 // only Unicode letters, digits, dollar signs, hyphens, and underscores.
 //
@@ -221,6 +229,12 @@ func isEmptyValue(v reflect.Value) bool {
 }
 
 func (e *encodeState) reflectValue(v reflect.Value) {
+	e.reflectValueQuoted(v, false)
+}
+
+// reflectValueQuoted writes the value in v to the output.
+// If quoted is true, the serialization is wrapped in a JSON string.
+func (e *encodeState) reflectValueQuoted(v reflect.Value, quoted bool) {
 	if !v.IsValid() {
 		e.WriteString("null")
 		return
@@ -238,26 +252,39 @@ func (e *encodeState) reflectValue(v reflect.Value) {
 		return
 	}
 
+	writeString := (*encodeState).WriteString
+	if quoted {
+		writeString = (*encodeState).string
+	}
+
 	switch v.Kind() {
 	case reflect.Bool:
 		x := v.Bool()
 		if x {
-			e.WriteString("true")
+			writeString(e, "true")
 		} else {
-			e.WriteString("false")
+			writeString(e, "false")
 		}
 
 	case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
-		e.WriteString(strconv.Itoa64(v.Int()))
+		writeString(e, strconv.Itoa64(v.Int()))
 
 	case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
-		e.WriteString(strconv.Uitoa64(v.Uint()))
+		writeString(e, strconv.Uitoa64(v.Uint()))
 
 	case reflect.Float32, reflect.Float64:
-		e.WriteString(strconv.FtoaN(v.Float(), 'g', -1, v.Type().Bits()))
+		writeString(e, strconv.FtoaN(v.Float(), 'g', -1, v.Type().Bits()))
 
 	case reflect.String:
-		e.string(v.String())
+		if quoted {
+			sb, err := Marshal(v.String())
+			if err != nil {
+				e.error(err)
+			}
+			e.string(string(sb))
+		} else {
+			e.string(v.String())
+		}
 
 	case reflect.Struct:
 		e.WriteByte('{')
@@ -269,17 +296,14 @@ func (e *encodeState) reflectValue(v reflect.Value) {
 			if f.PkgPath != "" {
 				continue
 			}
-			tag, omitEmpty := f.Name, false
+			tag, omitEmpty, quoted := f.Name, false, false
 			if tv := f.Tag.Get("json"); tv != "" {
-				ss := strings.SplitN(tv, ",", 2)
-				if isValidTag(ss[0]) {
-					tag = ss[0]
-				}
-				if len(ss) > 1 {
-					// Currently the only option is omitempty,
-					// so parsing is trivial.
-					omitEmpty = ss[1] == "omitempty"
+				name, opts := parseTag(tv)
+				if isValidTag(name) {
+					tag = name
 				}
+				omitEmpty = opts.Contains("omitempty")
+				quoted = opts.Contains("string")
 			}
 			fieldValue := v.Field(i)
 			if omitEmpty && isEmptyValue(fieldValue) {
@@ -292,7 +316,7 @@ func (e *encodeState) reflectValue(v reflect.Value) {
 			}
 			e.string(tag)
 			e.WriteByte(':')
-			e.reflectValue(fieldValue)
+			e.reflectValueQuoted(fieldValue, quoted)
 		}
 		e.WriteByte('}')
 
@@ -380,7 +404,8 @@ func (sv stringValues) Swap(i, j int)      { sv[i], sv[j] = sv[j], sv[i] }
 func (sv stringValues) Less(i, j int) bool { return sv.get(i) < sv.get(j) }
 func (sv stringValues) get(i int) string   { return sv[i].String() }
 
-func (e *encodeState) string(s string) {
+func (e *encodeState) string(s string) (int, os.Error) {
+	len0 := e.Len()
 	e.WriteByte('"')
 	start := 0
 	for i := 0; i < len(s); {
@@ -425,4 +450,5 @@ func (e *encodeState) string(s string) {
 		e.WriteString(s[start:])
 	}
 	e.WriteByte('"')
+	return e.Len() - len0, nil
 }

src/pkg/json/encode_test.go

@@ -5,6 +5,8 @@
 package json
 
 import (
+	"bytes"
+	"reflect"
 	"testing"
 )
 
@@ -42,3 +44,39 @@ func TestOmitEmpty(t *testing.T) {
 		t.Errorf(" got: %s\nwant: %s\n", got, optionalsExpected)
 	}
 }
+
+type StringTag struct {
+	BoolStr bool   `json:",string"`
+	IntStr  int64  `json:",string"`
+	StrStr  string `json:",string"`
+}
+
+var stringTagExpected = `{
+ "BoolStr": "true",
+ "IntStr": "42",
+ "StrStr": "\"xzbit\""
+}`
+
+func TestStringTag(t *testing.T) {
+	var s StringTag
+	s.BoolStr = true
+	s.IntStr = 42
+	s.StrStr = "xzbit"
+	got, err := MarshalIndent(&s, "", " ")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got := string(got); got != stringTagExpected {
+		t.Fatalf(" got: %s\nwant: %s\n", got, stringTagExpected)
+	}
+
+	// Verify that it round-trips.
+	var s2 StringTag
+	err = NewDecoder(bytes.NewBuffer(got)).Decode(&s2)
+	if err != nil {
+		t.Fatalf("Decode: %v", err)
+	}
+	if !reflect.DeepEqual(s, s2) {
+		t.Fatalf("decode didn't match.\nsource: %#v\nEncoded as:\n%s\ndecode: %#v", s, string(got), s2)
+	}
+}

src/pkg/json/tags.go

@@ -0,0 +1,44 @@
+// Copyright 2011 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package json
+
+import (
+	"strings"
+)
+
+// tagOptions is the string following a comma in a struct field's "json"
+// tag, or the empty string. It does not include the leading comma.
+type tagOptions string
+
+// parseTag splits a struct field's json tag into its name and
+// comma-separated options.
+func parseTag(tag string) (string, tagOptions) {
+	if idx := strings.Index(tag, ","); idx != -1 {
+		return tag[:idx], tagOptions(tag[idx+1:])
+	}
+	return tag, tagOptions("")
+}
+
+// Contains returns whether checks that a comma-separated list of options
+// contains a particular substr flag. substr must be surrounded by a
+// string boundary or commas.
+func (o tagOptions) Contains(optionName string) bool {
+	if len(o) == 0 {
+		return false
+	}
+	s := string(o)
+	for s != "" {
+		var next string
+		i := strings.Index(s, ",")
+		if i >= 0 {
+			s, next = s[:i], s[i+1:]
+		}
+		if s == optionName {
+			return true
+		}
+		s = next
+	}
+	return false
+}

src/pkg/json/tags_test.go

@@ -0,0 +1,28 @@
+// Copyright 2011 The Go Authors.  All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package json
+
+import (
+	"testing"
+)
+
+func TestTagParsing(t *testing.T) {
+	name, opts := parseTag("field,foobar,foo")
+	if name != "field" {
+		t.Fatalf("name = %q, want field", name)
+	}
+	for _, tt := range []struct {
+		opt  string
+		want bool
+	}{
+		{"foobar", true},
+		{"foo", true},
+		{"bar", false},
+	} {
+		if opts.Contains(tt.opt) != tt.want {
+			t.Errorf("Contains(%q) = %v", tt.opt, !tt.want)
+		}
+	}
+}

src/pkg/reflect/all_test.go

@@ -853,13 +853,13 @@ func TestIsNil(t *testing.T) {
 
 func TestInterfaceExtraction(t *testing.T) {
 	var s struct {
-		w io.Writer
+		W io.Writer
 	}
 
-	s.w = os.Stdout
+	s.W = os.Stdout
 	v := Indirect(ValueOf(&s)).Field(0).Interface()
-	if v != s.w.(interface{}) {
-		t.Error("Interface() on interface: ", v, s.w)
+	if v != s.W.(interface{}) {
+		t.Error("Interface() on interface: ", v, s.W)
 	}
 }
 
@@ -1190,18 +1190,18 @@ type D2 struct {
 }
 
 type S0 struct {
-	a, b, c int
+	A, B, C int
 	D1
 	D2
 }
 
 type S1 struct {
-	b int
+	B int
 	S0
 }
 
 type S2 struct {
-	a int
+	A int
 	*S1
 }
 
@@ -1216,36 +1216,36 @@ type S1y struct {
 type S3 struct {
 	S1x
 	S2
-	d, e int
+	D, E int
 	*S1y
 }
 
 type S4 struct {
 	*S4
-	a int
+	A int
 }
 
 var fieldTests = []FTest{
 	{struct{}{}, "", nil, 0},
-	{struct{}{}, "foo", nil, 0},
-	{S0{a: 'a'}, "a", []int{0}, 'a'},
-	{S0{}, "d", nil, 0},
-	{S1{S0: S0{a: 'a'}}, "a", []int{1, 0}, 'a'},
-	{S1{b: 'b'}, "b", []int{0}, 'b'},
+	{struct{}{}, "Foo", nil, 0},
+	{S0{A: 'a'}, "A", []int{0}, 'a'},
+	{S0{}, "D", nil, 0},
+	{S1{S0: S0{A: 'a'}}, "A", []int{1, 0}, 'a'},
+	{S1{B: 'b'}, "B", []int{0}, 'b'},
 	{S1{}, "S0", []int{1}, 0},
-	{S1{S0: S0{c: 'c'}}, "c", []int{1, 2}, 'c'},
-	{S2{a: 'a'}, "a", []int{0}, 'a'},
+	{S1{S0: S0{C: 'c'}}, "C", []int{1, 2}, 'c'},
+	{S2{A: 'a'}, "A", []int{0}, 'a'},
 	{S2{}, "S1", []int{1}, 0},
-	{S2{S1: &S1{b: 'b'}}, "b", []int{1, 0}, 'b'},
-	{S2{S1: &S1{S0: S0{c: 'c'}}}, "c", []int{1, 1, 2}, 'c'},
-	{S2{}, "d", nil, 0},
+	{S2{S1: &S1{B: 'b'}}, "B", []int{1, 0}, 'b'},
+	{S2{S1: &S1{S0: S0{C: 'c'}}}, "C", []int{1, 1, 2}, 'c'},
+	{S2{}, "D", nil, 0},
 	{S3{}, "S1", nil, 0},
-	{S3{S2: S2{a: 'a'}}, "a", []int{1, 0}, 'a'},
-	{S3{}, "b", nil, 0},
-	{S3{d: 'd'}, "d", []int{2}, 0},
-	{S3{e: 'e'}, "e", []int{3}, 'e'},
-	{S4{a: 'a'}, "a", []int{1}, 'a'},
-	{S4{}, "b", nil, 0},
+	{S3{S2: S2{A: 'a'}}, "A", []int{1, 0}, 'a'},
+	{S3{}, "B", nil, 0},
+	{S3{D: 'd'}, "D", []int{2}, 0},
+	{S3{E: 'e'}, "E", []int{3}, 'e'},
+	{S4{A: 'a'}, "A", []int{1}, 'a'},
+	{S4{}, "B", nil, 0},
 }
 
 func TestFieldByIndex(t *testing.T) {
@@ -1562,3 +1562,68 @@ func TestTagGet(t *testing.T) {
 		}
 	}
 }
+
+type Private struct {
+	x int
+	y **int
+}
+
+func (p *Private) m() {
+}
+
+type Public struct {
+	X int
+	Y **int
+}
+
+func (p *Public) M() {
+}
+
+func TestUnexported(t *testing.T) {
+	var pub Public
+	v := ValueOf(&pub)
+	isValid(v.Elem().Field(0))
+	isValid(v.Elem().Field(1))
+	isValid(v.Elem().FieldByName("X"))
+	isValid(v.Elem().FieldByName("Y"))
+	isValid(v.Type().Method(0).Func)
+	isNonNil(v.Elem().Field(0).Interface())
+	isNonNil(v.Elem().Field(1).Interface())
+	isNonNil(v.Elem().FieldByName("X").Interface())
+	isNonNil(v.Elem().FieldByName("Y").Interface())
+	isNonNil(v.Type().Method(0).Func.Interface())
+
+	var priv Private
+	v = ValueOf(&priv)
+	isValid(v.Elem().Field(0))
+	isValid(v.Elem().Field(1))
+	isValid(v.Elem().FieldByName("x"))
+	isValid(v.Elem().FieldByName("y"))
+	isValid(v.Type().Method(0).Func)
+	shouldPanic(func() { v.Elem().Field(0).Interface() })
+	shouldPanic(func() { v.Elem().Field(1).Interface() })
+	shouldPanic(func() { v.Elem().FieldByName("x").Interface() })
+	shouldPanic(func() { v.Elem().FieldByName("y").Interface() })
+	shouldPanic(func() { v.Type().Method(0).Func.Interface() })
+}
+
+func shouldPanic(f func()) {
+	defer func() {
+		if recover() == nil {
+			panic("did not panic")
+		}
+	}()
+	f()
+}
+
+func isNonNil(x interface{}) {
+	if x == nil {
+		panic("nil interface")
+	}
+}
+
+func isValid(v Value) {
+	if !v.IsValid() {
+		panic("zero Value")
+	}
+}

src/pkg/reflect/deepequal.go

@@ -104,7 +104,7 @@ func deepValueEqual(v1, v2 Value, visited map[uintptr]*visit, depth int) (b bool
 		return true
 	default:
 		// Normal equality suffices
-		return v1.Interface() == v2.Interface()
+		return valueInterface(v1, false) == valueInterface(v2, false)
 	}
 
 	panic("Not reached")

src/pkg/reflect/type.go

@@ -10,6 +10,9 @@
 // A call to ValueOf returns a Value representing the run-time data.
 // Zero takes a Type and returns a Value representing a zero value
 // for that type.
+//
+// See "The Laws of Reflection" for an introduction to reflection in Go:
+// http://blog.golang.org/2011/09/laws-of-reflection.html
 package reflect
 
 import (

src/pkg/reflect/value.go

@@ -841,14 +841,7 @@ func (v Value) CanInterface() bool {
 	if iv.kind == Invalid {
 		panic(&ValueError{"reflect.Value.CanInterface", iv.kind})
 	}
-	// TODO(rsc): Check flagRO too.  Decide what to do about asking for
-	// interface for a value obtained via an unexported field.
-	// If the field were of a known type, say chan int or *sync.Mutex,
-	// the caller could interfere with the data after getting the
-	// interface.  But fmt.Print depends on being able to look.
-	// Now that reflect is more efficient the special cases in fmt
-	// might be less important.
-	return v.InternalMethod == 0
+	return v.InternalMethod == 0 && iv.flag&flagRO == 0
 }
 
 // Interface returns v's value as an interface{}.
@@ -856,22 +849,28 @@ func (v Value) CanInterface() bool {
 // (as opposed to Type.Method), Interface cannot return an
 // interface value, so it panics.
 func (v Value) Interface() interface{} {
-	return v.internal().Interface()
+	return valueInterface(v, true)
 }
 
-func (iv internalValue) Interface() interface{} {
+func valueInterface(v Value, safe bool) interface{} {
+	iv := v.internal()
+	return iv.valueInterface(safe)
+}
+
+func (iv internalValue) valueInterface(safe bool) interface{} {
 	if iv.kind == 0 {
 		panic(&ValueError{"reflect.Value.Interface", iv.kind})
 	}
 	if iv.method {
 		panic("reflect.Value.Interface: cannot create interface value for method with bound receiver")
 	}
-	/*
-		if v.flag()&noExport != 0 {
-			panic("reflect.Value.Interface: cannot return value obtained from unexported struct field")
-		}
-	*/
 
+	if safe && iv.flag&flagRO != 0 {
+		// Do not allow access to unexported values via Interface,
+		// because they might be pointers that should not be 
+		// writable or methods or function that should not be callable.
+		panic("reflect.Value.Interface: cannot return value obtained from unexported field or method")
+	}
 	if iv.kind == Interface {
 		// Special case: return the element inside the interface.
 		// Won't recurse further because an interface cannot contain an interface.
@@ -1700,7 +1699,7 @@ func convertForAssignment(what string, addr unsafe.Pointer, dst Type, iv interna
 		if addr == nil {
 			addr = unsafe.Pointer(new(interface{}))
 		}
-		x := iv.Interface()
+		x := iv.valueInterface(false)
 		if dst.NumMethod() == 0 {
 			*(*interface{})(addr) = x
 		} else {

src/pkg/runtime/hashmap.c

@@ -54,7 +54,7 @@ struct hash_subtable {
 	uint8 datasize;      /* bytes of client data in an entry */
 	uint8 max_probes;    /* max number of probes when searching */
 	int16 limit_bytes;	   /* max_probes * (datasize+sizeof (hash_hash_t)) */
-	struct hash_entry *end;      /* points just past end of entry[] */
+	struct hash_entry *last;      /* points to last element of entry[] */
 	struct hash_entry entry[1];  /* 2**power+max_probes-1 elements of elemsize bytes */
 };
 
@@ -101,7 +101,7 @@ hash_subtable_new (Hmap *h, int32 power, int32 used)
 	st->datasize = h->datasize;
 	st->max_probes = max_probes;
 	st->limit_bytes = limit_bytes;
-	st->end = HASH_OFFSET (st->entry, bytes);
+	st->last = HASH_OFFSET (st->entry, bytes) - 1;
 	memset (st->entry, HASH_NIL_MEMSET, bytes);
 	return (st);
 }
@@ -160,7 +160,7 @@ hash_remove_n (struct hash_subtable *st, struct hash_entry *dst_e, int32 n)
 {
 	int32 elemsize = st->datasize + offsetof (struct hash_entry, data[0]);
 	struct hash_entry *src_e = HASH_OFFSET (dst_e, n * elemsize);
-	struct hash_entry *end_e = st->end;
+	struct hash_entry *last_e = st->last;
 	int32 shift = HASH_BITS - (st->power + st->used);
 	int32 index_mask = (((hash_hash_t)1) << st->power) - 1;
 	int32 dst_i = (((byte *) dst_e) - ((byte *) st->entry)) / elemsize;
@@ -170,10 +170,10 @@ hash_remove_n (struct hash_subtable *st, struct hash_entry *dst_e, int32 n)
 	int32 bytes;
 
 	while (dst_e != src_e) {
-		if (src_e != end_e) {
+		if (src_e <= last_e) {
 			struct hash_entry *cp_e = src_e;
 			int32 save_dst_i = dst_i;
-			while (cp_e != end_e && (hash = cp_e->hash) != HASH_NIL &&
+			while (cp_e <= last_e && (hash = cp_e->hash) != HASH_NIL &&
 			     ((hash >> shift) & index_mask) <= dst_i) {
 				cp_e = HASH_OFFSET (cp_e, elemsize);
 				dst_i++;
@@ -183,7 +183,7 @@ hash_remove_n (struct hash_subtable *st, struct hash_entry *dst_e, int32 n)
 			dst_e = HASH_OFFSET (dst_e, bytes);
 			src_e = cp_e;
 			src_i += dst_i - save_dst_i;
-			if (src_e != end_e && (hash = src_e->hash) != HASH_NIL) {
+			if (src_e <= last_e && (hash = src_e->hash) != HASH_NIL) {
 				skip = ((hash >> shift) & index_mask) - dst_i;
 			} else {
 				skip = src_i - dst_i;
@@ -224,7 +224,7 @@ hash_conv (Hmap *h,
 	}
 
 	de = e;
-	while (e != st->end &&
+	while (e <= st->last &&
 	    (e_hash = e->hash) != HASH_NIL &&
 	    (e_hash & HASH_MASK) != HASH_SUBHASH) {
 		struct hash_entry *target_e = HASH_OFFSET (st->entry, ((e_hash >> shift) & index_mask) * elemsize);
@@ -235,14 +235,14 @@ hash_conv (Hmap *h,
 			de = target_e;
 		}
 		if ((hash & prefix_mask) == current ||
-		   (ne != st->end && (e_hash = ne->hash) != HASH_NIL &&
+		   (ne <= st->last && (e_hash = ne->hash) != HASH_NIL &&
 		   (e_hash & prefix_mask) == current)) {
 			struct hash_subtable *new_st = hash_subtable_new (h, 1, HASH_USED (new_flags));
 			int32 rc = hash_insert_internal (&new_st, new_flags, e->hash, h, e->data, &dummy_result);
 			assert (rc == 0);
 			memcpy(dummy_result, e->data, h->datasize);
 			e = ne;
-			while (e != st->end && (e_hash = e->hash) != HASH_NIL && (e_hash & prefix_mask) == current) {
+			while (e <= st->last && (e_hash = e->hash) != HASH_NIL && (e_hash & prefix_mask) == current) {
 				assert ((e_hash & HASH_MASK) != HASH_SUBHASH);
 				rc = hash_insert_internal (&new_st, new_flags, e_hash, h, e->data, &dummy_result);
 				assert (rc == 0);
@@ -271,13 +271,13 @@ hash_grow (Hmap *h, struct hash_subtable **pst, int32 flags)
 	struct hash_subtable *old_st = *pst;
 	int32 elemsize = h->datasize + offsetof (struct hash_entry, data[0]);
 	*pst = hash_subtable_new (h, old_st->power + 1, HASH_USED (flags));
-	struct hash_entry *end_e = old_st->end;
+	struct hash_entry *last_e = old_st->last;
 	struct hash_entry *e;
 	void *dummy_result;
 	int32 used = 0;
 
 	flags |= HASH_REHASH;
-	for (e = old_st->entry; e != end_e; e = HASH_OFFSET (e, elemsize)) {
+	for (e = old_st->entry; e <= last_e; e = HASH_OFFSET (e, elemsize)) {
 		hash_hash_t hash = e->hash;
 		if (hash != HASH_NIL) {
 			int32 rc = hash_insert_internal (pst, flags, e->hash, h, e->data, &dummy_result);
@@ -428,13 +428,13 @@ hash_insert_internal (struct hash_subtable **pst, int32 flags, hash_hash_t hash,
 			ins_e_hash = 0;
 			/* move ins_e to point at the end of the contiguous block, but
 			   stop if any element can't be moved by one up */
-			while (ins_e != st->end && (ins_e_hash = ins_e->hash) != HASH_NIL &&
+			while (ins_e <= st->last && (ins_e_hash = ins_e->hash) != HASH_NIL &&
 			       ins_i + 1 - ((ins_e_hash >> shift) & index_mask) < st->max_probes &&
 			       (ins_e_hash & HASH_MASK) != HASH_SUBHASH) {
 				ins_e = HASH_OFFSET (ins_e, elemsize);
 				ins_i++;
 			}
-			if (e == end_e || ins_e == st->end || ins_e_hash != HASH_NIL) {
+			if (e == end_e || ins_e > st->last || ins_e_hash != HASH_NIL) {
 				e = end_e;    /* can't insert; must grow or convert to subtable */
 			} else {	      /* make space for element */
 				memmove (HASH_OFFSET (e, elemsize), e, ((byte *) ins_e) - (byte *) e);
@@ -477,17 +477,17 @@ iter_restart (struct hash_iter *it, struct hash_subtable *st, int32 used)
 	struct hash_entry *e;
 	hash_hash_t e_hash;
 	struct hash_iter_sub *sub = &it->subtable_state[it->i];
-	struct hash_entry *end;
+	struct hash_entry *last;
 
 	for (;;) {
 		int32 shift = HASH_BITS - (st->power + used);
 		int32 index_mask = (1 << st->power) - 1;
 		int32 i = (last_hash >> shift) & index_mask;
 
-		end = st->end;
+		last = st->last;
 		e = HASH_OFFSET (st->entry, i * elemsize);
 		sub->start = st->entry;
-		sub->end = end;
+		sub->last = last;
 
 		if ((e->hash & HASH_MASK) != HASH_SUBHASH) {
 			break;
@@ -497,7 +497,7 @@ iter_restart (struct hash_iter *it, struct hash_subtable *st, int32 used)
 		used += st->power;
 		st = *(struct hash_subtable **)e->data;
 	}
-	while (e != end && ((e_hash = e->hash) == HASH_NIL || e_hash <= last_hash)) {
+	while (e <= last && ((e_hash = e->hash) == HASH_NIL || e_hash <= last_hash)) {
 		e = HASH_OFFSET (e, elemsize);
 	}
 	sub->e = e;
@@ -509,7 +509,7 @@ hash_next (struct hash_iter *it)
 	int32 elemsize = it->elemsize;
 	struct hash_iter_sub *sub = &it->subtable_state[it->i];
 	struct hash_entry *e = sub->e;
-	struct hash_entry *end = sub->end;
+	struct hash_entry *last = sub->last;
 	hash_hash_t e_hash = 0;
 
 	if (it->changes != it->h->changes) {    /* hash table's structure changed; recompute */
@@ -518,7 +518,7 @@ hash_next (struct hash_iter *it)
 		iter_restart (it, it->h->st, 0);
 		sub = &it->subtable_state[it->i];
 		e = sub->e;
-		end = sub->end;
+		last = sub->last;
 	}
 	if (e != sub->start && it->last_hash != HASH_OFFSET (e, -elemsize)->hash) {
 		struct hash_entry *start = HASH_OFFSET (e, -(elemsize * it->h->max_probes));
@@ -531,16 +531,16 @@ hash_next (struct hash_iter *it)
 			e = pe;
 			pe = HASH_OFFSET (pe, -elemsize);
 		}
-		while (e != end && ((e_hash = e->hash) == HASH_NIL || e_hash <= last_hash)) {
+		while (e <= last && ((e_hash = e->hash) == HASH_NIL || e_hash <= last_hash)) {
 			e = HASH_OFFSET (e, elemsize);
 		}
 	}
 
 	for (;;) {
-		while (e != end && (e_hash = e->hash) == HASH_NIL) {
+		while (e <= last && (e_hash = e->hash) == HASH_NIL) {
 			e = HASH_OFFSET (e, elemsize);
 		}
-		if (e == end) {
+		if (e > last) {
 			if (it->i == 0) {
 				it->last_hash = HASH_OFFSET (e, -elemsize)->hash;
 				sub->e = e;
@@ -549,7 +549,7 @@ hash_next (struct hash_iter *it)
 				it->i--;
 				sub = &it->subtable_state[it->i];
 				e = sub->e;
-				end = sub->end;
+				last = sub->last;
 			}
 		} else if ((e_hash & HASH_MASK) != HASH_SUBHASH) {
 			it->last_hash = e->hash;
@@ -565,7 +565,7 @@ hash_next (struct hash_iter *it)
 			sub = &it->subtable_state[it->i];
 			sub->e = e = st->entry;
 			sub->start = st->entry;
-			sub->end = end = st->end;
+			sub->last = last = st->last;
 		}
 	}
 }
@@ -580,7 +580,7 @@ hash_iter_init (Hmap *h, struct hash_iter *it)
 	it->last_hash = 0;
 	it->subtable_state[0].e = h->st->entry;
 	it->subtable_state[0].start = h->st->entry;
-	it->subtable_state[0].end = h->st->end;
+	it->subtable_state[0].last = h->st->last;
 }
 
 static void
@@ -588,11 +588,11 @@ clean_st (struct hash_subtable *st, int32 *slots, int32 *used)
 {
 	int32 elemsize = st->datasize + offsetof (struct hash_entry, data[0]);
 	struct hash_entry *e = st->entry;
-	struct hash_entry *end = st->end;
-	int32 lslots = (((byte *) end) - (byte *) e) / elemsize;
+	struct hash_entry *last = st->last;
+	int32 lslots = (((byte *) (last+1)) - (byte *) e) / elemsize;
 	int32 lused = 0;
 
-	while (e != end) {
+	while (e <= last) {
 		hash_hash_t hash = e->hash;
 		if ((hash & HASH_MASK) == HASH_SUBHASH) {
 			clean_st (*(struct hash_subtable **)e->data, slots, used);
@@ -627,7 +627,7 @@ hash_visit_internal (struct hash_subtable *st,
 	int32 shift = HASH_BITS - (used + st->power);
 	int32 i = 0;
 
-	while (e != st->end) {
+	while (e <= st->last) {
 		int32 index = ((e->hash >> (shift - 1)) >> 1) & ((1 << st->power) - 1);
 		if ((e->hash & HASH_MASK) == HASH_SUBHASH) {
 			  (*data_visit) (arg, level, e->data);

src/pkg/runtime/hashmap.h

@@ -87,7 +87,7 @@ struct hash_iter {
 	struct hash_iter_sub {
 		struct hash_entry *e;		/* pointer into subtable */
 		struct hash_entry *start;	/* start of subtable */
-		struct hash_entry *end;		/* end of subtable */
+		struct hash_entry *last;		/* last entry in subtable */
 	} subtable_state[4];	/* Should be large enough unless the hashing is
 				   so bad that many distinct data values hash
 				   to the same hash value.  */

src/pkg/runtime/runtime-gdb.py

@@ -91,8 +91,8 @@ class MapTypePrinter:
 
 	def traverse_hash(self, stab):
 		ptr = stab['entry'].address
-		end = stab['end']
-		while ptr < end:
+		last = stab['last']
+		while ptr <= last:
 			v = ptr.dereference()
 			ptr = ptr + 1
 			if v['hash'] == 0: continue

src/pkg/template/doc.go

@@ -294,7 +294,7 @@ This defines two templates, T1 and T2, and a third T3 that invokes the other two
 when it is executed.
 
 The second way to build a template set is to use Set's Add method to add a
-parsed template to a set.  A template may be bound at most one set.  If it's
+parsed template to a set.  A template may be bound to at most one set.  If it's
 necessary to have a template in multiple sets, the template definition must be
 parsed multiple times to create distinct *Template values.
 

src/pkg/template/exec.go

@@ -196,23 +196,25 @@ func (s *state) walkRange(dot reflect.Value, r *parse.RangeNode) {
 	val, _ := indirect(s.evalPipeline(dot, r.Pipe))
 	// mark top of stack before any variables in the body are pushed.
 	mark := s.mark()
+	oneIteration := func(index, elem reflect.Value) {
+		// Set top var (lexically the second if there are two) to the element.
+		if len(r.Pipe.Decl) > 0 {
+			s.setVar(1, elem)
+		}
+		// Set next var (lexically the first if there are two) to the index.
+		if len(r.Pipe.Decl) > 1 {
+			s.setVar(2, index)
+		}
+		s.walk(elem, r.List)
+		s.pop(mark)
+	}
 	switch val.Kind() {
 	case reflect.Array, reflect.Slice:
 		if val.Len() == 0 {
 			break
 		}
 		for i := 0; i < val.Len(); i++ {
-			elem := val.Index(i)
-			// Set top var (lexically the second if there are two) to the element.
-			if len(r.Pipe.Decl) > 0 {
-				s.setVar(1, elem)
-			}
-			// Set next var (lexically the first if there are two) to the index.
-			if len(r.Pipe.Decl) > 1 {
-				s.setVar(2, reflect.ValueOf(i))
-			}
-			s.walk(elem, r.List)
-			s.pop(mark)
+			oneIteration(reflect.ValueOf(i), val.Index(i))
 		}
 		return
 	case reflect.Map:
@@ -220,17 +222,23 @@ func (s *state) walkRange(dot reflect.Value, r *parse.RangeNode) {
 			break
 		}
 		for _, key := range val.MapKeys() {
-			elem := val.MapIndex(key)
-			// Set top var (lexically the second if there are two) to the element.
-			if len(r.Pipe.Decl) > 0 {
-				s.setVar(1, elem)
+			oneIteration(key, val.MapIndex(key))
+		}
+		return
+	case reflect.Chan:
+		if val.IsNil() {
+			break
+		}
+		i := 0
+		for ; ; i++ {
+			elem, ok := val.Recv()
+			if !ok {
+				break
 			}
-			// Set next var (lexically the first if there are two) to the key.
-			if len(r.Pipe.Decl) > 1 {
-				s.setVar(2, key)
-			}
-			s.walk(elem, r.List)
-			s.pop(mark)
+			oneIteration(reflect.ValueOf(i), elem)
+		}
+		if i == 0 {
+			break
 		}
 		return
 	case reflect.Invalid:
@@ -498,7 +506,18 @@ func (s *state) validateType(value reflect.Value, typ reflect.Type) reflect.Valu
 		s.errorf("invalid value; expected %s", typ)
 	}
 	if !value.Type().AssignableTo(typ) {
-		s.errorf("wrong type for value; expected %s; got %s", typ, value.Type())
+		// Does one dereference or indirection work? We could do more, as we
+		// do with method receivers, but that gets messy and method receivers
+		// are much more constrained, so it makes more sense there than here.
+		// Besides, one is almost always all you need.
+		switch {
+		case value.Kind() == reflect.Ptr && value.Elem().Type().AssignableTo(typ):
+			value = value.Elem()
+		case reflect.PtrTo(value.Type()).AssignableTo(typ) && value.CanAddr():
+			value = value.Addr()
+		default:
+			s.errorf("wrong type for value; expected %s; got %s", typ, value.Type())
+		}
 	}
 	return value
 }

src/pkg/template/exec_test.go

@@ -391,6 +391,8 @@ var execTests = []execTest{
 	{"range $x $y MSIone", "{{range $x, $y := .MSIone}}<{{$x}}={{$y}}>{{end}}", "<one=1>", tVal, true},
 	{"range $x PSI", "{{range $x := .PSI}}<{{$x}}>{{end}}", "<21><22><23>", tVal, true},
 	{"declare in range", "{{range $x := .PSI}}<{{$foo:=$x}}{{$x}}>{{end}}", "<21><22><23>", tVal, true},
+	{"range count", `{{range $i, $x := count 5}}[{{$i}}]{{$x}}{{end}}`, "[0]a[1]b[2]c[3]d[4]e", tVal, true},
+	{"range nil count", `{{range $i, $x := count 0}}{{else}}empty{{end}}`, "empty", tVal, true},
 
 	// Cute examples.
 	{"or as if true", `{{or .SI "slice is empty"}}`, "[3 4 5]", tVal, true},
@@ -414,6 +416,11 @@ var execTests = []execTest{
 	{"bug4", "{{if .Empty0}}non-nil{{else}}nil{{end}}", "nil", tVal, true},
 	// Stringer.
 	{"bug5", "{{.Str}}", "foozle", tVal, true},
+	// Args need to be indirected and dereferenced sometimes.
+	{"bug6a", "{{vfunc .V0 .V1}}", "vfunc", tVal, true},
+	{"bug6b", "{{vfunc .V0 .V0}}", "vfunc", tVal, true},
+	{"bug6c", "{{vfunc .V1 .V0}}", "vfunc", tVal, true},
+	{"bug6d", "{{vfunc .V1 .V1}}", "vfunc", tVal, true},
 }
 
 func zeroArgs() string {
@@ -424,9 +431,35 @@ func oneArg(a string) string {
 	return "oneArg=" + a
 }
 
+// count returns a channel that will deliver n sequential 1-letter strings starting at "a"
+func count(n int) chan string {
+	if n == 0 {
+		return nil
+	}
+	c := make(chan string)
+	go func() {
+		for i := 0; i < n; i++ {
+			c <- "abcdefghijklmnop"[i : i+1]
+		}
+		close(c)
+	}()
+	return c
+}
+
+// vfunc takes a *V and a V
+func vfunc(V, *V) string {
+	return "vfunc"
+}
+
 func testExecute(execTests []execTest, set *Set, t *testing.T) {
 	b := new(bytes.Buffer)
-	funcs := FuncMap{"zeroArgs": zeroArgs, "oneArg": oneArg, "typeOf": typeOf}
+	funcs := FuncMap{
+		"count":    count,
+		"oneArg":   oneArg,
+		"typeOf":   typeOf,
+		"vfunc":    vfunc,
+		"zeroArgs": zeroArgs,
+	}
 	for _, test := range execTests {
 		tmpl := New(test.name).Funcs(funcs)
 		_, err := tmpl.ParseInSet(test.input, set)

src/pkg/template/parse/lex.go

@@ -278,53 +278,51 @@ func lexInsideAction(l *lexer) stateFn {
 	// Either number, quoted string, or identifier.
 	// Spaces separate and are ignored.
 	// Pipe symbols separate and are emitted.
-	for {
-		if strings.HasPrefix(l.input[l.pos:], rightDelim) {
-			return lexRightDelim
-		}
-		switch r := l.next(); {
-		case r == eof || r == '\n':
-			return l.errorf("unclosed action")
-		case isSpace(r):
-			l.ignore()
-		case r == ':':
-			if l.next() != '=' {
-				return l.errorf("expected :=")
-			}
-			l.emit(itemColonEquals)
-		case r == '|':
-			l.emit(itemPipe)
-		case r == '"':
-			return lexQuote
-		case r == '`':
-			return lexRawQuote
-		case r == '$':
-			return lexIdentifier
-		case r == '\'':
-			return lexChar
-		case r == '.':
-			// special look-ahead for ".field" so we don't break l.backup().
-			if l.pos < len(l.input) {
-				r := l.input[l.pos]
-				if r < '0' || '9' < r {
-					return lexIdentifier // itemDot comes from the keyword table.
-				}
-			}
-			fallthrough // '.' can start a number.
-		case r == '+' || r == '-' || ('0' <= r && r <= '9'):
-			l.backup()
-			return lexNumber
-		case isAlphaNumeric(r):
-			l.backup()
-			return lexIdentifier
-		case r <= unicode.MaxASCII && unicode.IsPrint(r):
-			l.emit(itemChar)
-			return lexInsideAction
-		default:
-			return l.errorf("unrecognized character in action: %#U", r)
-		}
+	if strings.HasPrefix(l.input[l.pos:], rightDelim) {
+		return lexRightDelim
 	}
-	return nil
+	switch r := l.next(); {
+	case r == eof || r == '\n':
+		return l.errorf("unclosed action")
+	case isSpace(r):
+		l.ignore()
+	case r == ':':
+		if l.next() != '=' {
+			return l.errorf("expected :=")
+		}
+		l.emit(itemColonEquals)
+	case r == '|':
+		l.emit(itemPipe)
+	case r == '"':
+		return lexQuote
+	case r == '`':
+		return lexRawQuote
+	case r == '$':
+		return lexIdentifier
+	case r == '\'':
+		return lexChar
+	case r == '.':
+		// special look-ahead for ".field" so we don't break l.backup().
+		if l.pos < len(l.input) {
+			r := l.input[l.pos]
+			if r < '0' || '9' < r {
+				return lexIdentifier // itemDot comes from the keyword table.
+			}
+		}
+		fallthrough // '.' can start a number.
+	case r == '+' || r == '-' || ('0' <= r && r <= '9'):
+		l.backup()
+		return lexNumber
+	case isAlphaNumeric(r):
+		l.backup()
+		return lexIdentifier
+	case r <= unicode.MaxASCII && unicode.IsPrint(r):
+		l.emit(itemChar)
+		return lexInsideAction
+	default:
+		return l.errorf("unrecognized character in action: %#U", r)
+	}
+	return lexInsideAction
 }
 
 // lexIdentifier scans an alphanumeric or field.

src/pkg/template/parse/lex_test.go

@@ -184,6 +184,20 @@ var lexTests = []lexTest{
 		tLeft,
 		{itemError, `bad number syntax: "3k"`},
 	}},
+
+	// Fixed bugs
+	// Many elements in an action blew the lookahead until
+	// we made lexInsideAction not loop.
+	{"long pipeline deadlock", "{{|||||}}", []item{
+		tLeft,
+		tPipe,
+		tPipe,
+		tPipe,
+		tPipe,
+		tPipe,
+		tRight,
+		tEOF,
+	}},
 }
 
 // collect gathers the emitted items into a slice.

src/pkg/template/parse/node.go

@@ -35,8 +35,8 @@ const (
 	NodeBool                       // A boolean constant.
 	NodeCommand                    // An element of a pipeline.
 	NodeDot                        // The cursor, dot.
-	NodeElse                       // An else action.
-	NodeEnd                        // An end action.
+	nodeElse                       // An else action. Not added to tree.
+	nodeEnd                        // An end action. Not added to tree.
 	NodeField                      // A field or method name.
 	NodeIdentifier                 // An identifier; always a function name.
 	NodeIf                         // An if action.
@@ -356,36 +356,37 @@ func (s *StringNode) String() string {
 	return fmt.Sprintf("S=%#q", s.Text)
 }
 
-// EndNode represents an {{end}} action. It is represented by a nil pointer.
-type EndNode bool
+// endNode represents an {{end}} action. It is represented by a nil pointer.
+// It does not appear in the final parse tree.
+type endNode bool
 
-func newEnd() *EndNode {
+func newEnd() *endNode {
 	return nil
 }
 
-func (e *EndNode) Type() NodeType {
-	return NodeEnd
+func (e *endNode) Type() NodeType {
+	return nodeEnd
 }
 
-func (e *EndNode) String() string {
+func (e *endNode) String() string {
 	return "{{end}}"
 }
 
-// ElseNode represents an {{else}} action.
-type ElseNode struct {
+// elseNode represents an {{else}} action. Does not appear in the final tree.
+type elseNode struct {
 	NodeType
 	Line int // The line number in the input.
 }
 
-func newElse(line int) *ElseNode {
-	return &ElseNode{NodeType: NodeElse, Line: line}
+func newElse(line int) *elseNode {
+	return &elseNode{NodeType: nodeElse, Line: line}
 }
 
-func (e *ElseNode) Type() NodeType {
-	return NodeElse
+func (e *elseNode) Type() NodeType {
+	return nodeElse
 }
 
-func (e *ElseNode) String() string {
+func (e *elseNode) String() string {
 	return "{{else}}"
 }
 

src/pkg/template/parse/parse.go

@@ -173,7 +173,7 @@ func (t *Tree) itemList(toEOF bool) (list *ListNode, next Node) {
 	for t.peek().typ != itemEOF {
 		n := t.textOrAction()
 		switch n.Type() {
-		case NodeEnd, NodeElse:
+		case nodeEnd, nodeElse:
 			return list, n
 		}
 		list.append(n)
@@ -278,10 +278,10 @@ func (t *Tree) parseControl(context string) (lineNum int, pipe *PipeNode, list,
 	var next Node
 	list, next = t.itemList(false)
 	switch next.Type() {
-	case NodeEnd: //done
-	case NodeElse:
+	case nodeEnd: //done
+	case nodeElse:
 		elseList, next = t.itemList(false)
-		if next.Type() != NodeEnd {
+		if next.Type() != nodeEnd {
 			t.errorf("expected end; found %s", next)
 		}
 		elseList = elseList

src/pkg/template/parse/set.go

@@ -37,7 +37,7 @@ func Set(text string, funcs ...map[string]interface{}) (tree map[string]*Tree, e
 		if end == nil {
 			t.errorf("unexpected EOF in %s", context)
 		}
-		if end.Type() != NodeEnd {
+		if end.Type() != nodeEnd {
 			t.errorf("unexpected %s in %s", end, context)
 		}
 		t.stopParse()

test/dwarf/linedirectives.go

@@ -0,0 +1,83 @@
+// $G $D/$F.go && $L $F.$A && ./$A.out
+
+// Copyright 2011 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+//line foo/bar.y:4
+package main
+//line foo/bar.y:60
+func main() { 
+//line foo/bar.y:297
+	f, l := 0, 0
+//line yacctab:1
+	f, l = 1, 1
+//line yaccpar:1
+	f, l = 2, 1
+//line foo/bar.y:82
+	f, l = 3, 82
+//line foo/bar.y:90
+	f, l = 3, 90
+//line foo/bar.y:92
+	f, l = 3, 92
+//line foo/bar.y:100
+	f, l = 3, 100
+//line foo/bar.y:104
+	l = 104
+//line foo/bar.y:112
+	l = 112
+//line foo/bar.y:117
+	l = 117
+//line foo/bar.y:121
+	l = 121
+//line foo/bar.y:125
+	l = 125
+//line foo/bar.y:133
+	l = 133
+//line foo/bar.y:146
+	l = 146
+//line foo/bar.y:148
+//line foo/bar.y:153
+//line foo/bar.y:155
+	l = 155
+//line foo/bar.y:160
+
+//line foo/bar.y:164
+//line foo/bar.y:173
+
+//line foo/bar.y:178
+//line foo/bar.y:180
+//line foo/bar.y:185
+//line foo/bar.y:195
+//line foo/bar.y:197
+//line foo/bar.y:202
+//line foo/bar.y:204
+//line foo/bar.y:208
+//line foo/bar.y:211
+//line foo/bar.y:213
+//line foo/bar.y:215
+//line foo/bar.y:217
+//line foo/bar.y:221
+//line foo/bar.y:229
+//line foo/bar.y:236
+//line foo/bar.y:238
+//line foo/bar.y:240
+//line foo/bar.y:244
+//line foo/bar.y:249
+//line foo/bar.y:253
+//line foo/bar.y:257
+//line foo/bar.y:262
+//line foo/bar.y:267
+//line foo/bar.y:272
+	if l == f {
+//line foo/bar.y:277
+	panic("aie!")
+//line foo/bar.y:281
+	}
+//line foo/bar.y:285
+	return
+//line foo/bar.y:288
+//line foo/bar.y:290
+}
+//line foo/bar.y:293
+//line foo/bar.y:295

test/dwarf/main.go

@@ -0,0 +1,29 @@
+// $G $D/$F.go $D/z*.go && $L $F.$A && ./$A.out
+
+// Copyright 2009 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package main
+func main() {
+F1()
+F2()
+F3()
+F4()
+F5()
+F6()
+F7()
+F8()
+F9()
+F10()
+F11()
+F12()
+F13()
+F14()
+F15()
+F16()
+F17()
+F18()
+F19()
+F20()
+}

test/dwarf/z1.go

@@ -0,0 +1,5 @@
+
+
+//line x1.go:4
+package main
+func F1() {}

test/dwarf/z10.go

@@ -0,0 +1,6 @@
+
+
+
+//line x10.go:4
+package main
+func F10() {}

test/dwarf/z11.go

@@ -0,0 +1,4 @@
+
+//line x11.go:4
+package main
+func F11() {}

test/dwarf/z12.go

@@ -0,0 +1,4 @@
+
+//line x12.go:4
+package main
+func F12() {}

test/dwarf/z13.go

@@ -0,0 +1,4 @@
+
+//line x13.go:4
+package main
+func F13() {}

test/dwarf/z14.go

@@ -0,0 +1,4 @@
+
+//line x14.go:4
+package main
+func F14() {}

test/dwarf/z15.go

@@ -0,0 +1,4 @@
+
+//line x15.go:4
+package main
+func F15() {}

test/dwarf/z16.go

@@ -0,0 +1,4 @@
+
+//line x16.go:4
+package main
+func F16() {}

test/dwarf/z17.go

@@ -0,0 +1,4 @@
+
+//line x17.go:4
+package main
+func F17() {}

test/dwarf/z18.go

@@ -0,0 +1,5 @@
+
+
+//line x18.go:4
+package main
+func F18() {}

test/dwarf/z19.go

@@ -0,0 +1,4 @@
+
+//line x19.go:4
+package main
+func F19() {}

test/dwarf/z2.go

@@ -0,0 +1,4 @@
+
+//line x2.go:4
+package main
+func F2() {}

test/dwarf/z20.go

@@ -0,0 +1,4 @@
+
+//line x20.go:4
+package main
+func F20() {}

test/dwarf/z3.go

@@ -0,0 +1,4 @@
+
+//line x3.go:4
+package main
+func F3() {}

test/dwarf/z4.go

@@ -0,0 +1,4 @@
+
+//line x4.go:4
+package main
+func F4() {}

test/dwarf/z5.go

@@ -0,0 +1,4 @@
+
+//line x5.go:4
+package main
+func F5() {}

test/dwarf/z6.go

@@ -0,0 +1,4 @@
+
+//line x6.go:4
+package main
+func F6() {}

test/dwarf/z7.go

@@ -0,0 +1,4 @@
+
+//line x7.go:4
+package main
+func F7() {}

test/dwarf/z8.go

@@ -0,0 +1,4 @@
+
+//line x8.go:4
+package main
+func F8() {}

test/dwarf/z9.go

@@ -0,0 +1,4 @@
+
+//line x9.go:4
+package main
+func F9() {}

test/golden.out

@@ -124,6 +124,8 @@ panic: interface conversion: *main.S is not main.I2: missing method Name
 
 == syntax/
 
+== dwarf/
+
 == fixedbugs/
 
 =========== fixedbugs/bug027.go

test/interface/fake.go

@@ -12,20 +12,20 @@ package main
 import "reflect"
 
 type T struct {
-	f float32
-	g float32
+	F float32
+	G float32
 
-	s string
-	t string
+	S string
+	T string
 
-	u uint32
-	v uint32
+	U uint32
+	V uint32
 
-	w uint32
-	x uint32
+	W uint32
+	X uint32
 
-	y uint32
-	z uint32
+	Y uint32
+	Z uint32
 }
 
 func add(s, t string) string {
@@ -40,16 +40,16 @@ func assert(b bool) {
 
 func main() {
 	var x T
-	x.f = 1.0
-	x.g = x.f
-	x.s = add("abc", "def")
-	x.t = add("abc", "def")
-	x.u = 1
-	x.v = 2
-	x.w = 1 << 28
-	x.x = 2 << 28
-	x.y = 0x12345678
-	x.z = x.y
+	x.F = 1.0
+	x.G = x.F
+	x.S = add("abc", "def")
+	x.T = add("abc", "def")
+	x.U = 1
+	x.V = 2
+	x.W = 1 << 28
+	x.X = 2 << 28
+	x.Y = 0x12345678
+	x.Z = x.Y
 
 	// check mem and string
 	v := reflect.ValueOf(x)

test/run

@@ -53,7 +53,7 @@ filterout() {
 	grep '^'"$2"'$' $1 >/dev/null
 }
 
-for dir in . ken chan interface nilptr syntax fixedbugs bugs
+for dir in . ken chan interface nilptr syntax dwarf fixedbugs bugs
 do
 	echo
 	echo '==' $dir'/'