Compare commits

...

24 Commits

Author SHA1 Message Date
Gopher Robot
a10e42f219 [release-branch.go1.22] go1.22.0
Change-Id: If7b9fa19ee424d94e1c259e1867b89e8bda7fcde
Reviewed-on: https://go-review.googlesource.com/c/go/+/562276
Auto-Submit: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
2024-02-06 22:28:04 +00:00
Michael Matloob
b0957cfcf9 [release-branch.go1.22] cmd/go/internal/generate: call modload.InitWorkFile
This is necessary for go generate to enter workspace mode for
recognizing package paths in the workspace.

For #56098
Fixes #65352

Change-Id: I25f68de24f4189259353f63194823516e9d3d505
Reviewed-on: https://go-review.googlesource.com/c/go/+/559195
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
(cherry picked from commit b91bad7819)
Reviewed-on: https://go-review.googlesource.com/c/go/+/559236
2024-02-02 16:17:53 +00:00
Michael Anthony Knyszek
58a35fe55b [release-branch.go1.22] runtime: traceAcquire and traceRelease across all P steals
Currently there are a few places where a P can get stolen where the
runtime doesn't traceAcquire and traceRelease across the steal itself.
What can happen then is the following scenario:
- Thread 1 enters a syscall and writes an event about it.
- Thread 2 steals Thread 1's P.
- Thread 1 exits the syscall and writes one or more events about it.
- Tracing ends (trace.gen is set to 0).
- Thread 2 checks to see if it should write an event for the P it just
  stole, sees that tracing is disabled, and doesn't.

This results in broken traces, because there's a missing ProcSteal
event. The parser always waits for a ProcSteal to advance a
GoSyscallEndBlocked event, and in this case, it never comes.

Fixes #65181.

Change-Id: I437629499bb7669bf7fe2fc6fc4f64c53002916b
Reviewed-on: https://go-review.googlesource.com/c/go/+/560235
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit c9d88ea2aa)
Reviewed-on: https://go-review.googlesource.com/c/go/+/559958
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-02-01 22:54:02 +00:00
Michael Anthony Knyszek
4c5517913c [release-branch.go1.22] runtime: clear trace map without write barriers
Currently the trace map is cleared with an assignment, but this ends up
invoking write barriers. Theoretically, write barriers could try to
write a trace event and eventually try to acquire the same lock. The
static lock ranking expresses this constraint.

This change replaces the assignment with a call to memclrNoHeapPointer
to clear the map, removing the write barriers.

Note that technically this problem is purely theoretical. The way the
trace maps are used today is such that reset is only ever called when
the tracer is no longer writing events that could emit data into a map.
Furthermore, reset is never called from an event-writing context.

Therefore another way to resolve this is to simply not hold the trace
map lock over the reset operation. However, this makes the trace map
implementation less robust because it needs to be used in a very
specific way. Furthermore, the rest of the trace map code avoids write
barriers already since its internal structures are all notinheap, so
it's actually more consistent to just avoid write barriers in the reset
method.

Fixes #56554.

Change-Id: Icd86472e75e25161b2c10c1c8aaae2c2fed4f67f
Reviewed-on: https://go-review.googlesource.com/c/go/+/560216
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 829f2ce3ba)
Reviewed-on: https://go-review.googlesource.com/c/go/+/559957
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-02-01 22:50:10 +00:00
Michael Anthony Knyszek
5d647ed9fc [release-branch.go1.22] runtime: fix trace EvGoStop Gosched reason to match function
Currently the stop reason for runtime.Gosched is labeled
"runtime.GoSched" which doesn't actually match the function name. Fix
the label to match the function name.

This change doesn't regenerate the internal/trace/v2 tests, because
regenerating the tests breaks summarization tests in internal/trace that
rely on very specific details in the example traces that aren't
guaranteed. Also, go122-gc-trace.test isn't generated at all, as it
turns out. I'll fix this all up in a follow-up CL. For now, just replace
runtime.GoSched with runtime.Gosched in the traces so we don't have a
problem later if a test wants to look for that string.

This change does regenerate the cmd/trace/v2 test, but it turns out the
cmd/trace/v2 tests are way too strict about network unblock events, and
3 usually pop up instead of 1 or 2, which is what the test expects.
AFAICT this looks plausible to me, so just lift the restriction on
"up to 2" events entirely.

Change-Id: Id7350132be19119c743c259f2f5250903bf41a04
Reviewed-on: https://go-review.googlesource.com/c/go/+/552275
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
(cherry picked from commit 287f791845)
Reviewed-on: https://go-review.googlesource.com/c/go/+/560555
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-02-01 22:47:23 +00:00
Michael Anthony Knyszek
e34f6a9928 [release-branch.go1.22] runtime: model wakeableSleep.lock in the race detector
Currently the flight recorder tests are failing in race mode because the
race detector doesn't see s.lock, leading to false positives. This has
also appeared in the trace tests. Model the lock in the race detector.

Fixes #65207.
Fixes #65283.

Change-Id: I1e9a5c9606536f55fdfc46b5f8443e9c7213c23d
Reviewed-on: https://go-review.googlesource.com/c/go/+/560215
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
(cherry picked from commit 0b12e3d81c)
Reviewed-on: https://go-review.googlesource.com/c/go/+/559956
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-02-01 22:45:45 +00:00
Russ Cox
7b3786bbb1 [release-branch.go1.22] go/version: fix package to accept go1.21.0-bigcorp
The proposal discussion made clear that suffixes should be accepted,
so that people who use custom VERSION files can still pass runtime.Version()
to this code. But we forgot to do that in the CL. Do that.

Note that cmd/go also strips space- and tab-prefixed suffixes,
but go.dev/doc/toolchain only mentions dash, so this code only
strips dash.

Fixes #65061.

Change-Id: I6a427b78f964eb41c024890dae30223beaef13eb
Cq-Include-Trybots: luci.golang.try:go1.22-linux-amd64-longtest,go1.22-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/559796
TryBot-Bypass: Russ Cox <rsc@golang.org>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/559802
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-01 18:32:47 +00:00
Robert Griesemer
9289b9c336 [release-branch.go1.22] spec: clarify iteration variable type for range over integer
Also: report language version (plus date) in spec header.

For #65137.

Change-Id: I4f1d220d5922c40a36264df2d0a7bb7cd0756bac
Reviewed-on: https://go-review.googlesource.com/c/go/+/557596
TryBot-Bypass: Robert Griesemer <gri@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/559875
2024-01-31 21:40:40 +00:00
Bryan C. Mills
aa721d1e7d [release-branch.go1.22] cmd/go/internal/toolchain: apply the -modcacherw flag when downloading a module to determine what toolchain it needs
Fixes #64282.

Change-Id: I3f211c599ee70cb58254d0bc07eeb3c135124e58
Reviewed-on: https://go-review.googlesource.com/c/go/+/555436
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
(cherry picked from commit cc38c68ae0)
Reviewed-on: https://go-review.googlesource.com/c/go/+/559218
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-31 14:16:07 +00:00
Filippo Valsorda
117d7b107e [release-branch.go1.22] Revert "crypto/internal/boring: upgrade module to fips-20220613" +1
This reverts commit 7383b2a4db
("crypto/internal/boring: upgrade module to fips-20220613") and commit
4106de901a ("crypto/tls: align FIPS-only
mode with BoringSSL policy").

Fixes #65324
Updates #65321
Updates #64717
Updates #62372

Change-Id: I0938b97e5b4904e6532448b8ae76e920d03d0508
Reviewed-on: https://go-review.googlesource.com/c/go/+/558796
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 09b5de48e6)
Reviewed-on: https://go-review.googlesource.com/c/go/+/558797
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-01-29 22:19:45 +00:00
Cherry Mui
333ecd4b40 [release-branch.go1.22] Revert "archive/tar: add FileInfoNames interface"
This reverts CL 514235. Also reverts CL 518056 which is a followup
fix.

Reason for revert: Proposal #50102 defined an interface that is
too specific to UNIX-y systems and also didn't make much sense.
The proposal is un-accepted, and we'll revisit in Go 1.23.

Fixes #65245.
Updates #50102.

Change-Id: I41ba0ee286c1d893e6564a337e5d76418d19435d
Reviewed-on: https://go-review.googlesource.com/c/go/+/558295
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
(cherry picked from commit 5000b51680)
Reviewed-on: https://go-review.googlesource.com/c/go/+/558296
2024-01-24 21:35:52 +00:00
Gopher Robot
1e1da49105 [release-branch.go1.22] go1.22rc2
Change-Id: Iac7129fa56d739ead8bac461e178b0e23104bcc5
Reviewed-on: https://go-review.googlesource.com/c/go/+/558235
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-01-24 16:35:35 +00:00
Michael Anthony Knyszek
817009da40 [release-branch.go1.22] cmd/trace/v2: emit user log annotations in all views
This was an oversight in porting over cmd/trace to the new trace format
and API.

Fixes #65153.

Change-Id: I883d302f95956fcc9abb60aa53165acb6d099d67
Reviewed-on: https://go-review.googlesource.com/c/go/+/557175
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
(cherry picked from commit 7cb98c1da1)
Reviewed-on: https://go-review.googlesource.com/c/go/+/557817
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-01-23 19:30:57 +00:00
Michael Anthony Knyszek
b29ec30780 [release-branch.go1.22] runtime: make a much better effort to emit CPU profile in a trace
Currently the new execution tracer's handling of CPU profile samples is
very best-effort. The same CPU profile buffer is used across
generations, leading to a high probability that CPU samples will bleed
across generations. Also, because the CPU profile buffer (not the trace
buffer the samples get written into) isn't guaranteed to be flushed when
we close out a generation, nor when tracing stops. This has led to test
failures, but can more generally just lead to lost samples.

In general, lost samples are considered OK. The CPU profile buffer is
only read from every 100 ms, so if it fills up too much before then, old
samples will get overwritten. The tests already account for this, and in
that sense the CPU profile samples are already best-effort. But with
actual CPU profiles, this is really the only condition under which
samples are dropped.

This CL aims to align CPU profiles better with traces by eliminating
all best-effort parts of the implementation aside from the possibility
of dropped samples from a full buffer.

To achieve this, this CL adds a second CPU profile buffer and has the
SIGPROF handler pick which CPU profile buffer to use based on the
generation, much like every other part of the tracer. The SIGPROF
handler then reads the trace generation, but not before ensuring it
can't change: it grabs its own thread's trace seqlock. It's possible
that a SIGPROF signal lands while this seqlock is already held by the
thread. Luckily this is detectable and the SIGPROF handler can simply
elide the locking if this happens (the tracer will already wait until
all threads exit their seqlock critical section).

Now that there are two CPU profile buffers written to, the read side
needs to change. Instead of calling traceAcquire/traceRelease for every
single CPU sample event, the trace CPU profile reader goroutine holds
this conceptual lock over the entirety of flushing a buffer. This means
it can pick the CPU profile buffer for the current generation to flush.

With all this machinery in place, we're now at a point where all CPU
profile samples get divided into either the previous generation or the
current generation. This is good, since it means that we're able to
emit profile samples into the correct generation, avoiding surprises in
the final trace. All that's missing is to flush the CPU profile buffer
from the previous generation, once the runtime has moved on from that
generation. That is, when the generation counter updates, there may yet
be CPU profile samples sitting in the last generation's buffer. So,
traceCPUFlush now first flushes the CPU profile buffer, followed by any
trace buffers containing CPU profile samples.

The end result of all this is that no sample gets left behind unless it
gets overwritten in the CPU profile buffer in the first place. CPU
profile samples in the trace will now also get attributed to the right
generation, since the SIGPROF handler now participates in the tracer's
synchronization across trace generations.

Fixes #55317.

Change-Id: I47719fad164c544eef0bb12f99c8f3c15358e344
Reviewed-on: https://go-review.googlesource.com/c/go/+/555495
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
(cherry picked from commit f5e475edaf)
Reviewed-on: https://go-review.googlesource.com/c/go/+/557838
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-01-23 19:27:14 +00:00
qmuntal
10e9ab55c8 [release-branch.go1.22] cmd/cgo/internal/test: skip TestCallbackCallersSEH when internal linking
TestCallbackCallersSEH is flaky when using the internal linker. Skip
it for now until the flakiness is resolved.

Updates #65116

Change-Id: I7628b07eaff8be00757d5604722f30aede25fce5
Reviewed-on: https://go-review.googlesource.com/c/go/+/556635
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit adead1a93f)
Reviewed-on: https://go-review.googlesource.com/c/go/+/557815
2024-01-23 17:49:49 +00:00
Michael Anthony Knyszek
ba8e9e14f4 [release-branch.go1.22] runtime: use the correct M ID for syscalling goroutines in traces
Earlier in the development of the new tracer, m.id was used as a the
canonical ID for threads. Later, we switched to m.procid because it
matches the underlying OS resource. However, in that switch, we missed a
spot.

The tracer catches and emits statuses for goroutines that have remained
in either waiting or syscall across a whole generation, and emits a
thread ID for the latter set. The ID being used here, however, was m.id
instead of m.procid, like the rest of the tracer.

This CL also adds a regression test. In order to make the regression
test actually catch the failure, we also have to make the parser a
little less lenient about GoStatus events with GoSyscall: if this isn't
the first generation, then we should've seen the goroutine bound to an
M already when its status is getting emitted for its context. If we emit
the wrong ID, then we'll catch the issue when we emit the right ID when
the goroutine exits the syscall.

Fixes #65196.

Change-Id: I78b64fbea65308de5e1291c478a082a732a8bf9f
Reviewed-on: https://go-review.googlesource.com/c/go/+/557456
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit c46966653f)
Reviewed-on: https://go-review.googlesource.com/c/go/+/557436
2024-01-22 22:50:43 +00:00
Dmitri Shuralyov
77e9c26960 [release-branch.go1.22] doc: delete go1.22.html copy
The motivation is the same as in the commit message of CL 511317.

For #61422.

Change-Id: I0e86cf35ec3501a931d6d7fffb0c83f3e57106e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/557515
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-22 17:55:34 +00:00
Carlos Amedee
fe55bbcfd1 [release-branch.go1.22] doc: remove last TODO item
For #61422

Change-Id: I50e427b78a533c3196aeb5291a34c05528ee0bed
Reviewed-on: https://go-review.googlesource.com/c/go/+/557475
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-22 17:02:41 +00:00
Carlos Amedee
9a70e17e0f [release-branch.go1.22] all: merge master into release-branch.go1.22
Change-Id: Icc6bf5dba9eae08fa3e4b63414f409727efb5197
2024-01-19 14:26:02 -05:00
Dmitri Shuralyov
66f8e1e817 [release-branch.go1.22] all: merge master (8eaa793) into release-branch.go1.22
Merge List:

+ 2024-01-08 8eaa7935db net: clarify maxListenerBacklog windows implementation
+ 2024-01-08 759849187f log/slog: clarify SetDefault behavior
+ 2024-01-08 10a66d6816 sync: use map[any]any instead of map[interface{}]interface{} in the godoc
+ 2024-01-08 881869dde0 cmd/compile: handle defined iter func type correctly
+ 2024-01-05 1ae729e6d3 doc: s/adjustements/adjustments
+ 2024-01-05 8088b6db23 slices: explicitly discard results of some functions
+ 2024-01-05 c0693f648a cmd/go: run cover tool before swig
+ 2024-01-04 8db131082d github: switch seen/expected order in issue forms
+ 2024-01-04 ead47b0ab3 net/http: respond with 400 Bad Request for empty hex number of chunk length
+ 2024-01-04 1e07c144c3 net/http/cgi: in TestCopyError, check for a Handler.ServeHTTP goroutine instead of a running PID
+ 2024-01-04 15dcdeb5aa cmd/api: fix panic on exported basic type aliases
+ 2024-01-03 6db1102605 pagetrace: fix build when experiment is on
+ 2024-01-03 7d1b82dbf1 net/http: make Request.Clone create fresh copies for matches and otherValues
+ 2024-01-03 aa0a6ad1db doc/go1.22: add links to go/types symbols
+ 2024-01-03 c95fe91d07 runtime: correct scavengeIndex.sysGrow min index handling
+ 2023-12-30 b25f5558c6 all: replace outdated links
+ 2023-12-30 3233542e85 reflect: fix typo in type.go
+ 2023-12-27 988b718f41 doc: fix typo in example in spec
+ 2023-12-27 26ba75fe59 doc: document new iteration variable semantics in spec
+ 2023-12-27 1dddd83c49 doc: document version at which new language features were introduced in spec
+ 2023-12-26 36a2463e7c lib/time: update to 2023d/2023d
+ 2023-12-21 2184a39477 runtime/metrics: godoc link fixes
+ 2023-12-21 9c01ecce48 doc: fix misspelling in go1.22 release notes
+ 2023-12-21 0b56804084 runtime: use racereleasemerge for godebugInc
+ 2023-12-21 f6509cf5cd cmd/compile: handle constant-folding of an out-of-range jump table index
+ 2023-12-20 adec22b9f7 doc/go1.22: document changes to vet loopclosure analyzer
+ 2023-12-20 a2a2c5b947 doc: update unsafe.Pointer rule in spec
+ 2023-12-19 35222eeb78 doc: add html/template release note

Change-Id: I0709455dc512c94d2cbdb15c10d5c91f4bdaaa71
2024-01-08 15:05:46 -05:00
Gopher Robot
fa72f3e034 [release-branch.go1.22] go1.22rc1
Change-Id: Ia3eeb58ffd7acc4dc519a304f3ef934ab9c82175
Reviewed-on: https://go-review.googlesource.com/c/go/+/551536
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2023-12-19 20:59:26 +00:00
Than McIntosh
fb23428a85 [release-branch.go1.22] all: merge master (0324250) into release-branch.go1.22
Merge List:

+ 2023-12-19 03242506de doc: comment out remaining TODOs in Go 1.22 relnotes (for now)
+ 2023-12-19 9dd1cde9ac doc/go1.22,cmd/go: document that 'go mod init' no longer imports from other vendoring tools
+ 2023-12-19 22284c34f2 doc/go1.22: document removal of 'go get' support in GOPATH mode
+ 2023-12-19 339177aa31 doc: typo fix for net/http.ServeMux
+ 2023-12-19 52dbffeac8 cmd/go/internal/toolchain: revert "make a best effort to parse 'go run' and 'go install' flags"

Change-Id: I102e8267373364d0ad6170d36442d19048268765
2023-12-19 10:57:36 -05:00
Than McIntosh
f06eaf0c4f [release-branch.go1.22] all: merge master (1d4b0b6) into release-branch.go1.22
Merge List:

+ 2023-12-19 1d4b0b6236 doc/go1.22.html: release notes for slog, testing/slogtest and net/http.ServeMux
+ 2023-12-19 6fe0d3758b cmd/compile: remove interfacecycles debug flag
+ 2023-12-19 90daaa0576 doc/go1.22: announcing support address sanitizer on Loong64
+ 2023-12-18 5b84d50038 test: skip rangegen.go on 32-bit platforms
+ 2023-12-18 4106de901a crypto/tls: align FIPS-only mode with BoringSSL policy
+ 2023-12-18 7383b2a4db crypto/internal/boring: upgrade module to fips-20220613
+ 2023-12-18 c564d4ae08 Revert "cmd/cgo/internal/testsanitizers: fix msan test failing with clang >= 16"
+ 2023-12-18 7058f09a8b cmd: go get golang.org/x/tools@83bceaf2 and revendor
+ 2023-12-18 761e10be88 cmd/link/internal/loadpe: update comment about @feat.00 symbol handling
+ 2023-12-18 450f5d90c2 doc: add math/rand/v2 release notes
+ 2023-12-18 08bec0db39 builtin: mention PanicNilError in comments of recover
+ 2023-12-18 2acbdd086d cmd/cgo/internal/testsanitizers: fix msan test failing with clang >= 16
+ 2023-12-18 a7097243e4 internal/syscall/windows: fix the signature of SetFileInformationByHandle
+ 2023-12-18 8e3930f258 runtime: skip TestRuntimeLockMetricsAndProfile for flakiness
+ 2023-12-15 9b4b3e5acc runtime: properly model rwmutex in lock ranking
+ 2023-12-15 793097161b all: fix copyright headers
+ 2023-12-15 f8170cc017 cmd/asm: for arm, rewrite argument shifted right by 0 to left by 0.
+ 2023-12-15 3313bbb405 runtime: add race annotations in IncNonDefault
+ 2023-12-15 b60bf8f8e1 cmd/asm: fix encoding for arm right shift by constant 0
+ 2023-12-15 5e939b3a9c doc: add crypto/tls and crypto/x509 release notes

Change-Id: I2a80e9d39aa1fbb22b06ecfa16f725bacb78eb3f
2023-12-19 09:29:26 -05:00
Michael Pratt
796f59df92 [release-branch.go1.22] update codereview.cfg for release-branch.go1.22
Change-Id: I3f5cd8d62cfb1e62240e2b9f0b22022ee57262a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/550255
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
2023-12-15 18:08:51 +00:00
52 changed files with 5111 additions and 5680 deletions

2
VERSION Normal file
View File

@@ -0,0 +1,2 @@
go1.22.0
time 2024-02-02T18:09:55Z

View File

@@ -1,13 +1,4 @@
pkg archive/tar, method (*Writer) AddFS(fs.FS) error #58000
pkg archive/tar, type FileInfoNames interface { Gname, IsDir, ModTime, Mode, Name, Size, Sys, Uname } #50102
pkg archive/tar, type FileInfoNames interface, Gname(int) (string, error) #50102
pkg archive/tar, type FileInfoNames interface, IsDir() bool #50102
pkg archive/tar, type FileInfoNames interface, ModTime() time.Time #50102
pkg archive/tar, type FileInfoNames interface, Mode() fs.FileMode #50102
pkg archive/tar, type FileInfoNames interface, Name() string #50102
pkg archive/tar, type FileInfoNames interface, Size() int64 #50102
pkg archive/tar, type FileInfoNames interface, Sys() interface{} #50102
pkg archive/tar, type FileInfoNames interface, Uname(int) (string, error) #50102
pkg archive/zip, method (*Writer) AddFS(fs.FS) error #54898
pkg cmp, func Or[$0 comparable](...$0) $0 #60204
pkg crypto/x509, func OIDFromInts([]uint64) (OID, error) #60665

View File

@@ -1 +1,2 @@
branch: master
branch: release-branch.go1.22
parent-branch: master

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
<!--{
"Title": "The Go Programming Language Specification",
"Subtitle": "Version of Dec 27, 2023",
"Subtitle": "Language version go1.22 (Jan 30, 2023)",
"Path": "/ref/spec"
}-->
@@ -6661,7 +6661,7 @@ array or slice a [n]E, *[n]E, or []E index i int a[i] E
string s string type index i int see below rune
map m map[K]V key k K m[k] V
channel c chan E, &lt;-chan E element e E
integer n integer type I value i I
integer n integer type value i see below
</pre>
<ol>
@@ -6703,26 +6703,33 @@ is <code>nil</code>, the range expression blocks forever.
<li>
For an integer value <code>n</code>, the iteration values 0 through <code>n-1</code>
are produced in increasing order, with the same type as <code>n</code>.
are produced in increasing order.
If <code>n</code> &lt= 0, the loop does not run any iterations.
</li>
</ol>
<p>
The iteration values are assigned to the respective
iteration variables as in an <a href="#Assignment_statements">assignment statement</a>.
</p>
<p>
The iteration variables may be declared by the "range" clause using a form of
<a href="#Short_variable_declarations">short variable declaration</a>
(<code>:=</code>).
In this case their types are set to the types of the respective iteration values
and their <a href="#Declarations_and_scope">scope</a> is the block of the "for" statement;
each iteration has its own separate variables [<a href="#Go_1.22">Go 1.22</a>]
In this case their <a href="#Declarations_and_scope">scope</a> is the block of the "for" statement
and each iteration has its own new variables [<a href="#Go_1.22">Go 1.22</a>]
(see also <a href="#For_clause">"for" statements with a ForClause</a>).
If the iteration variables are declared outside the “for” statement,
after execution their values will be those of the last iteration.
If the range expression is a (possibly untyped) integer expression <code>n</code>,
the variable has the same type as if it was
<a href="#Variable_declarations">declared</a> with initialization
expression <code>n</code>.
Otherwise, the variables have the types of their respective iteration values.
</p>
<p>
If the iteration variables are not explicitly declared by the "range" clause,
they must be preexisting.
In this case, the iteration values are assigned to the respective variables
as in an <a href="#Assignment_statements">assignment statement</a>.
If the range expression is a (possibly untyped) integer expression <code>n</code>,
<code>n</code> too must be <a href="#Assignability">assignable</a> to the iteration variable;
if there is no iteration variable, <code>n</code> must be assignable to <code>int</code>.
</p>
<pre>
@@ -6765,6 +6772,11 @@ for i := range 10 {
// type of i is int (default type for untyped constant 10)
f(i)
}
// invalid: 256 cannot be assigned to uint8
var u uint8
for u = range 256 {
}
</pre>

View File

@@ -614,8 +614,6 @@ func (fi headerFileInfo) String() string {
// sysStat, if non-nil, populates h from system-dependent fields of fi.
var sysStat func(fi fs.FileInfo, h *Header) error
var loadUidAndGid func(fi fs.FileInfo, uid, gid *int)
const (
// Mode constants from the USTAR spec:
// See http://pubs.opengroup.org/onlinepubs/9699919799/utilities/pax.html#tag_20_92_13_06
@@ -641,10 +639,6 @@ const (
// Since fs.FileInfo's Name method only returns the base name of
// the file it describes, it may be necessary to modify Header.Name
// to provide the full path name of the file.
//
// If fi implements [FileInfoNames]
// the Gname and Uname of the header are
// provided by the methods of the interface.
func FileInfoHeader(fi fs.FileInfo, link string) (*Header, error) {
if fi == nil {
return nil, errors.New("archive/tar: FileInfo is nil")
@@ -717,38 +711,12 @@ func FileInfoHeader(fi fs.FileInfo, link string) (*Header, error) {
}
}
}
if iface, ok := fi.(FileInfoNames); ok {
var err error
if loadUidAndGid != nil {
loadUidAndGid(fi, &h.Uid, &h.Gid)
}
h.Gname, err = iface.Gname(h.Gid)
if err != nil {
return nil, err
}
h.Uname, err = iface.Uname(h.Uid)
if err != nil {
return nil, err
}
return h, nil
}
if sysStat != nil {
return h, sysStat(fi, h)
}
return h, nil
}
// FileInfoNames extends [FileInfo] to translate UID/GID to names.
// Passing an instance of this to [FileInfoHeader] permits the caller
// to control UID/GID resolution.
type FileInfoNames interface {
fs.FileInfo
// Uname should translate a UID into a user name.
Uname(uid int) (string, error)
// Gname should translate a GID into a group name.
Gname(gid int) (string, error)
}
// isHeaderOnlyType checks if the given type flag is of the type that has no
// data section even if a size is specified.
func isHeaderOnlyType(flag byte) bool {

View File

@@ -17,7 +17,6 @@ import (
func init() {
sysStat = statUnix
loadUidAndGid = loadUidAndGidFunc
}
// userMap and groupMap caches UID and GID lookups for performance reasons.
@@ -100,12 +99,3 @@ func statUnix(fi fs.FileInfo, h *Header) error {
}
return nil
}
func loadUidAndGidFunc(fi fs.FileInfo, uid, gid *int) {
sys, ok := fi.Sys().(*syscall.Stat_t)
if !ok {
return
}
*uid = int(sys.Uid)
*gid = int(sys.Gid)
}

View File

@@ -848,71 +848,3 @@ func Benchmark(b *testing.B) {
})
}
const (
testUid = 10
testGid = 20
)
type fileInfoNames struct{}
func (f *fileInfoNames) Name() string {
return "tmp"
}
func (f *fileInfoNames) Size() int64 {
return 0
}
func (f *fileInfoNames) Mode() fs.FileMode {
return 0777
}
func (f *fileInfoNames) ModTime() time.Time {
return time.Time{}
}
func (f *fileInfoNames) IsDir() bool {
return false
}
func (f *fileInfoNames) Sys() any {
return nil
}
func (f *fileInfoNames) Uname(uid int) (string, error) {
if uid == testUid {
return "Uname", nil
}
return "", nil
}
func (f *fileInfoNames) Gname(gid int) (string, error) {
if gid == testGid {
return "Gname", nil
}
return "", nil
}
func TestFileInfoHeaderUseFileInfoNames(t *testing.T) {
origLoadUidAndGid := loadUidAndGid
defer func() {
loadUidAndGid = origLoadUidAndGid
}()
loadUidAndGid = func(fi fs.FileInfo, uid, gid *int) {
*uid = testUid
*gid = testGid
}
info := &fileInfoNames{}
header, err := FileInfoHeader(info, "")
if err != nil {
t.Fatal(err)
}
if header.Uname != "Uname" {
t.Fatalf("header.Uname: got %v, want %v", header.Uname, "Uname")
}
if header.Gname != "Gname" {
t.Fatalf("header.Gname: got %v, want %v", header.Gname, "Gname")
}
}

View File

@@ -0,0 +1,16 @@
// Copyright 2024 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build cgo && windows && internal
package cgotest
import (
"internal/testenv"
"testing"
)
func TestCallbackCallersSEH(t *testing.T) {
testenv.SkipFlaky(t, 65116)
}

View File

@@ -2,7 +2,7 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build cgo && windows
//go:build cgo && windows && !internal
package cgotest

View File

@@ -181,6 +181,8 @@ func init() {
}
func runGenerate(ctx context.Context, cmd *base.Command, args []string) {
modload.InitWorkfile()
if generateRunFlag != "" {
var err error
generateRunRE, err = regexp.Compile(generateRunFlag)

View File

@@ -8,6 +8,7 @@ package toolchain
import (
"context"
"errors"
"flag"
"fmt"
"go/build"
"io/fs"
@@ -24,6 +25,7 @@ import (
"cmd/go/internal/modfetch"
"cmd/go/internal/modload"
"cmd/go/internal/run"
"cmd/go/internal/work"
"golang.org/x/mod/module"
)
@@ -486,74 +488,132 @@ func goInstallVersion() bool {
// Note: We assume there are no flags between 'go' and 'install' or 'run'.
// During testing there are some debugging flags that are accepted
// in that position, but in production go binaries there are not.
if len(os.Args) < 3 || (os.Args[1] != "install" && os.Args[1] != "run") {
if len(os.Args) < 3 {
return false
}
// Check for pkg@version.
var arg string
var cmdFlags *flag.FlagSet
switch os.Args[1] {
default:
// Command doesn't support a pkg@version as the main module.
return false
case "install":
// We would like to let 'go install -newflag pkg@version' work even
// across a toolchain switch. To make that work, assume the pkg@version
// is the last argument and skip the flag parsing.
arg = os.Args[len(os.Args)-1]
cmdFlags = &work.CmdInstall.Flag
case "run":
// For run, the pkg@version can be anywhere on the command line,
// because it is preceded by run flags and followed by arguments to the
// program being run. To handle that precisely, we have to interpret the
// flags a little bit, to know whether each flag takes an optional argument.
// We can still allow unknown flags as long as they have an explicit =value.
args := os.Args[2:]
for i := 0; i < len(args); i++ {
a := args[i]
if !strings.HasPrefix(a, "-") {
arg = a
break
}
if a == "-" {
// non-flag but also non-pkg@version
cmdFlags = &run.CmdRun.Flag
}
// The modcachrw flag is unique, in that it affects how we fetch the
// requested module to even figure out what toolchain it needs.
// We need to actually set it before we check the toolchain version.
// (See https://go.dev/issue/64282.)
modcacherwFlag := cmdFlags.Lookup("modcacherw")
if modcacherwFlag == nil {
base.Fatalf("internal error: modcacherw flag not registered for command")
}
modcacherwVal, ok := modcacherwFlag.Value.(interface {
IsBoolFlag() bool
flag.Value
})
if !ok || !modcacherwVal.IsBoolFlag() {
base.Fatalf("internal error: modcacherw is not a boolean flag")
}
// Make a best effort to parse the command's args to find the pkg@version
// argument and the -modcacherw flag.
var (
pkgArg string
modcacherwSeen bool
)
for args := os.Args[2:]; len(args) > 0; {
a := args[0]
args = args[1:]
if a == "--" {
if len(args) == 0 {
return false
}
if a == "--" {
if i+1 >= len(args) {
return false
pkgArg = args[0]
break
}
a, ok := strings.CutPrefix(a, "-")
if !ok {
// Not a flag argument. Must be a package.
pkgArg = a
break
}
a = strings.TrimPrefix(a, "-") // Treat --flag as -flag.
name, val, hasEq := strings.Cut(a, "=")
if name == "modcacherw" {
if !hasEq {
val = "true"
}
if err := modcacherwVal.Set(val); err != nil {
return false
}
modcacherwSeen = true
continue
}
if hasEq {
// Already has a value; don't bother parsing it.
continue
}
f := run.CmdRun.Flag.Lookup(a)
if f == nil {
// We don't know whether this flag is a boolean.
if os.Args[1] == "run" {
// We don't know where to find the pkg@version argument.
// For run, the pkg@version can be anywhere on the command line,
// because it is preceded by run flags and followed by arguments to the
// program being run. Since we don't know whether this flag takes
// an argument, we can't reliably identify the end of the run flags.
// Just give up and let the user clarify using the "=" form..
return false
}
// We would like to let 'go install -newflag pkg@version' work even
// across a toolchain switch. To make that work, assume by default that
// the pkg@version is the last argument and skip the remaining args unless
// we spot a plausible "-modcacherw" flag.
for len(args) > 0 {
a := args[0]
name, _, _ := strings.Cut(a, "=")
if name == "-modcacherw" || name == "--modcacherw" {
break
}
arg = args[i+1]
break
if len(args) == 1 && !strings.HasPrefix(a, "-") {
pkgArg = a
}
args = args[1:]
}
a = strings.TrimPrefix(a, "-")
a = strings.TrimPrefix(a, "-")
if strings.HasPrefix(a, "-") {
// non-flag but also non-pkg@version
return false
}
if strings.Contains(a, "=") {
// already has value
continue
}
f := run.CmdRun.Flag.Lookup(a)
if f == nil {
// Unknown flag. Give up. The command is going to fail in flag parsing.
return false
}
if bf, ok := f.Value.(interface{ IsBoolFlag() bool }); ok && bf.IsBoolFlag() {
// Does not take value.
continue
}
i++ // Does take a value; skip it.
continue
}
if bf, ok := f.Value.(interface{ IsBoolFlag() bool }); !ok || !bf.IsBoolFlag() {
// The next arg is the value for this flag. Skip it.
args = args[1:]
continue
}
}
if !strings.Contains(arg, "@") || build.IsLocalImport(arg) || filepath.IsAbs(arg) {
if !strings.Contains(pkgArg, "@") || build.IsLocalImport(pkgArg) || filepath.IsAbs(pkgArg) {
return false
}
path, version, _ := strings.Cut(arg, "@")
path, version, _ := strings.Cut(pkgArg, "@")
if path == "" || version == "" || gover.IsToolchain(path) {
return false
}
if !modcacherwSeen && base.InGOFLAGS("-modcacherw") {
fs := flag.NewFlagSet("goInstallVersion", flag.ExitOnError)
fs.Var(modcacherwVal, "modcacherw", modcacherwFlag.Usage)
base.SetFromGOFLAGS(fs)
}
// It would be correct to simply return true here, bypassing use
// of the current go.mod or go.work, and let "go run" or "go install"
// do the rest, including a toolchain switch.

View File

@@ -0,0 +1,27 @@
# This is a regression test for Issue #56098: Go generate
# wasn't initializing workspace mode
[short] skip
go generate ./mod
cmp ./mod/got.txt want.txt
-- go.work --
go 1.22
use ./mod
-- mod/go.mod --
module example.com/mod
-- mod/gen.go --
//go:generate go run gen.go got.txt
package main
import "os"
func main() {
outfile := os.Args[1]
os.WriteFile(outfile, []byte("Hello World!\n"), 0644)
}
-- want.txt --
Hello World!

View File

@@ -0,0 +1,45 @@
# Regression test for https://go.dev/issue/64282.
#
# 'go install' and 'go run' with pkg@version arguments should make
# a best effort to parse flags relevant to downloading modules
# (currently only -modcacherw) before actually downloading the module
# to identify which toolchain version to use.
#
# However, the best-effort flag parsing should not interfere with
# actual flag parsing if we don't switch toolchains. In particular,
# unrecognized flags should still be diagnosed after the module for
# the requested package has been downloaded and checked for toolchain
# upgrades.
! go install -cake=delicious -modcacherw example.com/printversion@v0.1.0
stderr '^flag provided but not defined: -cake$'
# Because the -modcacherw flag was set, we should be able to modify the contents
# of a directory within the module cache.
cp $WORK/extraneous.txt $GOPATH/pkg/mod/example.com/printversion@v0.1.0/extraneous_file.go
go clean -modcache
! go install -unknownflag -tags -modcacherw example.com/printversion@v0.1.0
stderr '^flag provided but not defined: -unknownflag$'
cp $WORK/extraneous.txt $GOPATH/pkg/mod/example.com/printversion@v0.1.0/extraneous_file.go
go clean -modcache
# Also try it with a 'go install' that succeeds.
# (But skip in short mode, because linking a binary is expensive.)
[!short] go install -modcacherw example.com/printversion@v0.1.0
[!short] cp $WORK/extraneous.txt $GOPATH/pkg/mod/example.com/printversion@v0.1.0/extraneous_file.go
[!short] go clean -modcache
# The flag should also be applied if given in GOFLAGS
# instead of on the command line.
env GOFLAGS=-modcacherw
! go install -cake=delicious example.com/printversion@v0.1.0
stderr '^flag provided but not defined: -cake$'
cp $WORK/extraneous.txt $GOPATH/pkg/mod/example.com/printversion@v0.1.0/extraneous_file.go
-- $WORK/extraneous.txt --
This is not a Go source file.

View File

@@ -31,6 +31,9 @@ type generator interface {
ProcRange(ctx *traceContext, ev *tracev2.Event)
ProcTransition(ctx *traceContext, ev *tracev2.Event)
// User annotations.
Log(ctx *traceContext, ev *tracev2.Event)
// Finish indicates the end of the trace and finalizes generation.
Finish(ctx *traceContext)
}
@@ -69,6 +72,8 @@ func runGenerator(ctx *traceContext, g generator, parsed *parsedTrace, opts *gen
case tracev2.ResourceGoroutine:
g.GoroutineTransition(ctx, ev)
}
case tracev2.EventLog:
g.Log(ctx, ev)
}
}
for i, task := range opts.tasks {
@@ -357,3 +362,33 @@ type completedRange struct {
endStack tracev2.Stack
arg any
}
type logEventGenerator[R resource] struct {
// getResource is a function to extract a resource ID from a Log event.
getResource func(*tracev2.Event) R
}
// Log implements a log event handler. It expects ev to be one such event.
func (g *logEventGenerator[R]) Log(ctx *traceContext, ev *tracev2.Event) {
id := g.getResource(ev)
if id == R(noResource) {
// We have nowhere to put this in the UI.
return
}
// Construct the name to present.
log := ev.Log()
name := log.Message
if log.Category != "" {
name = "[" + log.Category + "] " + name
}
// Emit an instant event.
ctx.Instant(traceviewer.InstantEvent{
Name: name,
Ts: ctx.elapsed(ev.Time()),
Category: "user event",
Resource: uint64(id),
Stack: ctx.Stack(viewerFrames(ev.Stack())),
})
}

View File

@@ -14,6 +14,7 @@ type goroutineGenerator struct {
globalRangeGenerator
globalMetricGenerator
stackSampleGenerator[tracev2.GoID]
logEventGenerator[tracev2.GoID]
gStates map[tracev2.GoID]*gState[tracev2.GoID]
focus tracev2.GoID
@@ -22,9 +23,11 @@ type goroutineGenerator struct {
func newGoroutineGenerator(ctx *traceContext, focus tracev2.GoID, filter map[tracev2.GoID]struct{}) *goroutineGenerator {
gg := new(goroutineGenerator)
gg.stackSampleGenerator.getResource = func(ev *tracev2.Event) tracev2.GoID {
rg := func(ev *tracev2.Event) tracev2.GoID {
return ev.Goroutine()
}
gg.stackSampleGenerator.getResource = rg
gg.logEventGenerator.getResource = rg
gg.gStates = make(map[tracev2.GoID]*gState[tracev2.GoID])
gg.focus = focus
gg.filter = filter

View File

@@ -167,8 +167,8 @@ func checkNetworkUnblock(t *testing.T, data format.Data) {
if netBlockEv == nil {
t.Error("failed to find a network unblock")
}
if count == 0 || count > 2 {
t.Errorf("found too many network block events: want 1 or 2, found %d", count)
if count == 0 {
t.Errorf("found zero network block events, want at least one")
}
// TODO(mknyszek): Check for the flow of this event to some slice event of a goroutine running.
}

View File

@@ -18,6 +18,7 @@ type procGenerator struct {
globalMetricGenerator
procRangeGenerator
stackSampleGenerator[tracev2.ProcID]
logEventGenerator[tracev2.ProcID]
gStates map[tracev2.GoID]*gState[tracev2.ProcID]
inSyscall map[tracev2.ProcID]*gState[tracev2.ProcID]
@@ -26,9 +27,11 @@ type procGenerator struct {
func newProcGenerator() *procGenerator {
pg := new(procGenerator)
pg.stackSampleGenerator.getResource = func(ev *tracev2.Event) tracev2.ProcID {
rg := func(ev *tracev2.Event) tracev2.ProcID {
return ev.Proc()
}
pg.stackSampleGenerator.getResource = rg
pg.logEventGenerator.getResource = rg
pg.gStates = make(map[tracev2.GoID]*gState[tracev2.ProcID])
pg.inSyscall = make(map[tracev2.ProcID]*gState[tracev2.ProcID])
return pg

File diff suppressed because it is too large Load Diff

View File

@@ -17,6 +17,7 @@ type threadGenerator struct {
globalRangeGenerator
globalMetricGenerator
stackSampleGenerator[tracev2.ThreadID]
logEventGenerator[tracev2.ThreadID]
gStates map[tracev2.GoID]*gState[tracev2.ThreadID]
threads map[tracev2.ThreadID]struct{}
@@ -24,9 +25,11 @@ type threadGenerator struct {
func newThreadGenerator() *threadGenerator {
tg := new(threadGenerator)
tg.stackSampleGenerator.getResource = func(ev *tracev2.Event) tracev2.ThreadID {
rg := func(ev *tracev2.Event) tracev2.ThreadID {
return ev.Thread()
}
tg.stackSampleGenerator.getResource = rg
tg.logEventGenerator.getResource = rg
tg.gStates = make(map[tracev2.GoID]*gState[tracev2.ThreadID])
tg.threads = make(map[tracev2.ThreadID]struct{})
return tg

View File

@@ -13,21 +13,15 @@ WORKDIR /boring
ENV LANG=C
ENV LANGUAGE=
# Following NIST submission draft for In Progress module validation.
# This corresponds to boringssl.googlesource.com/boringssl tag fips-20220613.
# Following NIST submission draft dated July 3, 2021.
# This corresponds to boringssl.googlesource.com/boringssl tag fips-20210429.
ENV ClangV=12
RUN apt-get update && \
apt-get install --no-install-recommends -y cmake xz-utils wget unzip ca-certificates python lsb-release software-properties-common gnupg
# Install Clang.
ENV ClangV=14
RUN \
wget https://apt.llvm.org/llvm.sh && \
chmod +x llvm.sh && \
./llvm.sh $ClangV
apt-get install --no-install-recommends -y cmake xz-utils wget unzip ca-certificates clang-$ClangV python
# Download, validate, unpack, build, and install Ninja.
ENV NinjaV=1.10.1
ENV NinjaH=a6b6f7ac360d4aabd54e299cc1d8fa7b234cd81b9401693da21221c62569a23e
ENV NinjaV=1.10.2
ENV NinjaH=ce35865411f0490368a8fc383f29071de6690cbadc27704734978221f25e2bed
RUN \
wget https://github.com/ninja-build/ninja/archive/refs/tags/v$NinjaV.tar.gz && \
echo "$NinjaH v$NinjaV.tar.gz" >sha && sha256sum -c sha && \
@@ -39,9 +33,9 @@ RUN \
# Download, validate, unpack, and install Go.
ARG GOARCH
ENV GoV=1.18.1
ENV GoHamd64=b3b815f47ababac13810fc6021eb73d65478e0b2db4b09d348eefad9581a2334
ENV GoHarm64=56a91851c97fb4697077abbca38860f735c32b38993ff79b088dac46e4735633
ENV GoV=1.16.5
ENV GoHamd64=b12c23023b68de22f74c0524f10b753e7b08b1504cb7e417eccebdd3fae49061
ENV GoHarm64=d5446b46ef6f36fdffa852f73dfbbe78c1ddf010b99fa4964944b9ae8b4d6799
RUN \
eval GoH=\${GoH$GOARCH} && \
wget https://golang.org/dl/go$GoV.linux-$GOARCH.tar.gz && \
@@ -51,8 +45,8 @@ RUN \
ln -s /usr/local/go/bin/go /usr/local/bin/
# Download, validate, and unpack BoringCrypto.
ENV BoringV=0c6f40132b828e92ba365c6b7680e32820c63fa7
ENV BoringH=62f733289f2d677c2723f556aa58034c438f3a7bbca6c12b156538a88e38da8a
ENV BoringV=853ca1ea1168dff08011e5d42d94609cc0ca2e27
ENV BoringH=a4d069ccef6f3c7bc0c68de82b91414f05cb817494cd1ab483dcf3368883c7c2
RUN \
wget https://commondatastorage.googleapis.com/chromium-boringssl-fips/boringssl-$BoringV.tar.xz && \
echo "$BoringH boringssl-$BoringV.tar.xz" >sha && sha256sum -c sha && \

View File

@@ -6,7 +6,7 @@ When building with GOEXPERIMENT=boringcrypto, the following applies.
The goboringcrypto_linux_amd64.syso object file is built
from BoringSSL source code by build/build.sh and is covered
by the BoringSSL license reproduced below and also at
https://boringssl.googlesource.com/boringssl/+/fips-20220613/LICENSE.
https://boringssl.googlesource.com/boringssl/+/fips-20190808/LICENSE.
BoringSSL is a fork of OpenSSL. As such, large parts of it fall under OpenSSL
licensing. Files that are completely new have a Google copyright and an ISC

View File

@@ -27,14 +27,13 @@ syso/goboringcrypto_linux_arm64.syso is built with:
GOARCH=arm64 ./build.sh
Both run using Docker.
Both run on an x86 Debian Linux system using Docker.
For the arm64 build to run on an x86 system, you need
apt-get install qemu-user-static qemu-binfmt-support
to allow the x86 kernel to run arm64 binaries via QEMU.
For the amd64 build to run on an Apple Silicon macOS, you need Rosetta 2.
See build.sh for more details about the build.

View File

@@ -228,41 +228,26 @@ func (c *aesCipher) NewGCM(nonceSize, tagSize int) (cipher.AEAD, error) {
if tagSize != gcmTagSize {
return cipher.NewGCMWithTagSize(&noGCM{c}, tagSize)
}
return c.newGCM(0)
return c.newGCM(false)
}
const (
VersionTLS12 = 0x0303
VersionTLS13 = 0x0304
)
func NewGCMTLS(c cipher.Block) (cipher.AEAD, error) {
return c.(*aesCipher).newGCM(VersionTLS12)
return c.(*aesCipher).newGCM(true)
}
func NewGCMTLS13(c cipher.Block) (cipher.AEAD, error) {
return c.(*aesCipher).newGCM(VersionTLS13)
}
func (c *aesCipher) newGCM(tlsVersion uint16) (cipher.AEAD, error) {
func (c *aesCipher) newGCM(tls bool) (cipher.AEAD, error) {
var aead *C.GO_EVP_AEAD
switch len(c.key) * 8 {
case 128:
switch tlsVersion {
case VersionTLS12:
if tls {
aead = C._goboringcrypto_EVP_aead_aes_128_gcm_tls12()
case VersionTLS13:
aead = C._goboringcrypto_EVP_aead_aes_128_gcm_tls13()
default:
} else {
aead = C._goboringcrypto_EVP_aead_aes_128_gcm()
}
case 256:
switch tlsVersion {
case VersionTLS12:
if tls {
aead = C._goboringcrypto_EVP_aead_aes_256_gcm_tls12()
case VersionTLS13:
aead = C._goboringcrypto_EVP_aead_aes_256_gcm_tls13()
default:
} else {
aead = C._goboringcrypto_EVP_aead_aes_256_gcm()
}
default:

View File

@@ -122,7 +122,7 @@ awk -f boringx.awk goboringcrypto.h # writes goboringcrypto.x
awk -f boringh.awk goboringcrypto.h # writes goboringcrypto[01].h
ls -l ../boringssl/include
clang++ -fPIC -I../boringssl/include -O2 -o a.out goboringcrypto.cc
clang++ -std=c++11 -fPIC -I../boringssl/include -O2 -o a.out goboringcrypto.cc
./a.out || exit 2
# clang implements u128 % u128 -> u128 by calling __umodti3,

View File

@@ -22,12 +22,6 @@ platform=""
buildargs=""
case "$GOARCH" in
amd64)
if ! docker run --rm -t amd64/ubuntu:focal uname -m >/dev/null 2>&1; then
echo "# Docker cannot run amd64 binaries."
exit 1
fi
platform="--platform linux/amd64"
buildargs="--build-arg ubuntu=amd64/ubuntu"
;;
arm64)
if ! docker run --rm -t arm64v8/ubuntu:focal uname -m >/dev/null 2>&1; then

View File

@@ -125,9 +125,7 @@ void _goboringcrypto_EVP_AEAD_CTX_cleanup(GO_EVP_AEAD_CTX*);
int _goboringcrypto_EVP_AEAD_CTX_seal(const GO_EVP_AEAD_CTX*, uint8_t*, size_t*, size_t, const uint8_t*, size_t, const uint8_t*, size_t, const uint8_t*, size_t);
int _goboringcrypto_EVP_AEAD_CTX_open(const GO_EVP_AEAD_CTX*, uint8_t*, size_t*, size_t, const uint8_t*, size_t, const uint8_t*, size_t, const uint8_t*, size_t);
const GO_EVP_AEAD* _goboringcrypto_EVP_aead_aes_128_gcm_tls12(void);
const GO_EVP_AEAD* _goboringcrypto_EVP_aead_aes_128_gcm_tls13(void);
const GO_EVP_AEAD* _goboringcrypto_EVP_aead_aes_256_gcm_tls12(void);
const GO_EVP_AEAD* _goboringcrypto_EVP_aead_aes_256_gcm_tls13(void);
enum go_evp_aead_direction_t {
go_evp_aead_open = 0,
go_evp_aead_seal = 1

View File

@@ -50,7 +50,6 @@ func NewHMAC(h func() hash.Hash, key []byte) hash.Hash { panic("boringcrypto: no
func NewAESCipher(key []byte) (cipher.Block, error) { panic("boringcrypto: not available") }
func NewGCMTLS(cipher.Block) (cipher.AEAD, error) { panic("boringcrypto: not available") }
func NewGCMTLS13(cipher.Block) (cipher.AEAD, error) { panic("boringcrypto: not available") }
type PublicKeyECDSA struct{ _ int }
type PrivateKeyECDSA struct{ _ int }

View File

@@ -6,10 +6,9 @@
package tls
import "crypto/internal/boring/fipstls"
// The FIPS-only policies enforced here currently match BoringSSL's
// ssl_policy_fips_202205.
import (
"crypto/internal/boring/fipstls"
)
// needFIPS returns fipstls.Required(); it avoids a new import in common.go.
func needFIPS() bool {
@@ -18,19 +17,19 @@ func needFIPS() bool {
// fipsMinVersion replaces c.minVersion in FIPS-only mode.
func fipsMinVersion(c *Config) uint16 {
// FIPS requires TLS 1.2 or TLS 1.3.
// FIPS requires TLS 1.2.
return VersionTLS12
}
// fipsMaxVersion replaces c.maxVersion in FIPS-only mode.
func fipsMaxVersion(c *Config) uint16 {
// FIPS requires TLS 1.2 or TLS 1.3.
return VersionTLS13
// FIPS requires TLS 1.2.
return VersionTLS12
}
// default defaultFIPSCurvePreferences is the FIPS-allowed curves,
// in preference order (most preferable first).
var defaultFIPSCurvePreferences = []CurveID{CurveP256, CurveP384}
var defaultFIPSCurvePreferences = []CurveID{CurveP256, CurveP384, CurveP521}
// fipsCurvePreferences replaces c.curvePreferences in FIPS-only mode.
func fipsCurvePreferences(c *Config) []CurveID {
@@ -55,6 +54,8 @@ var defaultCipherSuitesFIPS = []uint16{
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
TLS_RSA_WITH_AES_128_GCM_SHA256,
TLS_RSA_WITH_AES_256_GCM_SHA384,
}
// fipsCipherSuites replaces c.cipherSuites in FIPS-only mode.
@@ -74,14 +75,8 @@ func fipsCipherSuites(c *Config) []uint16 {
return list
}
// defaultCipherSuitesTLS13FIPS are the FIPS-allowed cipher suites for TLS 1.3.
var defaultCipherSuitesTLS13FIPS = []uint16{
TLS_AES_128_GCM_SHA256,
TLS_AES_256_GCM_SHA384,
}
// fipsSupportedSignatureAlgorithms currently are a subset of
// defaultSupportedSignatureAlgorithms without Ed25519, SHA-1, and P-521.
// defaultSupportedSignatureAlgorithms without Ed25519 and SHA-1.
var fipsSupportedSignatureAlgorithms = []SignatureScheme{
PSSWithSHA256,
PSSWithSHA384,
@@ -91,6 +86,7 @@ var fipsSupportedSignatureAlgorithms = []SignatureScheme{
PKCS1WithSHA384,
ECDSAWithP384AndSHA384,
PKCS1WithSHA512,
ECDSAWithP521AndSHA512,
}
// supportedSignatureAlgorithms returns the supported signature algorithms.

View File

@@ -25,31 +25,6 @@ import (
"time"
)
func allCipherSuitesIncludingTLS13() []uint16 {
s := allCipherSuites()
for _, suite := range cipherSuitesTLS13 {
s = append(s, suite.id)
}
return s
}
func isTLS13CipherSuite(id uint16) bool {
for _, suite := range cipherSuitesTLS13 {
if id == suite.id {
return true
}
}
return false
}
func generateKeyShare(group CurveID) keyShare {
key, err := generateECDHEKey(rand.Reader, group)
if err != nil {
panic(err)
}
return keyShare{group: group, data: key.PublicKey().Bytes()}
}
func TestBoringServerProtocolVersion(t *testing.T) {
test := func(name string, v uint16, msg string) {
t.Run(name, func(t *testing.T) {
@@ -58,11 +33,8 @@ func TestBoringServerProtocolVersion(t *testing.T) {
clientHello := &clientHelloMsg{
vers: v,
random: make([]byte, 32),
cipherSuites: allCipherSuitesIncludingTLS13(),
cipherSuites: allCipherSuites(),
compressionMethods: []uint8{compressionNone},
supportedCurves: defaultCurvePreferences,
keyShares: []keyShare{generateKeyShare(CurveP256)},
supportedPoints: []uint8{pointFormatUncompressed},
supportedVersions: []uint16{v},
}
testClientHelloFailure(t, serverConfig, clientHello, msg)
@@ -76,25 +48,25 @@ func TestBoringServerProtocolVersion(t *testing.T) {
fipstls.Force()
defer fipstls.Abandon()
test("VersionSSL30/fipstls", VersionSSL30, "client offered only unsupported versions")
test("VersionTLS10/fipstls", VersionTLS10, "client offered only unsupported versions")
test("VersionTLS11/fipstls", VersionTLS11, "client offered only unsupported versions")
test("VersionTLS12/fipstls", VersionTLS12, "")
test("VersionTLS13/fipstls", VersionTLS13, "")
test("VersionSSL30", VersionSSL30, "client offered only unsupported versions")
test("VersionTLS10", VersionTLS10, "client offered only unsupported versions")
test("VersionTLS11", VersionTLS11, "client offered only unsupported versions")
test("VersionTLS12", VersionTLS12, "")
test("VersionTLS13", VersionTLS13, "client offered only unsupported versions")
}
func isBoringVersion(v uint16) bool {
return v == VersionTLS12 || v == VersionTLS13
return v == VersionTLS12
}
func isBoringCipherSuite(id uint16) bool {
switch id {
case TLS_AES_128_GCM_SHA256,
TLS_AES_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
case TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384:
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
TLS_RSA_WITH_AES_128_GCM_SHA256,
TLS_RSA_WITH_AES_256_GCM_SHA384:
return true
}
return false
@@ -102,7 +74,7 @@ func isBoringCipherSuite(id uint16) bool {
func isBoringCurve(id CurveID) bool {
switch id {
case CurveP256, CurveP384:
case CurveP256, CurveP384, CurveP521:
return true
}
return false
@@ -114,7 +86,7 @@ func isECDSA(id uint16) bool {
return suite.flags&suiteECSign == suiteECSign
}
}
return false // TLS 1.3 cipher suites are not tied to the signature algorithm.
panic(fmt.Sprintf("unknown cipher suite %#x", id))
}
func isBoringSignatureScheme(alg SignatureScheme) bool {
@@ -126,6 +98,7 @@ func isBoringSignatureScheme(alg SignatureScheme) bool {
PKCS1WithSHA384,
ECDSAWithP384AndSHA384,
PKCS1WithSHA512,
ECDSAWithP521AndSHA512,
PSSWithSHA256,
PSSWithSHA384,
PSSWithSHA512:
@@ -136,9 +109,10 @@ func isBoringSignatureScheme(alg SignatureScheme) bool {
func TestBoringServerCipherSuites(t *testing.T) {
serverConfig := testConfig.Clone()
serverConfig.CipherSuites = allCipherSuites()
serverConfig.Certificates = make([]Certificate, 1)
for _, id := range allCipherSuitesIncludingTLS13() {
for _, id := range allCipherSuites() {
if isECDSA(id) {
serverConfig.Certificates[0].Certificate = [][]byte{testECDSACertificate}
serverConfig.Certificates[0].PrivateKey = testECDSAPrivateKey
@@ -147,19 +121,14 @@ func TestBoringServerCipherSuites(t *testing.T) {
serverConfig.Certificates[0].PrivateKey = testRSAPrivateKey
}
serverConfig.BuildNameToCertificate()
t.Run(fmt.Sprintf("suite=%s", CipherSuiteName(id)), func(t *testing.T) {
t.Run(fmt.Sprintf("suite=%#x", id), func(t *testing.T) {
clientHello := &clientHelloMsg{
vers: VersionTLS12,
random: make([]byte, 32),
cipherSuites: []uint16{id},
compressionMethods: []uint8{compressionNone},
supportedCurves: defaultCurvePreferences,
keyShares: []keyShare{generateKeyShare(CurveP256)},
supportedPoints: []uint8{pointFormatUncompressed},
supportedVersions: []uint16{VersionTLS12},
}
if isTLS13CipherSuite(id) {
clientHello.supportedVersions = []uint16{VersionTLS13}
}
testClientHello(t, serverConfig, clientHello)
@@ -191,9 +160,7 @@ func TestBoringServerCurves(t *testing.T) {
cipherSuites: []uint16{TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256},
compressionMethods: []uint8{compressionNone},
supportedCurves: []CurveID{curveid},
keyShares: []keyShare{generateKeyShare(curveid)},
supportedPoints: []uint8{pointFormatUncompressed},
supportedVersions: []uint16{VersionTLS12},
}
testClientHello(t, serverConfig, clientHello)
@@ -312,7 +279,7 @@ func TestBoringClientHello(t *testing.T) {
}
if !isBoringVersion(hello.vers) {
t.Errorf("client vers=%#x", hello.vers)
t.Errorf("client vers=%#x, want %#x (TLS 1.2)", hello.vers, VersionTLS12)
}
for _, v := range hello.supportedVersions {
if !isBoringVersion(v) {

View File

@@ -556,13 +556,7 @@ func aeadAESGCMTLS13(key, nonceMask []byte) aead {
if err != nil {
panic(err)
}
var aead cipher.AEAD
if boring.Enabled {
aead, err = boring.NewGCMTLS13(aes)
} else {
boring.Unreachable()
aead, err = cipher.NewGCM(aes)
}
aead, err := cipher.NewGCM(aes)
if err != nil {
panic(err)
}

View File

@@ -139,9 +139,7 @@ func (c *Conn) makeClientHello() (*clientHelloMsg, *ecdh.PrivateKey, error) {
if len(hello.supportedVersions) == 1 {
hello.cipherSuites = nil
}
if needFIPS() {
hello.cipherSuites = append(hello.cipherSuites, defaultCipherSuitesTLS13FIPS...)
} else if hasAESGCMHardwareSupport {
if hasAESGCMHardwareSupport {
hello.cipherSuites = append(hello.cipherSuites, defaultCipherSuitesTLS13...)
} else {
hello.cipherSuites = append(hello.cipherSuites, defaultCipherSuitesTLS13NoAES...)

View File

@@ -41,6 +41,10 @@ type clientHandshakeStateTLS13 struct {
func (hs *clientHandshakeStateTLS13) handshake() error {
c := hs.c
if needFIPS() {
return errors.New("tls: internal error: TLS 1.3 reached in FIPS mode")
}
// The server must not select TLS 1.3 in a renegotiation. See RFC 8446,
// sections 4.1.2 and 4.1.3.
if c.handshakes > 0 {

View File

@@ -27,7 +27,6 @@ import (
)
func testClientHello(t *testing.T, serverConfig *Config, m handshakeMessage) {
t.Helper()
testClientHelloFailure(t, serverConfig, m, "")
}
@@ -53,32 +52,23 @@ func testClientHelloFailure(t *testing.T, serverConfig *Config, m handshakeMessa
ctx := context.Background()
conn := Server(s, serverConfig)
ch, err := conn.readClientHello(ctx)
if err == nil && conn.vers == VersionTLS13 {
hs := serverHandshakeStateTLS13{
c: conn,
ctx: ctx,
clientHello: ch,
}
hs := serverHandshakeState{
c: conn,
ctx: ctx,
clientHello: ch,
}
if err == nil {
err = hs.processClientHello()
} else if err == nil {
hs := serverHandshakeState{
c: conn,
ctx: ctx,
clientHello: ch,
}
err = hs.processClientHello()
if err == nil {
err = hs.pickCipherSuite()
}
}
if err == nil {
err = hs.pickCipherSuite()
}
s.Close()
if len(expectedSubStr) == 0 {
if err != nil && err != io.EOF {
t.Helper()
t.Errorf("Got error: %s; expected to succeed", err)
}
} else if err == nil || !strings.Contains(err.Error(), expectedSubStr) {
t.Helper()
t.Errorf("Got error: %v; expected to match substring '%s'", err, expectedSubStr)
}
}

View File

@@ -45,6 +45,10 @@ type serverHandshakeStateTLS13 struct {
func (hs *serverHandshakeStateTLS13) handshake() error {
c := hs.c
if needFIPS() {
return errors.New("tls: internal error: TLS 1.3 reached in FIPS mode")
}
// For an overview of the TLS 1.3 handshake, see RFC 8446, Section 2.
if err := hs.processClientHello(); err != nil {
return err
@@ -159,9 +163,6 @@ func (hs *serverHandshakeStateTLS13) processClientHello() error {
if !hasAESGCMHardwareSupport || !aesgcmPreferred(hs.clientHello.cipherSuites) {
preferenceList = defaultCipherSuitesTLS13NoAES
}
if needFIPS() {
preferenceList = defaultCipherSuitesTLS13FIPS
}
for _, suiteID := range preferenceList {
hs.suite = mutualCipherSuiteTLS13(hs.clientHello.cipherSuites, suiteID)
if hs.suite != nil {

View File

@@ -18,5 +18,3 @@ func fipsCurvePreferences(c *Config) []CurveID { panic("fipsCurvePreferences") }
func fipsCipherSuites(c *Config) []uint16 { panic("fipsCipherSuites") }
var fipsSupportedSignatureAlgorithms []SignatureScheme
var defaultCipherSuitesTLS13FIPS []uint16

View File

@@ -22,7 +22,7 @@ func boringAllowCert(c *Certificate) bool {
}
// The key must be RSA 2048, RSA 3072, RSA 4096,
// or ECDSA P-256 or P-384.
// or ECDSA P-256, P-384, P-521.
switch k := c.PublicKey.(type) {
default:
return false
@@ -31,7 +31,7 @@ func boringAllowCert(c *Certificate) bool {
return false
}
case *ecdsa.PublicKey:
if k.Curve != elliptic.P256() && k.Curve != elliptic.P384() {
if k.Curve != elliptic.P256() && k.Curve != elliptic.P384() && k.Curve != elliptic.P521() {
return false
}
}

View File

@@ -2,9 +2,12 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package version provides operations on [Go versions].
// Package version provides operations on [Go versions]
// in [Go toolchain name syntax]: strings like
// "go1.20", "go1.21.0", "go1.22rc2", and "go1.23.4-bigcorp".
//
// [Go versions]: https://go.dev/doc/toolchain#version
// [Go toolchain name syntax]: https://go.dev/doc/toolchain#name
package version // import "go/version"
import (
@@ -12,9 +15,10 @@ import (
"strings"
)
// stripGo converts from a "go1.21" version to a "1.21" version.
// stripGo converts from a "go1.21-bigcorp" version to a "1.21" version.
// If v does not start with "go", stripGo returns the empty string (a known invalid version).
func stripGo(v string) string {
v, _, _ = strings.Cut(v, "-") // strip -bigcorp suffix.
if len(v) < 2 || v[:2] != "go" {
return ""
}
@@ -50,8 +54,6 @@ func Lang(x string) string {
// valid versions and equal to each other.
// The language version "go1.21" compares less than the
// release candidate and eventual releases "go1.21rc1" and "go1.21.0".
// Custom toolchain suffixes are ignored during comparison:
// "go1.21.0" and "go1.21.0-bigcorp" are equal.
func Compare(x, y string) int {
return gover.Compare(stripGo(x), stripGo(y))
}

View File

@@ -23,13 +23,16 @@ var compareTests = []testCase2[string, string, int]{
{"go1.19", "go1.19.0", 0},
{"go1.19rc1", "go1.19", -1},
{"go1.20", "go1.20.0", 0},
{"go1.20", "go1.20.0-bigcorp", 0},
{"go1.20rc1", "go1.20", -1},
{"go1.21", "go1.21.0", -1},
{"go1.21", "go1.21.0-bigcorp", -1},
{"go1.21", "go1.21rc1", -1},
{"go1.21rc1", "go1.21.0", -1},
{"go1.6", "go1.19", -1},
{"go1.19", "go1.19.1", -1},
{"go1.19rc1", "go1.19", -1},
{"go1.19rc1", "go1.19", -1},
{"go1.19rc1", "go1.19.1", -1},
{"go1.19rc1", "go1.19rc2", -1},
{"go1.19.0", "go1.19.1", -1},

View File

@@ -302,6 +302,13 @@ func (o *ordering) advance(ev *baseEvent, evt *evTable, m ThreadID, gen uint64)
// Otherwise, we're talking about a G sitting in a syscall on an M.
// Validate the named M.
if mid == curCtx.M {
if gen != o.initialGen && curCtx.G != gid {
// If this isn't the first generation, we *must* have seen this
// binding occur already. Even if the G was blocked in a syscall
// for multiple generations since trace start, we would have seen
// a previous GoStatus event that bound the goroutine to an M.
return curCtx, false, fmt.Errorf("inconsistent thread for syscalling goroutine %d: thread has goroutine %d", gid, curCtx.G)
}
newCtx.G = gid
break
}

View File

@@ -0,0 +1,66 @@
// Copyright 2023 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Tests a goroutine sitting blocked in a syscall for
// an entire generation. This is a regression test for
// #65196.
//go:build ignore
package main
import (
"log"
"os"
"runtime/trace"
"syscall"
"time"
)
func main() {
// Create a pipe to block on.
var p [2]int
if err := syscall.Pipe(p[:]); err != nil {
log.Fatalf("failed to create pipe: %v", err)
}
rfd, wfd := p[0], p[1]
// Create a goroutine that blocks on the pipe.
done := make(chan struct{})
go func() {
var data [1]byte
_, err := syscall.Read(rfd, data[:])
if err != nil {
log.Fatalf("failed to read from pipe: %v", err)
}
done <- struct{}{}
}()
// Give the goroutine ample chance to block on the pipe.
time.Sleep(10 * time.Millisecond)
// Start tracing.
if err := trace.Start(os.Stdout); err != nil {
log.Fatalf("failed to start tracing: %v", err)
}
// This isn't enough to have a full generation pass by default,
// but it is generally enough in stress mode.
time.Sleep(100 * time.Millisecond)
// Write to the pipe to unblock it.
if _, err := syscall.Write(wfd, []byte{10}); err != nil {
log.Fatalf("failed to write to pipe: %v", err)
}
// Wait for the goroutine to unblock and start running.
// This is helpful to catch incorrect information written
// down for the syscall-blocked goroutine, since it'll start
// executing, and that execution information will be
// inconsistent.
<-done
// Stop tracing.
trace.Stop()
}

View File

@@ -896,7 +896,7 @@ String id=18
String id=19
data="sleep"
String id=20
data="runtime.GoSched"
data="runtime.Gosched"
String id=21
data="start trace"
String id=22

View File

@@ -220,7 +220,7 @@ String id=18
String id=19
data="sleep"
String id=20
data="runtime.GoSched"
data="runtime.Gosched"
String id=21
data="start trace"
String id=22

View File

@@ -4086,7 +4086,7 @@ String id=18
String id=19
data="sleep"
String id=20
data="runtime.GoSched"
data="runtime.Gosched"
String id=21
data="GC mark termination"
String id=22

View File

@@ -213,7 +213,7 @@ func TestTraceFutileWakeup(t *testing.T) {
// Check to make sure that no goroutine in the "special" trace region
// ends up blocking, unblocking, then immediately blocking again.
//
// The goroutines are careful to call runtime.GoSched in between blocking,
// The goroutines are careful to call runtime.Gosched in between blocking,
// so there should never be a clean block/unblock on the goroutine unless
// the runtime was generating extraneous events.
const (
@@ -521,6 +521,15 @@ func TestTraceManyStartStop(t *testing.T) {
testTraceProg(t, "many-start-stop.go", nil)
}
func TestTraceWaitOnPipe(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "freebsd", "linux", "netbsd", "openbsd", "solaris":
testTraceProg(t, "wait-on-pipe.go", nil)
return
}
t.Skip("no applicable syscall.Pipe on " + runtime.GOOS)
}
func testTraceProg(t *testing.T, progName string, extra func(t *testing.T, trace, stderr []byte, stress bool)) {
testenv.MustHaveGoRun(t)

View File

@@ -4404,8 +4404,8 @@ func entersyscall_gcwait() {
pp := gp.m.oldp.ptr()
lock(&sched.lock)
trace := traceAcquire()
if sched.stopwait > 0 && atomic.Cas(&pp.status, _Psyscall, _Pgcstop) {
trace := traceAcquire()
if trace.ok() {
if goexperiment.ExecTracer2 {
// This is a steal in the new tracer. While it's very likely
@@ -4428,6 +4428,8 @@ func entersyscall_gcwait() {
if sched.stopwait--; sched.stopwait == 0 {
notewakeup(&sched.stopnote)
}
} else if trace.ok() {
traceRelease(trace)
}
unlock(&sched.lock)
}
@@ -4605,12 +4607,19 @@ func exitsyscallfast(oldp *p) bool {
}
// Try to re-acquire the last P.
trace := traceAcquire()
if oldp != nil && oldp.status == _Psyscall && atomic.Cas(&oldp.status, _Psyscall, _Pidle) {
// There's a cpu for us, so we can run.
wirep(oldp)
exitsyscallfast_reacquired()
exitsyscallfast_reacquired(trace)
if trace.ok() {
traceRelease(trace)
}
return true
}
if trace.ok() {
traceRelease(trace)
}
// Try to get any other idle P.
if sched.pidle != 0 {
@@ -4646,10 +4655,9 @@ func exitsyscallfast(oldp *p) bool {
// syscall.
//
//go:nosplit
func exitsyscallfast_reacquired() {
func exitsyscallfast_reacquired(trace traceLocker) {
gp := getg()
if gp.m.syscalltick != gp.m.p.ptr().syscalltick {
trace := traceAcquire()
if trace.ok() {
// The p was retaken and then enter into syscall again (since gp.m.syscalltick has changed).
// traceGoSysBlock for this syscall was already emitted,
@@ -4666,7 +4674,6 @@ func exitsyscallfast_reacquired() {
// Denote completion of the current syscall.
trace.GoSysExit(true)
}
traceRelease(trace)
})
}
gp.m.p.ptr().syscalltick++
@@ -6146,8 +6153,8 @@ func retake(now int64) uint32 {
// Otherwise the M from which we retake can exit the syscall,
// increment nmidle and report deadlock.
incidlelocked(-1)
trace := traceAcquire()
if atomic.Cas(&pp.status, s, _Pidle) {
trace := traceAcquire()
if trace.ok() {
trace.GoSysBlock(pp)
trace.ProcSteal(pp, false)
@@ -6156,6 +6163,8 @@ func retake(now int64) uint32 {
n++
pp.syscalltick++
handoffp(pp)
} else if trace.ok() {
traceRelease(trace)
}
incidlelocked(1)
lock(&allpLock)

View File

@@ -71,7 +71,8 @@ var trace struct {
stringTab [2]traceStringTable // maps strings to unique ids
// cpuLogRead accepts CPU profile samples from the signal handler where
// they're generated. It uses a three-word header to hold the IDs of the P, G,
// they're generated. There are two profBufs here: one for gen%2, one for
// 1-gen%2. These profBufs use a three-word header to hold the IDs of the P, G,
// and M (respectively) that were active at the time of the sample. Because
// profBuf uses a record with all zeros in its header to indicate overflow,
// we make sure to make the P field always non-zero: The ID of a real P will
@@ -82,9 +83,9 @@ var trace struct {
// when sampling g0.
//
// Initialization and teardown of these fields is protected by traceAdvanceSema.
cpuLogRead *profBuf
signalLock atomic.Uint32 // protects use of the following member, only usable in signal handlers
cpuLogWrite atomic.Pointer[profBuf] // copy of cpuLogRead for use in signal handlers, set without signalLock
cpuLogRead [2]*profBuf
signalLock atomic.Uint32 // protects use of the following member, only usable in signal handlers
cpuLogWrite [2]atomic.Pointer[profBuf] // copy of cpuLogRead for use in signal handlers, set without signalLock
cpuSleep *wakeableSleep
cpuLogDone <-chan struct{}
cpuBuf [2]*traceBuf
@@ -334,7 +335,7 @@ func traceAdvance(stopTrace bool) {
if !s.dead {
ug.goid = s.g.goid
if s.g.m != nil {
ug.mid = s.g.m.id
ug.mid = int64(s.g.m.procid)
}
ug.status = readgstatus(s.g) &^ _Gscan
ug.waitreason = s.g.waitreason
@@ -931,7 +932,13 @@ func newWakeableSleep() *wakeableSleep {
func (s *wakeableSleep) sleep(ns int64) {
resetTimer(s.timer, nanotime()+ns)
lock(&s.lock)
if raceenabled {
raceacquire(unsafe.Pointer(&s.lock))
}
wakeup := s.wakeup
if raceenabled {
racerelease(unsafe.Pointer(&s.lock))
}
unlock(&s.lock)
<-wakeup
stopTimer(s.timer)
@@ -944,6 +951,9 @@ func (s *wakeableSleep) wake() {
// Grab the wakeup channel, which may be nil if we're
// racing with close.
lock(&s.lock)
if raceenabled {
raceacquire(unsafe.Pointer(&s.lock))
}
if s.wakeup != nil {
// Non-blocking send.
//
@@ -955,6 +965,9 @@ func (s *wakeableSleep) wake() {
default:
}
}
if raceenabled {
racerelease(unsafe.Pointer(&s.lock))
}
unlock(&s.lock)
}
@@ -968,11 +981,18 @@ func (s *wakeableSleep) wake() {
func (s *wakeableSleep) close() {
// Set wakeup to nil so that a late timer ends up being a no-op.
lock(&s.lock)
if raceenabled {
raceacquire(unsafe.Pointer(&s.lock))
}
wakeup := s.wakeup
s.wakeup = nil
// Close the channel.
close(wakeup)
if raceenabled {
racerelease(unsafe.Pointer(&s.lock))
}
unlock(&s.lock)
return
}

View File

@@ -16,8 +16,9 @@ func traceInitReadCPU() {
throw("traceInitReadCPU called with trace enabled")
}
// Create new profBuf for CPU samples that will be emitted as events.
profBuf := newProfBuf(3, profBufWordCount, profBufTagCount) // after the timestamp, header is [pp.id, gp.goid, mp.procid]
trace.cpuLogRead = profBuf
// Format: after the timestamp, header is [pp.id, gp.goid, mp.procid].
trace.cpuLogRead[0] = newProfBuf(3, profBufWordCount, profBufTagCount)
trace.cpuLogRead[1] = newProfBuf(3, profBufWordCount, profBufTagCount)
// We must not acquire trace.signalLock outside of a signal handler: a
// profiling signal may arrive at any time and try to acquire it, leading to
// deadlock. Because we can't use that lock to protect updates to
@@ -25,7 +26,8 @@ func traceInitReadCPU() {
// writes of the pointer must be atomic. (And although this field is never
// the sole pointer to the profBuf value, it's best to allow a write barrier
// here.)
trace.cpuLogWrite.Store(profBuf)
trace.cpuLogWrite[0].Store(trace.cpuLogRead[0])
trace.cpuLogWrite[1].Store(trace.cpuLogRead[1])
}
// traceStartReadCPU creates a goroutine to start reading CPU profile
@@ -52,7 +54,15 @@ func traceStartReadCPU() {
// we would still want to do a goroutine-level sleep in between
// reads to avoid frequent wakeups.
trace.cpuSleep.sleep(100_000_000)
if !traceReadCPU(trace.cpuLogRead) {
tl := traceAcquire()
if !tl.ok() {
// Tracing disabled.
break
}
keepGoing := traceReadCPU(tl.gen)
traceRelease(tl)
if !keepGoing {
break
}
}
@@ -76,8 +86,10 @@ func traceStopReadCPU() {
//
// Wake the goroutine so it can observe that their the buffer is
// closed an exit.
trace.cpuLogWrite.Store(nil)
trace.cpuLogRead.close()
trace.cpuLogWrite[0].Store(nil)
trace.cpuLogWrite[1].Store(nil)
trace.cpuLogRead[0].close()
trace.cpuLogRead[1].close()
trace.cpuSleep.wake()
// Wait until the logger goroutine exits.
@@ -85,20 +97,25 @@ func traceStopReadCPU() {
// Clear state for the next trace.
trace.cpuLogDone = nil
trace.cpuLogRead = nil
trace.cpuLogRead[0] = nil
trace.cpuLogRead[1] = nil
trace.cpuSleep.close()
}
// traceReadCPU attempts to read from the provided profBuf and write
// traceReadCPU attempts to read from the provided profBuf[gen%2] and write
// into the trace. Returns true if there might be more to read or false
// if the profBuf is closed or the caller should otherwise stop reading.
//
// The caller is responsible for ensuring that gen does not change. Either
// the caller must be in a traceAcquire/traceRelease block, or must be calling
// with traceAdvanceSema held.
//
// No more than one goroutine may be in traceReadCPU for the same
// profBuf at a time.
func traceReadCPU(pb *profBuf) bool {
func traceReadCPU(gen uintptr) bool {
var pcBuf [traceStackSize]uintptr
data, tags, eof := pb.read(profBufNonBlocking)
data, tags, eof := trace.cpuLogRead[gen%2].read(profBufNonBlocking)
for len(data) > 0 {
if len(data) < 4 || data[0] > uint64(len(data)) {
break // truncated profile
@@ -147,12 +164,7 @@ func traceReadCPU(pb *profBuf) bool {
}
// Write out a trace event.
tl := traceAcquire()
if !tl.ok() {
// Tracing disabled, exit without continuing.
return false
}
w := unsafeTraceWriter(tl.gen, trace.cpuBuf[tl.gen%2])
w := unsafeTraceWriter(gen, trace.cpuBuf[gen%2])
// Ensure we have a place to write to.
var flushed bool
@@ -163,7 +175,7 @@ func traceReadCPU(pb *profBuf) bool {
}
// Add the stack to the table.
stackID := trace.stackTab[tl.gen%2].put(pcBuf[:nstk])
stackID := trace.stackTab[gen%2].put(pcBuf[:nstk])
// Write out the CPU sample.
w.byte(byte(traceEvCPUSample))
@@ -173,8 +185,7 @@ func traceReadCPU(pb *profBuf) bool {
w.varint(goid)
w.varint(stackID)
trace.cpuBuf[tl.gen%2] = w.traceBuf
traceRelease(tl)
trace.cpuBuf[gen%2] = w.traceBuf
}
return !eof
}
@@ -187,6 +198,10 @@ func traceReadCPU(pb *profBuf) bool {
//
//go:systemstack
func traceCPUFlush(gen uintptr) {
// Read everything out of the last gen's CPU profile buffer.
traceReadCPU(gen)
// Flush any remaining trace buffers containing CPU samples.
if buf := trace.cpuBuf[gen%2]; buf != nil {
lock(&trace.lock)
traceBufFlush(buf, gen)
@@ -197,13 +212,38 @@ func traceCPUFlush(gen uintptr) {
// traceCPUSample writes a CPU profile sample stack to the execution tracer's
// profiling buffer. It is called from a signal handler, so is limited in what
// it can do.
// it can do. mp must be the thread that is currently stopped in a signal.
func traceCPUSample(gp *g, mp *m, pp *p, stk []uintptr) {
if !traceEnabled() {
// Tracing is usually turned off; don't spend time acquiring the signal
// lock unless it's active.
return
}
if mp == nil {
// Drop samples that don't have an identifiable thread. We can't render
// this in any useful way anyway.
return
}
// We're going to conditionally write to one of two buffers based on the
// generation. To make sure we write to the correct one, we need to make
// sure this thread's trace seqlock is held. If it already is, then we're
// in the tracer and we can just take advantage of that. If it isn't, then
// we need to acquire it and read the generation.
locked := false
if mp.trace.seqlock.Load()%2 == 0 {
mp.trace.seqlock.Add(1)
locked = true
}
gen := trace.gen.Load()
if gen == 0 {
// Tracing is disabled, as it turns out. Release the seqlock if necessary
// and exit.
if locked {
mp.trace.seqlock.Add(1)
}
return
}
now := traceClockNow()
// The "header" here is the ID of the M that was running the profiled code,
@@ -231,7 +271,7 @@ func traceCPUSample(gp *g, mp *m, pp *p, stk []uintptr) {
osyield()
}
if log := trace.cpuLogWrite.Load(); log != nil {
if log := trace.cpuLogWrite[gen%2].Load(); log != nil {
// Note: we don't pass a tag pointer here (how should profiling tags
// interact with the execution tracer?), but if we did we'd need to be
// careful about write barriers. See the long comment in profBuf.write.
@@ -239,4 +279,9 @@ func traceCPUSample(gp *g, mp *m, pp *p, stk []uintptr) {
}
trace.signalLock.Store(0)
// Release the seqlock if we acquired it earlier.
if locked {
mp.trace.seqlock.Add(1)
}
}

View File

@@ -141,5 +141,11 @@ func (tab *traceMap) reset() {
assertLockHeld(&tab.lock)
tab.mem.drop()
tab.seq.Store(0)
tab.tab = [1 << 13]atomic.UnsafePointer{}
// Clear table without write barriers. The table consists entirely
// of notinheap pointers, so this is fine.
//
// Write barriers may theoretically call into the tracer and acquire
// the lock again, and this lock ordering is expressed in the static
// lock ranking checker.
memclrNoHeapPointers(unsafe.Pointer(&tab.tab), unsafe.Sizeof(tab.tab))
}

View File

@@ -133,7 +133,7 @@ const (
var traceGoStopReasonStrings = [...]string{
traceGoStopGeneric: "unspecified",
traceGoStopGoSched: "runtime.GoSched",
traceGoStopGoSched: "runtime.Gosched",
traceGoStopPreempted: "preempted",
}
@@ -192,7 +192,12 @@ func traceAcquireEnabled() traceLocker {
// Prevent preemption.
mp := acquirem()
// Acquire the trace seqlock.
// Acquire the trace seqlock. This prevents traceAdvance from moving forward
// until all Ms are observed to be outside of their seqlock critical section.
//
// Note: The seqlock is mutated here and also in traceCPUSample. If you update
// usage of the seqlock here, make sure to also look at what traceCPUSample is
// doing.
seq := mp.trace.seqlock.Add(1)
if debugTraceReentrancy && seq%2 != 1 {
throw("bad use of trace.seqlock or tracer is reentrant")