Compare commits

..

153 Commits

Author SHA1 Message Date
Gopher Robot
d8392e6997 [release-branch.go1.21] go1.21.9
Change-Id: I6c69376d434dcf310336a0344051037bf58a4cf7
Reviewed-on: https://go-review.googlesource.com/c/go/+/576117
Commit-Queue: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
2024-04-03 15:35:16 +00:00
Damien Neil
ae5913347d [release-branch.go1.21] net/http: update bundled golang.org/x/net/http2
Disable cmd/internal/moddeps test, since this update includes PRIVATE
track fixes.

Fixes CVE-2023-45288
For #65051
Fixes #65387

Change-Id: I17da6da2fe0dd70062b49f94377875acb34829a1
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2197267
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/576075
TryBot-Bypass: Dmitri Shuralyov <dmitshur@google.com>
Commit-Queue: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2024-04-03 15:10:22 +00:00
Damien Neil
30d8550669 [release-branch.go1.21] all: update golang.org/x/net
Pulls in one HTTP/2 fix:

	0b0455d2c9 http2: reject DATA frames after 1xx and before final headers

For golang/go#65927
Fixes golang/go#66254

Change-Id: I257b2634f63e8c6039c44dea24c345043c23c8d2
Reviewed-on: https://go-review.googlesource.com/c/go/+/574916
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-28 19:12:36 +00:00
Robert Griesemer
efb7cc4275 [release-branch.go1.21] go/types, types2: don't do version checks for embedded types of imported interfaces
[This is a re-apply of CL 571075]

Imported interfaces don't have position information for embedded types.
When computing the type set of such interfaces, doing a version check
may fail because it will rely on the Go version of the current package.

We must not do a version check for features of types from imported
packages - those types have already been typechecked and are "correct".
The version check code does look at packages to avoid such incorrect
version checks, but we don't have the package information available
in an interface type (divorced from its object).

Instead, for now rely on the fact that imported interfaces don't have
position information for embedded types: if the position is unknown,
don't do a version check.

We may want to assert that positions are known in all other cases,
but since this is an older release, don't add such additional changes
to avoid introducing other bugs.

Fixes #66326.
Updates #66064.

Change-Id: I158cf51aa382f85d612ab958ba4b591de1c5fdb2
Reviewed-on: https://go-review.googlesource.com/c/go/+/574736
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-27 19:24:11 +00:00
Paul E. Murphy
0bd1a2289d [release-branch.go1.21] cmd/internal/obj/ppc64: don't modify runtime.elf_* symbols
The runtime.elf_* symbols are assembly functions which are used
to support the gcc/llvm -Os option when used with cgo.

When compiling Go for shared code, we attempt to strip out the
TOC regenation code added by the go assembler for these symbols.

This causes the symbol to no longer appear as an assembly
function which causes problems later on when handling other
implicit symbols.

Avoid adding a TOC regeneration prologue to these functions
to avoid this issue.

Fixes #66411

Change-Id: Icbf8e4438d177082a57bb228e39b232e7a0d7ada
Reviewed-on: https://go-review.googlesource.com/c/go/+/571835
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/572876
2024-03-26 19:16:20 +00:00
Cherry Mui
140b37d659 [release-branch.go1.21] Revert "go/types, types2: don't do version checks for embedded types of imported interfaces"
This reverts CL 571075.

Reason for revert: We might want to do a security-only minor release. Back off the release branch to a clean state from the previous minor release. Sorry for the inconvenience.

Change-Id: Ifc8c7e00e6faea3aa547b883eed44180ddb447de
Reviewed-on: https://go-review.googlesource.com/c/go/+/571355
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Findley <rfindley@google.com>
2024-03-13 19:10:25 +00:00
Robert Griesemer
6d229889d8 [release-branch.go1.21] go/types, types2: don't do version checks for embedded types of imported interfaces
Imported interfaces don't have position information for embedded types.
When computing the type set of such interfaces, doing a version check
may fail because it will rely on the Go version of the current package.

We must not do a version check for features of types from imported
packages - those types have already been typechecked and are "correct".
The version check code does look at packages to avoid such incorrect
version checks, but we don't have the package information available
in an interface type (divorced from its object).

Instead, for now rely on the fact that imported interfaces don't have
position information for embedded types: if the position is unknown,
don't do a version check.

We may want to assert that positions are known in all other cases,
but since this is an older release, don't add such additional changes
to avoid introducing other bugs.

Fixes #66064.

Change-Id: I773d57e5410c3d4a911ab3e018b3233c2972b3c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/571075
Reviewed-by: Robert Findley <rfindley@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Griesemer <gri@google.com>
2024-03-12 20:30:58 +00:00
Gopher Robot
63992defa8 [release-branch.go1.21] go1.21.8
Change-Id: I44203158172ca3e66f8ce4ab84f54c9247dacb28
Reviewed-on: https://go-review.googlesource.com/c/go/+/569256
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-05 17:38:51 +00:00
Damien Neil
bf80213b12 [release-branch.go1.21] net/textproto, mime/multipart: avoid unbounded read in MIME header
mime/multipart.Reader.ReadForm allows specifying the maximum amount
of memory that will be consumed by the form. While this limit is
correctly applied to the parsed form data structure, it was not
being applied to individual header lines in a form.

For example, when presented with a form containing a header line
that never ends, ReadForm will continue to read the line until it
runs out of memory.

Limit the amount of data consumed when reading a header.

Fixes CVE-2023-45290
Fixes #65389
For #65383

Change-Id: I7f9264d25752009e95f6b2c80e3d76aaf321d658
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2134435
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2173776
Reviewed-by: Carlos Amedee <amedee@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/569240
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
2024-03-05 16:51:36 +00:00
Damien Neil
20586c0dbe [release-branch.go1.21] net/http, net/http/cookiejar: avoid subdomain matches on IPv6 zones
When deciding whether to forward cookies or sensitive headers
across a redirect, do not attempt to interpret an IPv6 address
as a domain name.

Avoids a case where a maliciously-crafted redirect to an
IPv6 address with a scoped addressing zone could be
misinterpreted as a within-domain redirect. For example,
we could interpret "::1%.www.example.com" as a subdomain
of "www.example.com".

Thanks to Juho Nurminen of Mattermost for reporting this issue.

Fixes CVE-2023-45289
Fixes #65385
For #65065

Change-Id: I8f463f59f0e700c8a18733d2b264a8bcb3a19599
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2131938
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2173775
Reviewed-by: Carlos Amedee <amedee@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/569239
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
2024-03-05 16:51:34 +00:00
Roland Shoemaker
be5b52bea6 [release-branch.go1.21] crypto/x509: make sure pub key is non-nil before interface conversion
alreadyInChain assumes all keys fit a interface which contains the
Equal method (which they do), but this ignores that certificates may
have a nil key when PublicKeyAlgorithm is UnknownPublicKeyAlgorithm. In
this case alreadyInChain panics.

Check that the key is non-nil as part of considerCandidate (we are never
going to build a chain containing UnknownPublicKeyAlgorithm anyway).

For #65390
Fixes #65392
Fixes CVE-2024-24783

Change-Id: Ibdccc0a487e3368b6812be35daad2512220243f3
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2137282
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2173774
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Carlos Amedee <amedee@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/569238
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
2024-03-05 16:44:27 +00:00
Roland Shoemaker
3643147a29 [release-branch.go1.21] html/template: escape additional tokens in MarshalJSON errors
Escape "</script" and "<!--" in errors returned from MarshalJSON errors
when attempting to marshal types in script blocks. This prevents any
user controlled content from prematurely terminating the script block.

Updates #65697
Fixes #65968

Change-Id: Icf0e26c54ea7d9c1deed0bff11b6506c99ddef1b
Reviewed-on: https://go-review.googlesource.com/c/go/+/564196
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
(cherry picked from commit ccbc725f2d)
Reviewed-on: https://go-review.googlesource.com/c/go/+/567515
Reviewed-by: Carlos Amedee <carlos@golang.org>
2024-02-28 19:53:41 +00:00
Bryan C. Mills
3a588774a5 [release-branch.go1.21] internal/syscall/windows: fix the signature of SetFileInformationByHandle
Also fix its call site in internal/poll to pass the length of the
actual buffer instead of an unrelated variable, and update the
definition of FILE_BASIC_INFO to match the documented field types
and add padding that is empirically needed on the 386 architecture.

Passing a pointer to a Go-allocated buffer as type uintptr violates
the unsafe.Pointer conversion rules, which allow such a conversion
only in the call expression itself for a call to syscall.Syscall or
equivalent. That can allow the buffer to be corrupted arbitrarily if
the Go runtime happens to garbage-collect it while the call to
SetFileInformationByHandle is in progress.

The Microsoft documentation for SetFileInformationByHandle specifies
its third argument type as LPVOID, which corresponds to Go's
unsafe.Pointer, not uintptr.

Fixes #65882.
Updates #58933.

Change-Id: If577b57adea9922f5fcca55e46030c703d8f035c
Cq-Include-Trybots: luci.golang.try:go1.21-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/549256
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
(cherry picked from commit a7097243e4)
Reviewed-on: https://go-review.googlesource.com/c/go/+/566155
Reviewed-by: Bryan Mills <bcmills@google.com>
2024-02-28 19:46:24 +00:00
Roland Shoemaker
263c059b09 [release-branch.go1.21] net/mail: properly handle special characters in phrase and obs-phrase
Fixes a couple of misalignments with RFC 5322 which introduce
significant diffs between (mostly) conformant parsers.

This change reverts the changes made in CL50911, which allowed certain
special RFC 5322 characters to appear unquoted in the "phrase" syntax.
It is unclear why this change was made in the first place, and created
a divergence from comformant parsers. In particular this resulted in
treating comments in display names incorrectly.

Additionally properly handle trailing malformed comments in the group
syntax.

For #65083
Fixes #65848

Change-Id: I00dddc044c6ae3381154e43236632604c390f672
Reviewed-on: https://go-review.googlesource.com/c/go/+/555596
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/566195
Reviewed-by: Carlos Amedee <carlos@golang.org>
2024-02-28 19:10:06 +00:00
Michael Matloob
99e44c71f6 [release-branch.go1.21] cmd/go/internal/modcmd: correctly filter out main modules in verify
This change fixes a bug where we incorrectly filtered out the main
modules from the beginning of the build list before verifying them. We
made the assumption that the first MainModules.Len() entries of the
build list were the main modules, but now it can contain the go and
toolchain version entries, so removing the first MainModules.Len()
entries could leave main module names in the build list if any of
their names sorted after the string 'go'.

For #62663
Fixes #65851

Change-Id: I35ab6857a556f58d306303322afe24c48fc8b38f
Reviewed-on: https://go-review.googlesource.com/c/go/+/565378
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 3f60da7944)
Reviewed-on: https://go-review.googlesource.com/c/go/+/565755
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2024-02-28 17:54:30 +00:00
Ian Lance Taylor
6d31b27150 [release-branch.go1.21] runtime: don't let the tests leave core files behind
Also add a check that we didn't leave any core files behind.

For #65476.
Fixes #65478. 

Change-Id: I30444ef43ad1a8cc1cacd3b75280f2128e104939
Reviewed-on: https://go-review.googlesource.com/c/go/+/525175
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
(cherry picked from commit cffdfe8d2c)
Reviewed-on: https://go-review.googlesource.com/c/go/+/560896
2024-02-27 21:42:04 +00:00
Michael Anthony Knyszek
f38fca30a7 [release-branch.go1.21] cmd/cgo/internal/testsanitizers: disable location checking for clang
Pending a resolution to #65606, this CL marks clang's ASAN runtime as
unable to symbolize stack traces to unblock the LUCI clang builder.

For #65606.
For #65469.
Fixes #65640.

Change-Id: I649773085aff30e5703e7f7ac2c72a0430a015c2
Cq-Include-Trybots: luci.golang.try:go1.21-linux-amd64-clang15
Reviewed-on: https://go-review.googlesource.com/c/go/+/562675
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit d94ab597af)
Reviewed-on: https://go-review.googlesource.com/c/go/+/562999
2024-02-16 15:51:02 +00:00
Michael Anthony Knyszek
b214108e72 [release-branch.go1.21] internal/testenv: support the LUCI mobile builders in tests
This change updates the testenv tests to correctly match on future LUCI
builder names for mobile builders. This isn't a problem today because
those haven't been set up yet, but the builder names are structured and
it's clear where the modifiers will appear. Might as well set them up
now.

For #65473.
Fixes #65475.

Change-Id: I244b88a62a90312c0f3ff2360527d58531070362
Reviewed-on: https://go-review.googlesource.com/c/go/+/558597
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 5c7c24ce82)
Reviewed-on: https://go-review.googlesource.com/c/go/+/560895
2024-02-08 16:19:05 +00:00
Michael Anthony Knyszek
f997dfd33a [release-branch.go1.21] internal/testenv: allow "-noopt" anywhere in builder name in test
testenv's TestHasGoBuild test is supposed to allow noopt builders to not
have go build, but the pattern match is failing on the LUCI builders
where a test shard might have an additional "-test_only" suffix in the
builder name. Furthermore, in the LUCI world, "run mods" (the builder
type suffixes) are supposed to be well-defined and composable, so it
doesn't make sense to restrict "-noopt" to the builder suffix anyway.

This change modifies the test to allow "-noopt" to appear anywhere in
the builder name when checking if it's running on a noopt builder.

For #65470.
Fixes #65472.

Change-Id: I393818e3e8e452c7b0927cbc65726d552aa8ff8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/558596
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
(cherry picked from commit 93f0c0b25e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/560518
2024-02-08 16:18:55 +00:00
Gopher Robot
f29208030a [release-branch.go1.21] go1.21.7
Change-Id: I2c0bbd094a1e9a12576869437a362da40f76f22d
Reviewed-on: https://go-review.googlesource.com/c/go/+/562117
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2024-02-06 18:00:12 +00:00
Keith Randall
2fdad8af6d [release-branch.go1.21] cmd/internal/obj/arm64: fix frame pointer restore in epilogue
For leaf but nonzero-frame functions.

Currently we're not restoring it properly. We also need to restore
it before popping the stack frame, so that the frame won't get
clobbered by a signal handler in the meantime.

For #63830
Fixes #65449

Needs a test, but I'm not at all sure how we would actually do that. Leaving for inspiration.

Change-Id: I273a25f2a838f05a959c810145cccc5428eaf164
Reviewed-on: https://go-review.googlesource.com/c/go/+/538635
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Eric Fang <eric.fang@arm.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
(cherry picked from commit c9888bdfe2)
Reviewed-on: https://go-review.googlesource.com/c/go/+/560735
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2024-02-02 16:26:24 +00:00
Michael Matloob
01c93ad049 [release-branch.go1.21] cmd/go/internal/generate: call modload.InitWorkFile
This is necessary for go generate to enter workspace mode for
recognizing package paths in the workspace.

For #56098
Fixes #65351

Change-Id: I25f68de24f4189259353f63194823516e9d3d505
Reviewed-on: https://go-review.googlesource.com/c/go/+/559195
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
(cherry picked from commit b91bad7819)
Reviewed-on: https://go-review.googlesource.com/c/go/+/559235
2024-02-01 20:19:50 +00:00
Filippo Valsorda
db74bfba18 [release-branch.go1.21] Revert "crypto/internal/boring: upgrade module to fips-20220613" +1
This reverts CL 553855 ("crypto/internal/boring: upgrade module to
fips-20220613") and CL 553856 ("crypto/tls: align FIPS-only mode with
BoringSSL policy").

Fixes #65323
Updates #65321
Updates #64717
Updates #62372

Change-Id: I0938b97e5b4904e6532448b8ae76e920d03d0508
Reviewed-on: https://go-review.googlesource.com/c/go/+/558796
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 09b5de48e6)
Reviewed-on: https://go-review.googlesource.com/c/go/+/560275
2024-02-01 18:25:33 +00:00
qiulaidongfeng
916e6cddf1 [release-branch.go1.21] runtime: fix Pinner.Pin documentation
Fixes #63768

Change-Id: I01a9bb8f9af22a6b3f6534d431e3ea623875ed48
GitHub-Last-Rev: 7c5dd4edb1
GitHub-Pull-Request: golang/go#64920
Reviewed-on: https://go-review.googlesource.com/c/go/+/553395
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2024-01-31 21:20:32 +00:00
Bryan C. Mills
6552f3d4ac [release-branch.go1.21] cmd/go/internal/toolchain: apply the -modcacherw flag when downloading a module to determine what toolchain it needs
Fixes #64497.
Updates #64282.

Change-Id: I3f211c599ee70cb58254d0bc07eeb3c135124e58
Reviewed-on: https://go-review.googlesource.com/c/go/+/555436
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
(cherry picked from commit cc38c68ae0)
Reviewed-on: https://go-review.googlesource.com/c/go/+/559198
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-31 17:34:35 +00:00
Roland Shoemaker
3960318b87 [release-branch.go1.21] crypto/x509: properly gate test on macos version
Fixes the gating of TestIssue51759 by shelling out to sw_vers to check
what version of macOS we are on.

For #64677
Fixes #65380

Change-Id: I5eef4fa39e5449e7b2aa73864625c3abf002aef8
Reviewed-on: https://go-review.googlesource.com/c/go/+/549195
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 400e24a8be)
Reviewed-on: https://go-review.googlesource.com/c/go/+/559517
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-01-31 17:30:51 +00:00
Robert Griesemer
00f974eb1f [release-branch.go1.21] go/types, types2: don't lose position info of interface embeddings
Accurate position information for embedded types in interfaces is
crucial to identify the corresponding source file, and with that
the Go language version associated with that file. (The position
information is also important for proper error messages.)

Before this CL, the position information for embedded types was
discarded after type set computation, in the assumption that it
was not needed anymore. However, substitutions that update the
interface may lead to repeated type set computations which then
won't have the correct position information.

This CL does preserve the position information for embedded
types until the end of type checking (cleanup phase), and also
copy the position information during a substitution of the
interface.

The respective bug (#64759) doesn't seem to appear in 1.22 (most
likely because it's hidden by some of the changes made with respect
to the file version logic), but the existing code is still wrong.
The backport of this code to 1.21 and 1.20 fixes the issue in those
releases.

For #64759.
Fixes #65053.

Change-Id: I80f4004c9d79cb02eac6739c324c477706615102
Reviewed-on: https://go-review.googlesource.com/c/go/+/555296
Run-TryBot: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/555415
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-25 17:49:05 +00:00
Michael Pratt
2f91c16e68 [release-branch.go1.21] runtime: properly model rwmutex in lock ranking
(This cherry-pick combines CL 549536 and the follow-up fix CL 555055.)

Currently, lock ranking doesn't really try to model rwmutex. It records
the internal locks rLock and wLock, but in a subpar fashion:

1. wLock is held from lock to unlock, so it works OK, but it conflates
   write locks of all rwmutexes as rwmutexW, rather than allowing
   different rwmutexes to have different rankings.
2. rLock is an internal implementation detail that is only taken when
   there is contention in rlock. As as result, the reader lock path is
   almost never checked.

Add proper modeling. rwmutexR and rwmutexW remain as the ranks of the
internal locks, which have their own ordering. The new init method is
passed the ranks of the higher level lock that this represents, just
like lockInit for mutex.

execW ordered before MALLOC captures the case from #64722. i.e., there
can be allocation between BeforeFork and AfterFork.

For #64722.
Fixes #64761.

------

runtime: replace rwmutexR/W with per-rwmutex lock rank

CL 549536 intended to decouple the internal implementation of rwmutex
from the semantic meaning of an rwmutex read/write lock in the static
lock ranking.

Unfortunately, it was not thought through well enough. The internals
were represented with the rwmutexR and rwmutexW lock ranks. The idea was
that the internal lock ranks need not model the higher-level ordering,
since those have separate rankings. That is incorrect; rwmutexW is held
for the duration of a write lock, so it must be ranked before any lock
taken while any write lock is held, which is precisely what we were
trying to avoid.

This is visible in violations like:

        0 : execW 11 0x0
        1 : rwmutexW 51 0x111d9c8
        2 : fin 30 0x111d3a0
        fatal error: lock ordering problem

execW < fin is modeled, but rwmutexW < fin is missing.

Fix this by eliminating the rwmutexR/W lock ranks shared across
different types of rwmutex. Instead require users to define an
additional "internal" lock rank to represent the implementation details
of rwmutex.rLock. We can avoid an additional "internal" lock rank for
rwmutex.wLock because the existing writeRank has the same semantics for
semantic and internal locking. i.e., writeRank is held for the duration
of a write lock, which is exactly how rwmutex.wLock is used, so we can
use writeRank directly on wLock.

For #64722.

Cq-Include-Trybots: luci.golang.try:go1.21-linux-amd64-staticlockranking
Change-Id: I23335b28faa42fb04f1bc9da02fdf54d1616cd28
Reviewed-on: https://go-review.googlesource.com/c/go/+/549536
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 9b4b3e5acc)
(cherry picked from commit dcbe772469)
Reviewed-on: https://go-review.googlesource.com/c/go/+/554976
2024-01-25 17:20:02 +00:00
Alan Donovan
2540b1436f [release-branch.go1.21] slices: explicitly discard results of some functions
This will otherwise trigger an "unusedresult" vet check.

For #64978.
Fixes #65023.
Fixes #60058.

Change-Id: Ie19aded0f808d394f389452c3ff7f3edc1ed710d
Reviewed-on: https://go-review.googlesource.com/c/go/+/554196
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
(cherry picked from commit 8088b6db23)
Reviewed-on: https://go-review.googlesource.com/c/go/+/554756
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
2024-01-10 20:16:07 +00:00
Michael Pratt
7e34c4308f [release-branch.go1.21] runtime: clear g0 stack bounds in dropm
After CL 527715, needm uses callbackUpdateSystemStack to set the stack
bounds for g0 on an M from the extra M list. Since
callbackUpdateSystemStack is also used for recursive cgocallback, it
does nothing if the stack is already in bounds.

Currently, the stack bounds in an extra M may contain stale bounds from
a previous thread that used this M and then returned it to the extra
list in dropm.

Typically a new thread will not have an overlapping stack with an old
thread, but because the old thread has exited there is a small chance
that the C memory allocator will allocate the new thread's stack
partially or fully overlapping with the old thread's stack.

If this occurs, then callbackUpdateSystemStack will not update the stack
bounds. If in addition, the overlap is partial such that SP on
cgocallback is close to the recorded stack lower bound, then Go may
quickly "overflow" the stack and crash with "morestack on g0".

Fix this by clearing the stack bounds in dropm, which ensures that
callbackUpdateSystemStack will unconditionally update the bounds in
needm.

For #62440.
Fixes #63209.

Change-Id: Ic9e2052c2090dd679ed716d1a23a86d66cbcada7
Reviewed-on: https://go-review.googlesource.com/c/go/+/537695
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Bypass: Michael Pratt <mpratt@google.com>
(cherry picked from commit 1af424c196)
Reviewed-on: https://go-review.googlesource.com/c/go/+/549495
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2024-01-10 19:40:22 +00:00
Michael Pratt
491c1e7e95 [release-branch.go1.21] runtime: allow update of system stack bounds on callback from C thread
[This cherry-pick combines CL 527715, CL 527775, CL 527797, and
CL 529216.]

[This is a redo of CL 525455 with the test fixed on darwin by defining
_XOPEN_SOURCE, and disabled with android, musl, and openbsd, which do
not provide getcontext.]

Since CL 495855, Ms are cached for C threads calling into Go, including
the stack bounds of the system stack.

Some C libraries (e.g., coroutine libraries) do manual stack management
and may change stacks between calls to Go on the same thread.

Changing the stack if there is more Go up the stack would be
problematic. But if the calls are completely independent there is no
particular reason for Go to care about the changing stack boundary.

Thus, this CL allows the stack bounds to change in such cases. The
primary downside here (besides additional complexity) is that normal
systems that do not manipulate the stack may not notice unintentional
stack corruption as quickly as before.

Note that callbackUpdateSystemStack is written to be usable for the
initial setup in needm as well as updating the stack in cgocallbackg.

For #62440.
For #62130.
For #63209.

Change-Id: I0fe0134f865932bbaff1fc0da377c35c013bd768
Reviewed-on: https://go-review.googlesource.com/c/go/+/527715
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
(cherry picked from commit 4f9fe6d509)
(cherry picked from commit e8ba0579e2)
(cherry picked from commit a843991fdd)
(cherry picked from commit d110d7c42d)
Reviewed-on: https://go-review.googlesource.com/c/go/+/530480
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Bypass: Michael Pratt <mpratt@google.com>
2024-01-10 19:40:13 +00:00
Cherry Mui
caafb50c0d [release-branch.go1.21] runtime: don't print "unexpected SPWRITE" when printing traceback
The system stack often starts with a stack transition function
like "systemstack" or "mcall", which is marked as SPWRITE. When
unwinding a system stack for printing, we want the traceback stop
at the stack switching frame, but not print the "unexpected
SPWRITE" message.

Previously before CL 525835, we don't print the "unexpected
SPWRITE" message if unwindPrintErrors is set, i.e. printing a
stack trace. This CL restores this behavior.

Another possibility is not printing the message only on the system
stack. We don't expect a stack transition function to appear in a
user G.

Fixes #64935.

Change-Id: I173e89ead2cd4fbf1f0f8cca225f28718b5baebe
Reviewed-on: https://go-review.googlesource.com/c/go/+/531815
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
(cherry picked from commit 15a274b621)
Reviewed-on: https://go-review.googlesource.com/c/go/+/553476
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org>
2024-01-10 19:17:07 +00:00
Gopher Robot
cc85462b3d [release-branch.go1.21] go1.21.6
Change-Id: I85e48d54e6938b3882a0bb8f2de75fee6ecb2668
Reviewed-on: https://go-review.googlesource.com/c/go/+/554839
Commit-Queue: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-01-09 18:17:19 +00:00
Filippo Valsorda
d2cb140194 [release-branch.go1.21] crypto/tls: align FIPS-only mode with BoringSSL policy
This enables TLS 1.3, disables P-521, and disables non-ECDHE suites.

Updates #64717
Updates #62372
Fixes #64719

Change-Id: I3a65b239ef0198bbdbe5e55e0810e7128f90a091
Reviewed-on: https://go-review.googlesource.com/c/go/+/549975
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/553856
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2024-01-04 23:16:07 +00:00
Filippo Valsorda
368e2a9461 [release-branch.go1.21] crypto/internal/boring: upgrade module to fips-20220613
Also, add EVP_aead_aes_*_gcm_tls13 to the build, which we will need in a
following CL, to avoid rebuilding the syso twice.

Updates #64717
Updates #62372
Updates #64719

Change-Id: Ie4d853ad9b914c1095cad60694a1ae6f77dc22ce
Cq-Include-Trybots: luci.golang.try:go1.21-linux-amd64-boringcrypto
Reviewed-on: https://go-review.googlesource.com/c/go/+/549695
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/553855
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2024-01-04 23:16:05 +00:00
David Chase
8c6078adfb [release-branch.go1.21] runtime: add race annotations in IncNonDefault
Also use CompareAndSwap to make the code actually less racy.

Added a test which will be meaningful when run under the race
detector (tested it -race with broken fix in runtime, it failed).

This backport incorporates the correction in CL 551856,
using racereleasemerge instead of racerelease. 

Fixes #64757

Change-Id: I5972e08901d1adc8ba74858edad7eba91be1b0ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/549796
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit 3313bbb405)
Reviewed-on: https://go-review.googlesource.com/c/go/+/550236
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
TryBot-Bypass: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2024-01-04 21:37:29 +00:00
Keith Randall
f6b203c828 [release-branch.go1.21] maps: fix aliasing problems with Clone
Make sure to alloc+copy large keys and values instead of aliasing them,
when they might be updated by a future assignment.

Fixes #64475

Change-Id: Ie2226a81cf3897e4e2ee24472f2966d397ace53f
Reviewed-on: https://go-review.googlesource.com/c/go/+/546515
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
(cherry picked from commit 16d3040a84)
Reviewed-on: https://go-review.googlesource.com/c/go/+/547375
TryBot-Bypass: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
2024-01-04 21:37:00 +00:00
Michael Anthony Knyszek
43818206dc [release-branch.go1.21] runtime: put ReadMemStats debug assertions behind a double-check mode
ReadMemStats has a few assertions it makes about the consistency of the
stats it's about to produce. Specifically, how those stats line up with
runtime-internal stats. These checks are generally useful, but crashing
just because some stats are wrong is a heavy price to pay.

For a long time this wasn't a problem, but very recently it became a
real problem. It turns out that there's real benign skew that can happen
wherein sysmon (which doesn't synchronize with a STW) generates a trace
event when tracing is enabled, and may mutate some stats while
ReadMemStats is running its checks.

Fix this by synchronizing with both sysmon and the tracer. This is a bit
heavy-handed, but better that than false positives.

Also, put the checks behind a debug mode. We want to reduce the risk of
backporting this change, and again, it's not great to crash just because
user-facing stats are off. Still, enable this debug mode during the
runtime tests so we don't lose quite as much coverage from disabling
these checks by default.

For #64401.
Fixes #64410.

Change-Id: I9adb3e5c7161d207648d07373a11da8a5f0fda9a
Reviewed-on: https://go-review.googlesource.com/c/go/+/545277
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
(cherry picked from commit b2efd1de97)
Reviewed-on: https://go-review.googlesource.com/c/go/+/545557
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
TryBot-Bypass: Matthew Dempsky <mdempsky@google.com>
2024-01-04 21:34:32 +00:00
Michael Anthony Knyszek
cf65d74bc5 [release-branch.go1.21] runtime: add the disablethp GODEBUG setting
Go 1.21.1 and Go 1.22 have ceased working around an issue with Linux
kernel defaults for transparent huge pages that can result in excessive
memory overheads. (https://bugzilla.kernel.org/show_bug.cgi?id=93111)

Many Linux distributions disable huge pages altogether these days, so
this problem isn't quite as far-reaching as it used to be. Also, the
problem only affects Go programs with very particular memory usage
patterns.

That being said, because the runtime used to actively deal with this
problem (but with some unpredictable behavior), it's preventing users
that don't have a lot of control over their execution environment from
upgrading to Go beyond Go 1.20.

This change adds a GODEBUG to smooth over the transition. The GODEBUG
setting disables transparent huge pages for all heap memory on Linux,
which is much more predictable than restoring the old behavior.

For #64332.
Fixes #64561.

Change-Id: I73b1894337f0f0b1a5a17b90da1221e118e0b145
Reviewed-on: https://go-review.googlesource.com/c/go/+/547475
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit c915215af6)
Reviewed-on: https://go-review.googlesource.com/c/go/+/547636
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
2024-01-04 17:32:44 +00:00
Tolya Korniltsev
ad1ec60a5b [release-branch.go1.21] runtime/pprof: fix generics function names
profileBuilder is using Frame->Function as key for checking if we already
emitted a function. However for generics functions it has dots there [...],
so sometimes for different functions with different generics types,
the profileBuilder emits wrong functions.

For #64528
For #64609

Change-Id: I8b39245e0b18f4288ce758c912c6748f87cba39a
Reviewed-on: https://go-review.googlesource.com/c/go/+/546815
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
(cherry picked from commit 20a03fc713)
Reviewed-on: https://go-review.googlesource.com/c/go/+/549535
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
2024-01-04 17:22:01 +00:00
Michael Pratt
bbab863ada [release-branch.go1.21] os/signal: skip nohup tests on darwin builders
The new LUCI builders have a temporary limitation that breaks nohup.
Skip nohup tests there.

For #63875.
Fixes #63911.

Cq-Include-Trybots: luci.golang.try:go1.21-darwin-amd64_13
Change-Id: Ia9ffecea7310f84a21f6138d8f8cdfc5e1392307
Reviewed-on: https://go-review.googlesource.com/c/go/+/538698
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
(cherry picked from commit a334c45273)
Reviewed-on: https://go-review.googlesource.com/c/go/+/546022
2023-12-08 18:47:59 +00:00
Michael Pratt
7dc67e8f29 [release-branch.go1.21] os/signal: remove go t.Run from TestNohup
Since CL 226138, TestNohup has a bit of a strange construction: it wants
to run the "uncaught" subtests in parallel with each other, and the
"nohup" subtests in parallel with each other, but also needs join
between "uncaught" and "nohop" so it can Stop notifying for SIGHUP.

It achieves this by doing `go t.Run` with a WaitGroup rather than using
`t.Parallel` in the subtest (which would make `t.Run` return immediately).

However, this makes things more difficult to understand than necessary.
As noted on https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks,
a second layer of subtest can be used to join parallel subtests.

Switch to this form, which makes the test simpler to follow
(particularly the cleanup that goes with "uncaught").

For #63799.
For #63911.

Change-Id: Ibfce0f439508a7cfca848c7ccfd136c9c453ad8b
Reviewed-on: https://go-review.googlesource.com/c/go/+/538899
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
(cherry picked from commit 5622a4b205)
Reviewed-on: https://go-review.googlesource.com/c/go/+/546021
2023-12-07 20:35:27 +00:00
Matthew Dempsky
9f8b3ac8c4 [release-branch.go1.21] cmd/compile: fix escape analysis of string min/max
When I was plumbing min/max support through the compiler, I was
thinking mostly about numeric argument types. As a result, I forgot
that escape analysis would need to be aware that min/max can operate
on string values, which contain pointers.

Updates #64565.
Fixes #64567.

Change-Id: I36127ce5a2da942401910fa0f9de922726c9f94d
Reviewed-on: https://go-review.googlesource.com/c/go/+/547715
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 34416d7f6f)
Reviewed-on: https://go-review.googlesource.com/c/go/+/547757
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
2023-12-06 21:31:12 +00:00
Keith Randall
8c840b10d0 [release-branch.go1.21] cmd/compile: fix memcombine pass for big endian, > 1 byte elements
The shift amounts were wrong in this case, leading to miscompilation
of load combining.

Also the store combining was not triggering when it should.

Fixes #64472

Change-Id: Iaeb08972c5fc1d6f628800334789c6af7216e87b
Reviewed-on: https://go-review.googlesource.com/c/go/+/546355
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/546356
2023-12-06 21:29:58 +00:00
Gopher Robot
6018ad99a4 [release-branch.go1.21] go1.21.5
Change-Id: I203463900d2c63bc398c22593d0908c4f33a6990
Reviewed-on: https://go-review.googlesource.com/c/go/+/547416
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-12-05 18:12:56 +00:00
Damien Neil
ec8c526e4b [release-branch.go1.21] net/http: limit chunked data overhead
The chunked transfer encoding adds some overhead to
the content transferred. When writing one byte per
chunk, for example, there are five bytes of overhead
per byte of data transferred: "1\r\nX\r\n" to send "X".

Chunks may include "chunk extensions",
which we skip over and do not use.
For example: "1;chunk extension here\r\nX\r\n".

A malicious sender can use chunk extensions to add
about 4k of overhead per byte of data.
(The maximum chunk header line size we will accept.)

Track the amount of overhead read in chunked data,
and produce an error if it seems excessive.

Updates #64433
Fixes #64435
Fixes CVE-2023-39326

Change-Id: I40f8d70eb6f9575fb43f506eb19132ccedafcf39
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2076135
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
(cherry picked from commit 3473ae72ee66c60744665a24b2fde143e8964d4f)
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2095408
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/547356
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-05 17:18:23 +00:00
Jorropo
f7a79cb5fc [release-branch.go1.21] cmd/compile: fix findIndVar so it does not match disjointed loop headers
Fix #63984

parseIndVar, prove and maybe more are on the assumption that the loop header
is a single block. This can be wrong, ensure we don't match theses cases we
don't know how to handle.

In the future we could update them so that they know how to handle such cases
but theses cases seems rare so I don't think the value would be really high.
We could also run a loop canonicalization pass first which could handle this.

The repro case looks weird because I massaged it so it would crash with the
previous compiler.

Change-Id: I4aa8afae9e90a17fa1085832250fc1139c97faa6
Reviewed-on: https://go-review.googlesource.com/c/go/+/539977
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 8b4e1259d0)
Reviewed-on: https://go-review.googlesource.com/c/go/+/540535
Reviewed-by: Jorropo <jorropo.pgm@gmail.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-11-28 20:12:59 +00:00
Michael Anthony Knyszek
3684d19c20 [release-branch.go1.21] runtime: call enableMetadataHugePages and its callees on the systemstack
These functions acquire the heap lock. If they're not called on the
systemstack, a stack growth could cause a self-deadlock since stack
growth may allocate memory from the page heap.

This has been a problem for a while. If this is what's plaguing the
ppc64 port right now, it's very surprising (and probably just
coincidental) that it's showing up now.

For #64050.
For #64062.
For #64067.
Fixes #64073.

Change-Id: I2b95dc134d17be63b9fe8f7a3370fe5b5438682f
Reviewed-on: https://go-review.googlesource.com/c/go/+/541635
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Paul Murphy <murp@ibm.com>
(cherry picked from commit 5f08b44799)
Reviewed-on: https://go-review.googlesource.com/c/go/+/541955
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-11-28 18:46:34 +00:00
Roland Shoemaker
9e43850a32 [release-branch.go1.21] crypto/rand,runtime: switch RtlGenRandom for ProcessPrng
RtlGenRandom is a semi-undocumented API, also known as
SystemFunction036, which we use to generate random data on Windows.
It's definition, in cryptbase.dll, is an opaque wrapper for the
documented API ProcessPrng. Instead of using RtlGenRandom, switch to
using ProcessPrng, since the former is simply a wrapper for the latter,
there should be no practical change on the user side, other than a minor
change in the DLLs we load.

Updates #53192
Fixes #64413

Change-Id: Ie6891bf97b1d47f5368cccbe92f374dba2c2672a
Reviewed-on: https://go-review.googlesource.com/c/go/+/536235
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Auto-Submit: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
(cherry picked from commit 693def151a)
Reviewed-on: https://go-review.googlesource.com/c/go/+/545355
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-11-28 17:48:47 +00:00
Damien Neil
8caf4bb3e7 [release-branch.go1.21] path/filepath: consider \\?\c: as a volume on Windows
While fixing several bugs in path handling on Windows,
beginning with \\?\.

Prior to #540277, VolumeName considered the first path component
after the \\?\ prefix to be part of the volume name.
After, it considered only the \\? prefix to be the volume name.

Restore the previous behavior.

For #64028.
Fixes #64041.

Change-Id: I6523789e61776342800bd607fb3f29d496257e68
Reviewed-on: https://go-review.googlesource.com/c/go/+/541175
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
(cherry picked from commit eda42f7c60)
Reviewed-on: https://go-review.googlesource.com/c/go/+/541521
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-11-28 16:59:27 +00:00
Bryan C. Mills
23c943e529 [release-branch.go1.21] cmd/go/internal/vcs: error out if the requested repo does not support a secure protocol
Updates #63845.
Fixes #63973.

Change-Id: If86d6b13d3b55877b35c087112bd76388c9404b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/539321
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
(cherry picked from commit be26ae18ca)
Reviewed-on: https://go-review.googlesource.com/c/go/+/540257
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-11-27 21:12:17 +00:00
Bryan C. Mills
4952f41180 [release-branch.go1.21] cmd/go: allow 'go mod download' to switch toolchains if called with explicit arguments
Fixes #62055.
Updates #62054.

Change-Id: I4ea24070f7d9aa4964c2f215836602068058f718
(cherry picked from CL 540779 and CL 537480)
Reviewed-on: https://go-review.googlesource.com/c/go/+/539697
Auto-Submit: Heschi Kreinick <heschi@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2023-11-09 21:37:26 +00:00
Bryan C. Mills
f26fa05522 [release-branch.go1.21] os: report IO_REPARSE_TAG_DEDUP files as regular in Stat and Lstat
Prior to CL 460595, Lstat reported most reparse points as regular
files. However, reparse points can in general implement unusual
behaviors (consider IO_REPARSE_TAG_AF_UNIX or IO_REPARSE_TAG_LX_CHR),
and Windows allows arbitrary user-defined reparse points, so in
general we must not assume that an unrecognized reparse tag represents
a regular file; in CL 460595, we began marking them as irregular.

As it turns out, the Data Deduplication service on Windows Server runs
an Optimization job that turns regular files into reparse files with
the tag IO_REPARSE_TAG_DEDUP. Those files still behave more-or-less
like regular files, in that they have well-defined sizes and support
random-access reads and writes, so most programs can treat them as
regular files without difficulty. However, they are still reparse
files: as a result, on servers with the Data Deduplication service
enabled, files could arbitrarily change from “regular” to “irregular”
without explicit user intervention.

Since dedup files are converted in the background and otherwise behave
like regular files, this change adds a special case to report DEDUP
reparse points as regular.

Fixes #63764.
Updates #63429.

No test because to my knowledge we don't have any Windows builders
that have the deduplication service enabled, nor do we have a way to
reliably guarantee the existence of an IO_REPARSE_TAG_DEDUP file.

(In theory we could add a builder with the service enabled on a
specific volume, write a test that encodes knowledge of that volume,
and use the GO_BUILDER_NAME environment variable to run that test only
on the specially-configured builders. However, I don't currently have
the bandwidth to reconfigure the builders in this way, and given the
simplicity of the change I think it is unlikely to regress
accidentally.)

Change-Id: I649e7ef0b67e3939a980339ce7ec6a20b31b23a1
Cq-Include-Trybots: luci.golang.try:go1.21-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/538218
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Heschi Kreinick <heschi@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-08 18:13:06 +00:00
Andy Pan
8ae493b5b8 [release-branch.go1.21] internal/poll: add SPLICE_F_NONBLOCK flag for splice to avoid inconsistency with O_NONBLOCK
Fixes #63801
Updates #59041
Updates #63795

Details: https://github.com/golang/go/issues/59041#issuecomment-1766610087

Change-Id: Id3fc1df6d86b7c4cc383d09f9465fa8f4cc2a559
Reviewed-on: https://go-review.googlesource.com/c/go/+/536015
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
(cherry picked from commit 40cdf69fc9)
Reviewed-on: https://go-review.googlesource.com/c/go/+/538117
Auto-Submit: Heschi Kreinick <heschi@google.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2023-11-08 18:02:42 +00:00
Keith Randall
b9f245b8d3 [release-branch.go1.21] cmd/compile: ensure pointer arithmetic happens after the nil check
Have nil checks return a pointer that is known non-nil. Users of
that pointer can use the result, ensuring that they are ordered
after the nil check itself.

The order dependence goes away after scheduling, when we've fixed
an order. At that point we move uses back to the original pointer
so it doesn't change regalloc any.

This prevents pointer arithmetic on nil from being spilled to the
stack and then observed by a stack scan.

Fixes #63743

Change-Id: I1a5fa4f2e6d9000d672792b4f90dfc1b7b67f6ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/537775
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
(cherry picked from commit 962ccbef91)
Reviewed-on: https://go-review.googlesource.com/c/go/+/538717
Auto-Submit: Heschi Kreinick <heschi@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2023-11-07 21:29:46 +00:00
Keith Randall
caacf3a09a [release-branch.go1.21] cmd/compile: handle constant pointer offsets in dead store elimination
Update #63743

Change-Id: I163c6038c13d974dc0ca9f02144472bc05331826
Reviewed-on: https://go-review.googlesource.com/c/go/+/538595
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@google.com>
(cherry picked from commit 43b57b8516)
Reviewed-on: https://go-review.googlesource.com/c/go/+/538857
Auto-Submit: Heschi Kreinick <heschi@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2023-11-07 21:29:17 +00:00
Dmitri Shuralyov
1e91861f67 [release-branch.go1.21] syscall: copy rlimit.go's build constraint to rlimit_test.go
Tests in rlimit_test.go exist to test the behavior of automatically
bumping RLIMIT_NOFILE on Unix implemented in rlimit.go (issue #46279),
with darwin-specific behavior split out into rlimit_darwin.go and
the rest left empty in rlimit_stub.go.

Since the behavior happens only on Unix, it doesn't make sense to test
it on other platforms. Copy rlimit.go's 'unix' build constraint to
rlimit_test.go to accomplish that.

Leave out the simplification of the build constraint in rlimit_stub.go
so that this CL remains a test-only fix.

In particular, this fixes a problem where TestOpenFileLimit was
failing in some environments when testing the wasip1/wasm port.
The RLIMIT_NOFILE bumping behavior isn't implemented there, so
the test was testing the environment and not the Go project.

Updates #46279.
For #61116.
Fixes #63994.

Change-Id: Ic993f9cfc021d4cda4fe3d7fed8e2e180f78a2ca
Cq-Include-Trybots: luci.golang.try:go1.21-wasip1-wasm_wasmtime
Reviewed-on: https://go-review.googlesource.com/c/go/+/539435
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
(cherry picked from commit b7cbcf0c27)
Reviewed-on: https://go-review.googlesource.com/c/go/+/540615
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Heschi Kreinick <heschi@google.com>
2023-11-07 20:32:49 +00:00
Gopher Robot
ed817f1c40 [release-branch.go1.21] go1.21.4
Change-Id: I3d607ba9f701a76a46f3ab3223fa83e5c517d285
Reviewed-on: https://go-review.googlesource.com/c/go/+/540517
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-11-07 17:38:04 +00:00
Damien Neil
9e933c189c [release-branch.go1.21] path/filepath: fix various issues in parsing Windows paths
On Windows, A root local device path is a path which begins with
\\?\ or \??\.  A root local device path accesses the DosDevices
object directory, and permits access to any file or device on the
system. For example \??\C:\foo is equivalent to common C:\foo.

The Clean, IsAbs, IsLocal, and VolumeName functions did not
recognize root local device paths beginning with \??\.

Clean could convert a rooted path such as \a\..\??\b into
the root local device path \??\b. It will now convert this
path into .\??\b.

IsAbs now correctly reports paths beginning with \??\
as absolute.

IsLocal now correctly reports paths beginning with \??\
as non-local.

VolumeName now reports the \??\ prefix as a volume name.

Join(`\`, `??`, `b`) could convert a seemingly innocent
sequence of path elements into the root local device path
\??\b. It will now convert this to \.\??\b.

In addition, the IsLocal function did not correctly
detect reserved names in some cases:

  - reserved names followed by spaces, such as "COM1 ".
  - "COM" or "LPT" followed by a superscript 1, 2, or 3.

IsLocal now correctly reports these names as non-local.

For #63713
Fixes #63715
Fixes CVE-2023-45283
Fixes CVE-2023-45284

Change-Id: I446674a58977adfa54de7267d716ac23ab496c54
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2040691
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2072596
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/540276
Auto-Submit: Heschi Kreinick <heschi@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-07 16:36:54 +00:00
Mauri de Souza Meneguzzo
434af8537e [release-branch.go1.21] net/http: pull http2 underflow fix from x/net/http2
After CL 534295 was merged to fix a CVE it introduced
an underflow when we try to decrement sc.curHandlers
in handlerDone.

Pull in a fix from x/net/http2:
http2: fix underflow in http2 server push
https://go-review.googlesource.com/c/net/+/535595

For #63511
Fixes #63560

Change-Id: I5c678ce7dcc53635f3ad5e4999857cb120dfc1ab
GitHub-Last-Rev: 587ffa3caf
GitHub-Pull-Request: golang/go#63561
Reviewed-on: https://go-review.googlesource.com/c/go/+/535575
Run-TryBot: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit 0046c1414c)
Reviewed-on: https://go-review.googlesource.com/c/go/+/537996
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-10-30 21:11:25 +00:00
Cherry Mui
7b04d81cbc [release-branch.go1.21] runtime/cgo: avoid taking the address of crosscall2 in code
Currently, set_crosscall2 takes the address of crosscall2 without
using the GOT, which, on some architectures, results in a
PC-relative relocation (e.g. R_AARCH64_ADR_PREL_PG_HI21 on ARM64)
to the crosscall2 symbol. But crosscall2 is dynamically exported,
so the C linker thinks it may bind to a symbol from a different
DSO. Some C linker may not like a PC-relative relocation to such a
symbol. Using a local trampoline to avoid taking the address of a
dynamically exported symbol.

It may be possible to not dynamically export crosscall2. But this
CL is safer for backport. Later we may remove the trampolines
after unexport crosscall2, if they are not needed.

Fixes #63509.
Updates #62556.

Change-Id: Id28457f65ef121d3f87d8189803abc65ed453283
Reviewed-on: https://go-review.googlesource.com/c/go/+/533535
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
(cherry picked from commit 872d7181f4)
Reviewed-on: https://go-review.googlesource.com/c/go/+/534915
Reviewed-by: David Chase <drchase@google.com>
2023-10-30 20:36:26 +00:00
Matthew Dempsky
f9a31cda3c [release-branch.go1.21] cmd/compile/internal/typecheck: fix closure field naming
When creating the struct type to hold variables captured by a function
literal, we currently reuse the captured variable names as fields.

However, there's no particular reason to do this: these struct types
aren't visible to users, and it adds extra complexity in making sure
fields belong to the correct packages.

Further, it turns out we were getting that subtly wrong. If two
function literals from different packages capture variables with
identical names starting with an uppercase letter (and in the same
order and with corresponding identical types) end up in the same
function (e.g., due to inlining), then we could end up creating
closure struct types that are "different" (i.e., not types.Identical)
yet end up with equal LinkString representations (which violates
LinkString's contract).

The easy fix is to just always use simple, exported, generated field
names in the struct. This should allow further struct reuse across
packages too, and shrink binary sizes slightly.

For #62498.
Fixes #62545.

Change-Id: I9c973f5087bf228649a8f74f7dc1522d84a26b51
Reviewed-on: https://go-review.googlesource.com/c/go/+/527135
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit e3ce312621)
Reviewed-on: https://go-review.googlesource.com/c/go/+/534916
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-10-13 16:12:31 +00:00
Robert Griesemer
64b6c48107 [release-branch.go1.21] go/types, types2: don't use generics
This fixes cherry-pick CL 531998.

For #63339.

Change-Id: I6dac0909ca85d68684ce36025284d25db32e0b15
Reviewed-on: https://go-review.googlesource.com/c/go/+/535135
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
2023-10-13 15:10:45 +00:00
Michael Anthony Knyszek
ef6993f327 [release-branch.go1.21] runtime: don't eagerly collapse hugepages
This has caused performance issues in production environments.

MADV_COLLAPSE can go into direct reclaim, but we call it with the heap
lock held. This means that the process could end up stalled fairly
quickly if just one allocating goroutine ends up in the madvise call, at
least until the madvise(MADV_COLLAPSE) call returns. A similar issue
occurred with madvise(MADV_HUGEPAGE), because that could go into direct
reclaim on any page fault for MADV_HUGEPAGE-marked memory.

My understanding was that the calls to madvise(MADV_COLLAPSE) were
fairly rare, and it's "best-effort" nature prevented it from going into
direct reclaim often, but this was wrong. It tends to be fairly
heavyweight even when it doesn't end up in direct reclaim, and it's
almost certainly not worth it.

Disable it until further notice and let the kernel fully dictate
hugepage policy. The updated scavenger policy is still more hugepage
friendly by delaying scavening until hugepages are no longer densely
packed, so we don't lose all that much.

The Sweet benchmarks show a minimal difference. A couple less realistic
benchmarks seem to slow down a bit; they might just be getting unlucky
with what the kernel decides to back with a huge page. Some benchmarks
on the other hand improve. Overall, it's a wash.

name                  old time/op            new time/op            delta
BiogoIgor                        13.1s ± 1%             13.2s ± 2%    ~     (p=0.182 n=9+10)
BiogoKrishna                     12.0s ± 1%             12.1s ± 1%  +1.23%  (p=0.002 n=9+10)
BleveIndexBatch100               4.51s ± 4%             4.56s ± 3%    ~     (p=0.393 n=10+10)
EtcdPut                         20.2ms ± 4%            19.8ms ± 2%    ~     (p=0.079 n=10+9)
EtcdSTM                          109ms ± 3%             111ms ± 3%  +1.63%  (p=0.035 n=10+10)
GoBuildKubelet                   31.2s ± 1%             31.3s ± 1%    ~     (p=0.780 n=9+10)
GoBuildKubeletLink               7.77s ± 0%             7.81s ± 2%    ~     (p=0.237 n=8+10)
GoBuildIstioctl                  31.8s ± 1%             31.7s ± 0%    ~     (p=0.136 n=9+9)
GoBuildIstioctlLink              7.88s ± 1%             7.89s ± 1%    ~     (p=0.720 n=9+10)
GoBuildFrontend                  11.7s ± 1%             11.8s ± 1%    ~     (p=0.278 n=10+9)
GoBuildFrontendLink              1.15s ± 4%             1.15s ± 5%    ~     (p=0.387 n=9+9)
GopherLuaKNucleotide             19.7s ± 1%             20.6s ± 0%  +4.48%  (p=0.000 n=10+10)
MarkdownRenderXHTML              194ms ± 3%             196ms ± 3%    ~     (p=0.356 n=9+10)
Tile38QueryLoad                  633µs ± 2%             629µs ± 2%    ~     (p=0.075 n=10+10)

name                  old average-RSS-bytes  new average-RSS-bytes  delta
BiogoIgor                       69.2MB ± 3%            68.4MB ± 1%    ~     (p=0.190 n=10+10)
BiogoKrishna                    4.40GB ± 0%            4.40GB ± 0%    ~     (p=0.605 n=9+9)
BleveIndexBatch100               195MB ± 3%             195MB ± 2%    ~     (p=0.853 n=10+10)
EtcdPut                          107MB ± 4%             108MB ± 3%    ~     (p=0.190 n=10+10)
EtcdSTM                         91.6MB ± 5%            92.6MB ± 4%    ~     (p=0.481 n=10+10)
GoBuildKubelet                  2.26GB ± 1%            2.28GB ± 1%  +1.22%  (p=0.000 n=10+10)
GoBuildIstioctl                 1.53GB ± 0%            1.53GB ± 0%  +0.21%  (p=0.017 n=9+10)
GoBuildFrontend                  556MB ± 1%             554MB ± 2%    ~     (p=0.497 n=9+10)
GopherLuaKNucleotide            39.0MB ± 3%            39.0MB ± 1%    ~     (p=1.000 n=10+8)
MarkdownRenderXHTML             21.2MB ± 2%            21.4MB ± 3%    ~     (p=0.190 n=10+10)
Tile38QueryLoad                 5.99GB ± 2%            6.02GB ± 0%    ~     (p=0.243 n=10+9)

name                  old peak-RSS-bytes     new peak-RSS-bytes     delta
BiogoIgor                       90.2MB ± 4%            89.2MB ± 2%    ~     (p=0.143 n=10+10)
BiogoKrishna                    4.49GB ± 0%            4.49GB ± 0%    ~     (p=0.190 n=10+10)
BleveIndexBatch100               283MB ± 8%             274MB ± 6%    ~     (p=0.075 n=10+10)
EtcdPut                          147MB ± 4%             149MB ± 2%  +1.55%  (p=0.034 n=10+8)
EtcdSTM                          117MB ± 5%             117MB ± 4%    ~     (p=0.905 n=9+10)
GopherLuaKNucleotide            44.9MB ± 1%            44.6MB ± 1%    ~     (p=0.083 n=8+8)
MarkdownRenderXHTML             22.0MB ± 8%            22.1MB ± 9%    ~     (p=0.436 n=10+10)
Tile38QueryLoad                 6.24GB ± 2%            6.29GB ± 2%    ~     (p=0.218 n=10+10)

name                  old peak-VM-bytes      new peak-VM-bytes      delta
BiogoIgor                       1.33GB ± 0%            1.33GB ± 0%    ~     (p=0.504 n=10+9)
BiogoKrishna                    5.77GB ± 0%            5.77GB ± 0%    ~     (p=1.000 n=10+9)
BleveIndexBatch100              3.53GB ± 0%            3.53GB ± 0%    ~     (p=0.642 n=10+10)
EtcdPut                         12.1GB ± 0%            12.1GB ± 0%    ~     (p=0.564 n=10+10)
EtcdSTM                         12.1GB ± 0%            12.1GB ± 0%    ~     (p=0.633 n=10+10)
GopherLuaKNucleotide            1.26GB ± 0%            1.26GB ± 0%    ~     (p=0.297 n=9+10)
MarkdownRenderXHTML             1.26GB ± 0%            1.26GB ± 0%    ~     (p=0.069 n=10+10)
Tile38QueryLoad                 7.47GB ± 2%            7.53GB ± 2%    ~     (p=0.280 n=10+10)

name                  old p50-latency-ns     new p50-latency-ns     delta
EtcdPut                          19.8M ± 5%             19.3M ± 3%  -2.74%  (p=0.043 n=10+9)
EtcdSTM                          81.4M ± 4%             83.4M ± 4%  +2.46%  (p=0.029 n=10+10)
Tile38QueryLoad                   241k ± 1%              240k ± 1%    ~     (p=0.393 n=10+10)

name                  old p90-latency-ns     new p90-latency-ns     delta
EtcdPut                          30.4M ± 5%             30.6M ± 5%    ~     (p=0.971 n=10+10)
EtcdSTM                           222M ± 3%              226M ± 4%    ~     (p=0.063 n=10+10)
Tile38QueryLoad                   687k ± 2%              691k ± 1%    ~     (p=0.173 n=10+8)

name                  old p99-latency-ns     new p99-latency-ns     delta
EtcdPut                          42.3M ±10%             41.4M ± 7%    ~     (p=0.353 n=10+10)
EtcdSTM                           486M ± 7%              487M ± 4%    ~     (p=0.579 n=10+10)
Tile38QueryLoad                  6.43M ± 2%             6.37M ± 3%    ~     (p=0.280 n=10+10)

name                  old ops/s              new ops/s              delta
EtcdPut                          48.6k ± 3%             49.5k ± 2%    ~     (p=0.065 n=10+9)
EtcdSTM                          9.09k ± 2%             8.95k ± 3%  -1.56%  (p=0.045 n=10+10)
Tile38QueryLoad                  28.4k ± 1%             28.6k ± 1%  +0.87%  (p=0.016 n=9+10)

Fixes #63335.
For #63334.
Related to #61718 and #59960.

Change-Id: If84c5a8685825d43c912a71418f2597e44e867e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/531816
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
(cherry picked from commit 595deec3dd)
Reviewed-on: https://go-review.googlesource.com/c/go/+/532255
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-10-12 23:27:00 +00:00
Robert Griesemer
bae01521f3 [release-branch.go1.21] go/types, types2: don't implicitly modify an argument function's type
See the comment in the (very small) fix for a detailed description.
Use the opportunity to introduce a generic clone function which may
be useful elsewhere.

Fixes #63339.

Change-Id: Ic63c6b8bc443011b1a201908254f7d062e1aec71
Reviewed-on: https://go-review.googlesource.com/c/go/+/532157
Run-TryBot: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/531998
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-10-12 23:16:08 +00:00
Than McIntosh
236c07c049 [release-branch.go1.21] cmd/link: split text sections for arm 32-bit
This CL is a roll-forward (tweaked slightly) of CL 467715, which
turned on text section splitting for GOARCH=arm. The intent is to
avoid recurrent problems with external linking where there is a
disagreement between the Go linker and the external linker over
whether a given branch will reach. In the past our approach has been
to tweak the reachability calculations slightly to try to work around
potential linker problems, but this hasn't proven to be very robust;
section splitting seems to offer a better long term fix.

Updates #58425.
Fixes #63317.

Change-Id: I7372d41abce84097906a3d0805b6b9c486f345d6
Reviewed-on: https://go-review.googlesource.com/c/go/+/531795
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit 1e69040920)
Reviewed-on: https://go-review.googlesource.com/c/go/+/532096
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-10-12 22:36:36 +00:00
Dmitri Shuralyov
9465990e0e [release-branch.go1.21] all: tidy dependency versioning after release
Done with:

go get golang.org/x/net@internal-branch.go1.21-vendor
go mod tidy
go mod vendor
go generate net/http  # zero diff since CL 534235 already did this

For #63417.
For #63427.
For CVE-2023-39325.

Change-Id: Ib258e0d8165760a1082e02c2f4c5ce7d2a3c3c90
Reviewed-on: https://go-review.googlesource.com/c/go/+/534415
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-10-10 18:10:52 +00:00
Gopher Robot
883f062fc0 [release-branch.go1.21] go1.21.3
Change-Id: Icdde32852b305006d7064cfe8bbe143128191842
Reviewed-on: https://go-review.googlesource.com/c/go/+/534337
TryBot-Bypass: Dmitri Shuralyov <dmitshur@google.com>
Commit-Queue: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
2023-10-10 16:31:03 +00:00
Damien Neil
24ae2d9272 [release-branch.go1.21] net/http: regenerate h2_bundle.go
Pull in a security fix from x/net/http2:
http2: limit maximum handler goroutines to MaxConcurrentStreamso

For #63417
Fixes #63427
Fixes CVE-2023-39325

Change-Id: I70626734e6d56edf508f27a5b055ddf96d806eeb
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2047402
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/534235
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-10-10 16:10:13 +00:00
Gopher Robot
26b5783b72 [release-branch.go1.21] go1.21.2
Change-Id: Ie06bc8d8e8e170f13ed938a33038931513eaf067
Reviewed-on: https://go-review.googlesource.com/c/go/+/533238
Auto-Submit: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-10-05 19:06:58 +00:00
Ian Lance Taylor
2ddfc04d12 [release-branch.go1.21] cmd/compile: use absolute file name in isCgo check
For #23672
Updates #63211
Fixes #63214
Fixes CVE-2023-39323

Change-Id: I4586a69e1b2560036afec29d53e53cf25e6c7352
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2032884
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
(cherry picked from commit 9b19e751918dd218035811b1ef83a8c2693b864a)
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2037958
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/533215
Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-10-05 18:07:27 +00:00
Keith Randall
a15ef1bb0f [release-branch.go1.21] cmd/compile: absorb InvertFlags into Noov comparisons
Unfortunately, there isn't a single op that provides the resulting
computation.
At least, I couldn't find one.

Fixes #62506

Change-Id: I236f3965b827aaeb3d70ef9fe89be66b116494f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/526276
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@google.com>
(cherry picked from commit fb5bdb4cc9)
Reviewed-on: https://go-review.googlesource.com/c/go/+/526521
Reviewed-by: Than McIntosh <thanm@google.com>
2023-09-25 15:53:16 +00:00
Keith Randall
41d71a5afa [release-branch.go1.21] cmd/compile: reset memcombine correctly between basic blocks
Not sure why this bug didn't cause a complete failure, but it
certainly makes for doing a lot more work than is necessary.

Fixes #62668

Change-Id: If0be4acb6eafc3d7eeb42d2f4263c21b4e6e1c7d
Reviewed-on: https://go-review.googlesource.com/c/go/+/527699
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
(cherry picked from commit 08cdfd06ed)
Reviewed-on: https://go-review.googlesource.com/c/go/+/528795
Reviewed-by: Keith Randall <khr@google.com>
2023-09-25 13:05:39 +00:00
Michael Anthony Knyszek
0b6b0a275a [release-branch.go1.21] runtime: always lock OS thread in debugcall
Right now debuggers like Delve rely on the new goroutine created to run
a debugcall function to run on the same thread it started on, up until
it hits itself with a SIGINT as part of the debugcall protocol.

That's all well and good, except debugCallWrap1 isn't particularly
careful about not growing the stack. For example, if the new goroutine
happens to have a stale preempt flag, then it's possible a stack growth
will cause a roundtrip into the scheduler, possibly causing the
goroutine to switch to another thread.

Previous attempts to just be more careful around debugCallWrap1 were
helpful, but insufficient. This change takes everything a step further
and always locks the debug call goroutine and the new goroutine it
creates to the OS thread.

For #61732.
Fixes #62509.

Change-Id: I038f3a4df30072833e27e6a5a1ec01806a32891f
Reviewed-on: https://go-review.googlesource.com/c/go/+/515637
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alessandro Arzilli <alessandro.arzilli@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
(cherry picked from commit d9a4b24a17)
Reviewed-on: https://go-review.googlesource.com/c/go/+/526576
2023-09-25 13:03:10 +00:00
Cherry Mui
cd671a1180 [release-branch.go1.21] runtime: increase g0 stack size in non-cgo case
Currently, for non-cgo programs, the g0 stack size is 8 KiB on
most platforms. With PGO which could cause aggressive inlining in
the runtime, the runtime stack frames are larger and could
overflow the 8 KiB g0 stack. Increase it to 16 KiB. This is only
one per OS thread, so it shouldn't increase memory use much.

Updates #62120.
Updates #62489.
Fixes #62537.

Change-Id: I565b154517021f1fd849424dafc3f0f26a755cac
Reviewed-on: https://go-review.googlesource.com/c/go/+/526995
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit c6d550a668)
Reviewed-on: https://go-review.googlesource.com/c/go/+/527055
2023-09-22 18:58:01 +00:00
Cherry Mui
fc57cc31a0 [release-branch.go1.21] cmd/link: suppress -bind_at_load deprecation warning for ld-prime
ld-prime emits a deprecation warning for -bind_at_load. The flag
is needed for plugins to not deadlock (#38824) when linking with
older darwin linker. It is supposedly not needed with newer linker
when chained fixups are used. For now, we always pass it, and
suppress the warning.

Updates #61229.
For #62598.

Change-Id: I4b8a6f864a460c40dc38adbb533f664f7fd5343c
Reviewed-on: https://go-review.googlesource.com/c/go/+/508696
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com>
(cherry picked from commit 040dbf9c18)
Reviewed-on: https://go-review.googlesource.com/c/go/+/527817
2023-09-21 22:22:44 +00:00
Cherry Mui
9bec49cf52 [release-branch.go1.21] cmd/link: disable DWARF by default in c-shared mode on darwin
[This is a (manual) backport of CL 527415 to Go 1.21.]

Currently, linking a Go c-shared object with C code using Apple's
new linker, it fails with

% cc a.c go.so
ld: segment '__DWARF' filesize exceeds vmsize in 'go.so'

Apple's new linker has more checks for unmapped segments. It is
very hard to make it accept a Mach-O shared object with an
additional DWARF segment.

We may want to stop combinding DWARF into the shared object (see
also #62577). For now, disable DWARF by default in c-shared mode
on darwin.

Updates #61229.
For #62598.

Change-Id: I525987b7fe1a4e64571327cb4696f98cc7b419a1
Reviewed-on: https://go-review.googlesource.com/c/go/+/527816
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-09-21 22:21:24 +00:00
Cherry Mui
3ef4f939c3 [release-branch.go1.21] cmd/link: force old Apple linker in plugin mode
There are some bugs in Apple's new linker that causes plugins to
be built incorrectly. And the bugs probably will not be fixed when
Xcode 15 is released (some time soon). Force old Apple linker to
work around.

Updates #61229.
For #62598.

Change-Id: I01ba5caadec6dc14f8c85dd02f78c1ed2e8b7d4d
Reviewed-on: https://go-review.googlesource.com/c/go/+/527815
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2023-09-21 22:21:10 +00:00
Than McIntosh
556e9c36ba [release-branch.go1.21] cmd/link: avoid deadcode of global map vars for programs using plugins
If a program imports the plugin package, the mechanisms in place for
detecting and deleting unused global map variables are no longer safe,
since it's possibly for a given global map var to be unreferenced in
the main program but referenced by a plugin. This patch changes the
linker to test for plugin use and to avoid removing any unused global
map variables if the main program could possibly load up a plugin.

Fixes #62505.
Updates #62430.

Change-Id: Ie00b18b681cb0d259e3c859ac947ade5778cd6c8
(cherry picked from commit 660620dd45)
Reviewed-on: https://go-review.googlesource.com/c/go/+/526575
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-09-20 16:24:52 +00:00
Bryan C. Mills
b64dc5f499 [release-branch.go1.21] cmd/go: in TestScript/gotoolchain_path, remove the user's PATH
This test checks a behavior of GOTOOLCHAIN when an appropriate
toolchain is found in PATH. That requires it to exclude any suitable
toolchain binaries from the user's $PATH, which may otherwise
interfere.

Fixes #62711.
Updates #62709.

Change-Id: Ie9161e52d33a65be0b5265cb49e9f2bc8473e057
Reviewed-on: https://go-review.googlesource.com/c/go/+/529217
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 30886b1b1e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/529435
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-09-19 16:21:36 +00:00
Nayef Ghattas
cd66ca0636 [release-branch.go1.21] runtime/metrics: fix /gc/scan/* metrics
In the existing implementation, all /gc/scan/* metrics are
always equal to 0 due to the dependency on gcStatDep not being
set. This leads to gcStatAggregate always containing zeros, and
always reporting 0 for those metrics.

Also, add a test to ensure that /gc/scan/* metrics are not empty.

For #62477.
Fixes #62478.

Change-Id: I67497347d50ed5c3ce1719a18714c062ec938cab
Reviewed-on: https://go-review.googlesource.com/c/go/+/526116
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Heschi Kreinick <heschi@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-09-11 20:06:42 +00:00
Austin Clements
d7a0626806 [release-branch.go1.21] runtime: ignore SPWrite on innermost traceback frame
Prior to CL 458218, gentraceback ignored the SPWrite function flag on
the innermost frame when doing a precise traceback on the assumption
that precise tracebacks could only be started from the morestack
prologue, and that meant that the innermost function could not have
modified SP yet.

CL 458218 rearranged this logic a bit and unintentionally lost this
particular case. As a result, if traceback starts in an assembly
function that modifies SP (either as a result of stack growth or stack
scanning during a GC preemption), traceback stop at the SPWrite
function and then crash with "traceback did not unwind completely".

Fix this by restoring the earlier special case for when the innermost
frame is SPWrite.

This is a fairly minimal change that should be easy to backport. I
think a more robust change would be to encode this per-PC in the
spdelta table, so it would be clear that we're unwinding from the
morestack prologue and wouldn't rely on a complicated and potentially
fragile set of conditions.

Fixes #62464.

Change-Id: I34f38157631890d33a79d0bd32e32c0fcc2574e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/526100
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Heschi Kreinick <heschi@google.com>
2023-09-11 20:06:38 +00:00
Gopher Robot
2c1e5b05fe [release-branch.go1.21] go1.21.1
Change-Id: I1c208f9cd77af66e98a4d159c2e6c3b6072ea7ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/526039
Run-TryBot: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Joedian Reid <joedian@golang.org>
Auto-Submit: Gopher Robot <gobot@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-09-06 15:31:51 +00:00
Roland Shoemaker
bbd043ff0d [release-branch.go1.21] html/template: properly handle special tags within the script context
The HTML specification has incredibly complex rules for how to handle
"<!--", "<script", and "</script" when they appear within literals in
the script context. Rather than attempting to apply these restrictions
(which require a significantly more complex state machine) we apply
the workaround suggested in section 4.12.1.3 of the HTML specification [1].

More precisely, when "<!--", "<script", and "</script" appear within
literals (strings and regular expressions, ignoring comments since we
already elide their content) we replace the "<" with "\x3C". This avoids
the unintuitive behavior that using these tags within literals can cause,
by simply preventing the rendered content from triggering it. This may
break some correct usages of these tags, but on balance is more likely
to prevent XSS attacks where users are unknowingly either closing or not
closing the script blocks where they think they are.

Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for
reporting this issue.

Fixes #62197
Fixes #62398
Fixes CVE-2023-39319

[1] https://html.spec.whatwg.org/#restrictions-for-contents-of-script-elements

Change-Id: Iab57b0532694827e3eddf57a7497ba1fab1746dc
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1976594
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2014619
Reviewed-on: https://go-review.googlesource.com/c/go/+/526097
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-09-06 14:20:20 +00:00
Roland Shoemaker
b0e1d3ea26 [release-branch.go1.21] html/template: support HTML-like comments in script contexts
Per Appendix B.1.1 of the ECMAScript specification, support HTML-like
comments in script contexts. Also per section 12.5, support hashbang
comments. This brings our parsing in-line with how browsers treat these
comment types.

Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for
reporting this issue.

Fixes #62196
Fixes #62396
Fixes CVE-2023-39318

Change-Id: Id512702c5de3ae46cf648e268cb10e1eb392a181
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1976593
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2014618
Reviewed-on: https://go-review.googlesource.com/c/go/+/526096
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com>
2023-09-06 14:20:08 +00:00
Bryan C. Mills
d25a935574 [release-branch.go1.21] cmd/go: reject toolchain directives containing path separators
If GOTOOLCHAIN="path" or "auto", the go command uses exec.LookPath to
search for it in order to allow toolchains to refer to local-only
toolchain variants (such as toolchains built from enterprise- or
distro-patched source). However, those toolchains should only be
resolved from $PATH, not relative to the working directory of the
command.

Thanks to Juho Nurminen of Mattermost for reporting this issue.

Fixes #62198.
Fixes #62394.
Fixes CVE-2023-39320.

Change-Id: I247c7acea95d737362dd0475e9fc8515430d0fcc
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1996318
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
(cherry picked from commit e41c0a55d45e9a9acbc5d7c1143ea4fff8fb9283)
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2014013
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/526095
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com>
2023-09-06 14:19:53 +00:00
haruyama480
e3ba569c78 [release-branch.go1.21] net/http: revert "support streaming POST content in wasm"
CL 458395 added support for streaming POST content in Wasm.
Unfortunately, this breaks requests to servers that only support HTTP/1.1.
Revert the change until a suitable fallback or opt-in strategy can be decided.

For #61889.
Fixes #62328.

Change-Id: If53a77e1890132063b39abde867d34515d4ac2af
Reviewed-on: https://go-review.googlesource.com/c/go/+/522955
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/524855
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Commit-Queue: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-31 16:40:31 +00:00
Nick Ripley
8dc6ad1c61 [release-branch.go1.21] runtime: restore caller's frame pointer when recovering from panic
When recovering from a panic, restore the caller's frame pointer before
returning control to the caller. Otherwise, if the function proceeds to
run more deferred calls before returning, the deferred functions will
get invalid frame pointers pointing to an address lower in the stack.
This can cause frame pointer unwinding to crash, such as if an execution
trace event is recorded during the deferred call on architectures which
support frame pointer unwinding.

Original CL by Nick Ripley, includes fix from CL 523697, and includes a
test update from CL 524315.

This CL also deviates from the original fix by doing some extra
computation to figure out the fp from the sp, since we don't have the fp
immediately available to us in `recovery` on the Go 1.21 branch, and it
would probably be complicated to plumb that through its caller.

For #61766
Fixes #62046

Change-Id: I5a99ca4f909f6b6e209a330d595d1c99987d4359
Reviewed-on: https://go-review.googlesource.com/c/go/+/523698
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-08-30 22:33:03 +00:00
Michael Anthony Knyszek
06df3292a8 [release-branch.go1.21] runtime: avoid MADV_HUGEPAGE for heap memory
Currently the runtime marks all new memory as MADV_HUGEPAGE on Linux and
manages its hugepage eligibility status. Unfortunately, the default
THP behavior on most Linux distros is that MADV_HUGEPAGE blocks while
the kernel eagerly reclaims and compacts memory to allocate a hugepage.

This direct reclaim and compaction is unbounded, and may result in
significant application thread stalls. In really bad cases, this can
exceed 100s of ms or even seconds.

Really all we want is to undo MADV_NOHUGEPAGE marks and let the default
Linux paging behavior take over, but the only way to unmark a region as
MADV_NOHUGEPAGE is to also mark it MADV_HUGEPAGE.

The overall strategy of trying to keep hugepages for the heap unbroken
however is sound. So instead let's use the new shiny MADV_COLLAPSE if it
exists.

MADV_COLLAPSE makes a best-effort synchronous attempt at collapsing the
physical memory backing a memory region into a hugepage. We'll use
MADV_COLLAPSE where we would've used MADV_HUGEPAGE, and stop using
MADV_NOHUGEPAGE altogether.

Because MADV_COLLAPSE is synchronous, it's also important to not
re-collapse huge pages if the huge pages are likely part of some large
allocation. Although in many cases it's advantageous to back these
allocations with hugepages because they're contiguous, eagerly
collapsing every hugepage means having to page in at least part of the
large allocation.

However, because we won't use MADV_NOHUGEPAGE anymore, we'll no longer
handle the fact that khugepaged might come in and back some memory we
returned to the OS with a hugepage. I've come to the conclusion that
this is basically unavoidable without a new madvise flag and that it's
just not a good default. If this change lands, advice about Linux huge
page settings will be added to the GC guide.

Verified that this change doesn't regress Sweet, at least not on my
machine with:

/sys/kernel/mm/transparent_hugepage/enabled [always or madvise]
/sys/kernel/mm/transparent_hugepage/defrag [madvise]
/sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none [0 or 511]

Unfortunately, this workaround means that we only get forced hugepages
on Linux 6.1+.

For #61718.
Fixes #62329.

Change-Id: I7f4a7ba397847de29f800a99f9cb66cb2720a533
Reviewed-on: https://go-review.googlesource.com/c/go/+/516795
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
(cherry picked from commit 9f9bb26880)
Reviewed-on: https://go-review.googlesource.com/c/go/+/523655
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-30 21:59:35 +00:00
Robert Griesemer
b120517ffd [release-branch.go1.21] go/types, types2: remove order dependency in inference involving channels
In inexact unification, when a named type matches against an inferred
unnamed type, we change the previously inferred type to the named type.
This preserves the type name and assignability.

We have to do the same thing when encountering a directional channel:
a bidirectional channel can always be assigned to a directional channel
but not the other way around. Thus, if we see a directional channel, we
must choose the directional channel.

This CL extends the previously existing logic for named types to
directional channels and also makes the code conditional on inexact
unification. The latter is an optimization - if unification is exact,
type differences don't exist and updating an already inferred type has
no effect.

Fixes #62205.

Change-Id: I807e3b9f9ab363f9ed848bdb18b2577b1d680ea7
Reviewed-on: https://go-review.googlesource.com/c/go/+/524256
Run-TryBot: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
2023-08-30 21:35:21 +00:00
Bryan C. Mills
0a9582163c [release-branch.go1.21] cmd/go: retry ETXTBSY errors when running test binaries
An ETXTBSY error when starting a test binary is almost certainly
caused by the race reported in #22315. That race will resolve quickly
on its own, so we should just retry the command instead of reporting a
spurious failure.

Fixes #62222.
Updates #62221.

Change-Id: I408f3eaa7ab5d7efbc7a2b1c8bea3dbc459fc794
Reviewed-on: https://go-review.googlesource.com/c/go/+/522015
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
(cherry picked from commit 4dc2564933)
Reviewed-on: https://go-review.googlesource.com/c/go/+/522176
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-30 21:32:23 +00:00
Damien Neil
91a4e74b98 [release-branch.go1.21] crypto/tls: QUIC: fix panics when processing post-handshake messages
The check for fragmentary post-handshake messages in QUICConn.HandleData
was reversed, resulting in a potential panic when HandleData receives
a partial message.

In addition, HandleData wasn't checking the size of buffered
post-handshake messages. Produce an error when a post-handshake
message is larger than maxHandshake.

TestQUICConnectionState was using an onHandleCryptoData hook
in runTestQUICConnection that was never being called.
(I think it was inadvertently removed at some point while
the CL was in review.) Fix this test while making the hook
more general.

For #62266
Fixes #62290

Change-Id: I210b70634e50beb456ab3977eb11272b8724c241
Reviewed-on: https://go-review.googlesource.com/c/go/+/522595
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Marten Seemann <martenseemann@gmail.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
(cherry picked from commit e92c0f846c)
Reviewed-on: https://go-review.googlesource.com/c/go/+/523039
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-08-30 21:27:45 +00:00
Bryan C. Mills
6385a6fb18 [release-branch.go1.21] cmd/go: find GOROOT using os.Executable when installed to GOROOT/bin/GOOS_GOARCH
When running make.bash in a cross-compiled configuration
(for example, GOARCH different from GOHOSTARCH), cmd/go
is installed to GOROOT/bin/GOOS_GOARCH instead of GOROOT/bin.

That means that we need to look for GOROOT in both ../.. and ../../..,
not just the former.

Fixes #62144.
Updates #62119.
Updates #18678.

Change-Id: I283c6a10c46df573ff44da826f870417359226a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/521015
Reviewed-by: Michael Matloob <matloob@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit 9e9556d328)
Reviewed-on: https://go-review.googlesource.com/c/go/+/521695
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-30 21:25:08 +00:00
xuri
2d07bb86f0 [release-branch.go1.21] encoding/xml: overriding by empty namespace when no new name declaration
The unmarshal and marshal XML text should be consistent if not modified deserialize variable.

For #61881
Fixes #62051

Change-Id: I475f7b05211b618685597d3ff20b97e3bbeaf8f8
GitHub-Last-Rev: 6831c770c3
GitHub-Pull-Request: golang/go#58401
Reviewed-on: https://go-review.googlesource.com/c/go/+/522316
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-08-30 20:17:11 +00:00
Andy Pan
745b81b6e6 [release-branch.go1.21] encoding/gob: prevent panic from index out of range in Decoder.typeString
I believe this bug is introduced by CL 460543 which optimizes the allocations
by changing the type of `idToType` from map to slice, but didn't update the
access code in `Decoder.typeString` that is safe for map but not for slice.

For #62117
Fixes #62154

Change-Id: I0f2e4cc2f34c54dada1f83458ba512a6fde6dcbe
Reviewed-on: https://go-review.googlesource.com/c/go/+/520757
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Andy Pan <panjf2000@gmail.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
(cherry picked from commit ba626ac327)
Reviewed-on: https://go-review.googlesource.com/c/go/+/521156
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-08-25 20:52:06 +00:00
Keith Randall
13339c75b8 [release-branch.go1.21] runtime: fix maps.Clone bug when cloning a map mid-grow
Fixes #62204

Change-Id: I0459d3f481b0cd20102f6d9fd3ea84335a7739a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/522317
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit b303fb4855)
Reviewed-on: https://go-review.googlesource.com/c/go/+/522936
Reviewed-by: Carlos Amedee <carlos@golang.org>
2023-08-25 16:37:04 +00:00
Bryan C. Mills
2977709875 [release-branch.go1.21] context: fix synchronization in ExampleAfterFunc_cond
Condition variables are subtle and error-prone, and this example
demonstrates exactly the sorts of problems that they introduce.
Unfortunately, we're stuck with them for the foreseeable future.

As previously implemented, this example was racy: since the callback
passed to context.AfterFunc did not lock the mutex before calling
Broadcast, it was possible for the Broadcast to occur before the
goroutine was parked in the call to Wait, causing in a missed wakeup
resulting in deadlock.

The example also had a more insidious problem: it was not safe for
multiple goroutines to call waitOnCond concurrently, but the whole
point of using a sync.Cond is generally to synchronize concurrent
goroutines. waitOnCond must use Broadcast to ensure that it wakes up
the target goroutine, but the use of Broadcast in this way would
produce spurious wakeups for all of the other goroutines waiting on
the same condition variable. Since waitOnCond did not recheck the
condition in a loop, those spurious wakeups would cause waitOnCond
to spuriously return even if its own ctx was not yet done.

Fixing the aforementioned bugs exposes a final problem, inherent to
the use of condition variables in this way. This one is a performance
problem: for N concurrent calls to waitOnCond, the resulting CPU cost
is at least O(N²). This problem cannot be addressed without either
reintroducing one of the above bugs or abandoning sync.Cond in the
example entirely. Given that this example was already published in Go
1.21, I worry that Go users may think that it is appropriate to use a
sync.Cond in conjunction with context.AfterFunc, so I have chosen to
retain the Cond-based example and document its pitfalls instead of
removing or replacing it entirely.

I described this class of bugs and performance issues — and suggested
some channel-based alternatives — in my GopherCon 2018 talk,
“Rethinking Classical Concurrency Patterns”. The section on condition
variables starts on slide 37. (https://youtu.be/5zXAHh5tJqQ?t=679)

Fixes #62189.
Updates #62180.
For #20491.

Change-Id: If987cd9d112997c56171a7ef4fccadb360bb79bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/521596
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
(cherry picked from commit 1081f8c058)
Reviewed-on: https://go-review.googlesource.com/c/go/+/521598
2023-08-24 21:13:14 +00:00
Robert Griesemer
2d4746f37b [release-branch.go1.21] go/types, types2: disable interface inference for versions before Go 1.21
Change the internal constant enableInterfaceInference to a unifier
field that can be controlled dynamically and set it for Go 1.21
or later.

This restores Go 1.20 unification behavior for interfaces.

Fixes #61930.

Change-Id: Iefd6c0899811f8208a8be9cef2650a07787ae177
Reviewed-on: https://go-review.googlesource.com/c/go/+/519855
Reviewed-by: Robert Findley <rfindley@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Robert Griesemer <gri@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/520601
2023-08-24 21:12:30 +00:00
David Chase
2b8026f025 [release-branch.go1.21] cmd/compile: in expandCalls, move all arg marshalling into call block
For aggregate-typed arguments passed to a call, expandCalls
decomposed them into parts in the same block where the value
was created.  This is not necessarily the call block, and in
the case where stores are involved, can change the memory
leaving that block, and getting that right is problematic.

Instead, do all the expanding in the same block as the call,
which avoids the problems of (1) not being able to reorder
loads/stores across a block boundary to conform to memory
order and (2) (incorrectly, not) exposing the new memory to
consumers in other blocks.  Putting it all in the same block
as the call allows reordering, and the call creates its own
new memory (which is already dealt with correctly).

Fixes #62057.
Updates #61992.

Change-Id: Icc7918f0d2dd3c480cc7f496cdcd78edeca7f297
Reviewed-on: https://go-review.googlesource.com/c/go/+/519276
Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit e72ecc6a6b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/520058
2023-08-24 21:10:25 +00:00
Bryan Mills
7c97cc7d97 [release-branch.go1.21] Revert "os: use handle based APIs to read directories on windows"
This reverts CL 452995.

Reason for revert: caused os.File.ReadDir to fail on
filesystems that do not support FILE_ID_BOTH_DIR_INFO.

This is an alternative to a fix-forward change in CL 518196.
Since the original change was mostly a performance improvement,
reverting to the previous implementation seems less risky than
backporting a larger fix.

Fixes #61910
Fixes #61964

Change-Id: I60f1602b9eb6ea353e7eb23429f19f1ffa16b394
Reviewed-on: https://go-review.googlesource.com/c/go/+/520156
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
2023-08-23 19:57:00 +00:00
Keith Randall
cb6ea94996 [release-branch.go1.21] Revert "cmd/compile: omit redundant sign/unsign extension on arm64"
This reverts CL 427454.

Reason for revert: causes incorrect generated code in some rare cases

We'll fix-forward at tip, so the revert just needs to be done for 1.21.

Fixes #62143

Change-Id: Id242230481ff4d4ba5f58236c6d8237729fc3b80
Reviewed-on: https://go-review.googlesource.com/c/go/+/520976
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Ruinan Sun <Ruinan.Sun@arm.com>
Reviewed-by: Keith Randall <khr@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-08-23 17:57:50 +00:00
Damien Neil
45b98bfb79 [release-branch.go1.21] path/filepath: don't drop .. elements when cleaning invalid Windows paths
Fix a bug where Clean could improperly drop .. elements from a
path on Windows, when the path contains elements containing a ':'.

For example, Clean("a/../b:/../../c") now correctly returns "..\c"
rather than "c".

For #61866.
Fixes #61868.

Change-Id: I97b0238953c183b2ce19ca89c14f26700008ea72
Reviewed-on: https://go-review.googlesource.com/c/go/+/517216
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
(cherry picked from commit 6e43407931)
Reviewed-on: https://go-review.googlesource.com/c/go/+/519655
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
2023-08-23 17:53:26 +00:00
Cherry Mui
bac083a584 [release-branch.go1.21] cmd/link: don't mangle string symbol names
String symbol names could contain weird characters as we put the
string literal into the symbol name. So it may appear to need
mangling. However, as string symbols are grouped into a single
"go:string.*" symbol, the individual symbol names actually don't
matter. So don't mangle them.

Also make the mangling code more defensive in case of weird
symbol names.

Updates #62098.
Fixes #62140.

Change-Id: I533012567a9fffab69debda934f426421c7abb04
Reviewed-on: https://go-review.googlesource.com/c/go/+/520856
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit b65e34f038)
Reviewed-on: https://go-review.googlesource.com/c/go/+/520857
2023-08-23 17:35:53 +00:00
Chris O'Hara
70aa116c4a [release-branch.go1.21] runtime/internal/wasitest: skip racy TCP echo test
The wasip1 TCP echo test introduced in CL 493358 has a race
condition with port selection. The test runner probes for a free
port and then asks the WASM runtime to listen on the port, which
may be taken by another process in the interim.

Due to limitations with WASI preview 1, the guest is unable to
query the port it's listening on. The test cannot ask the WASM
runtime to listen on port 0 (choose a free port) since there's
currently no way for the test to query the selected port and
connect to it.

Given the race condition is unavoidable, this test is now disabled
by default and requires opt-in via an environment variable.

This commit also eliminates the hard-coded connection timeout.

Updates #61820.
Fixes #61821.

Change-Id: I375145c1a1d03ad45c44f528da3347397e6dcb01
Reviewed-on: https://go-review.googlesource.com/c/go/+/519895
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
(cherry picked from commit 795e779610)
Reviewed-on: https://go-review.googlesource.com/c/go/+/520955
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-18 18:00:35 +00:00
Mauri de Souza Meneguzzo
31c5a236bc [release-branch.go1.21] runtime: mark traceEnabled and traceShuttingDown as no:split
This fixes a regression from CL 494181.
The traceEnabled function splits the stack and is being
called by reentersyscall that shouldn't call anything
that splits the stack. Same with traceShuttingDown.

For #61975
Fixes #61987

Change-Id: I5eca0ba74cfa6acb0259e8400b03c2093cd59dd1
GitHub-Last-Rev: 9e55ae9d7c
GitHub-Pull-Request: golang/go#61981
Reviewed-on: https://go-review.googlesource.com/c/go/+/519055
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@google.com>
(cherry picked from commit aa5d483f25)
Reviewed-on: https://go-review.googlesource.com/c/go/+/519495
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
2023-08-17 21:19:23 +00:00
Keith Randall
25ec110e51 [release-branch.go1.21] cmd/compile: ensure empty blocks in write barriers are marked unpreemptible
Fixes #61958

Change-Id: I242ab77ad2f1ea1dad2d14ef756fa92f9378429f
Reviewed-on: https://go-review.googlesource.com/c/go/+/518755
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2023-08-17 21:19:19 +00:00
David Chase
6634ce2f41 [release-branch.go1.21] runtime: profiling on Darwin cannot use blocking reads
On Darwin (and assume also on iOS but not sure), notetsleepg
cannot be called in a signal-handling context.  Avoid this
by disabling block reads on Darwin.

An alternate approach was to add "sigNote" with a pipe-based
implementation on Darwin, but that ultimately would have required
at least one more linkname between runtime and syscall to avoid
racing with fork and opening the pipe, so, not.

Fixes #62019.
Updates #61768.

Change-Id: I0e8dd4abf9a606a3ff73fc37c3bd75f55924e07e
Reviewed-on: https://go-review.googlesource.com/c/go/+/518836
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
(cherry picked from commit c6ee8e31e3)
Reviewed-on: https://go-review.googlesource.com/c/go/+/519375
Reviewed-by: Austin Clements <austin@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-17 21:10:08 +00:00
Cuong Manh Le
25c6dce188 [release-branch.go1.21] cmd/compile: make backingArrayPtrLen to return typecheck-ed nodes
Fixes #61909

Change-Id: Ief8e3a6c42c0644c9f71ebef5f28a294cd7c153f
Reviewed-on: https://go-review.googlesource.com/c/go/+/517936
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/518115
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-17 21:07:51 +00:00
Robert Findley
4e34f2e81d [release-branch.go1.21] go/types, types2: don't panic during interface completion
It should be possible for the importer to construct an invalid
interface, as would have been produced by type checking.

Updates #61737
Fixes #61743

Change-Id: I72e063f4f1a6205d273a623acce2ec08c34c3cc2
Reviewed-on: https://go-review.googlesource.com/c/go/+/515555
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Findley <rfindley@google.com>
Reviewed-by: Olif Oftimis <oftimisolif@gmail.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit d2ee7821d3)
Reviewed-on: https://go-review.googlesource.com/c/go/+/515636
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-17 21:05:37 +00:00
Robert Griesemer
d91843ff67 [release-branch.go1.21] go/types, types2: use correct parameter list when checking argument passing
The existing code was simply wrong: we cannot ever use the result
signature parameter list (rsig.params) if sigParams was adjusted
for variadic functions. If it was adjusted, we always must either
use sigParams or its separately instantiated version.

In the condition "n > 0 && adjusted", the "n > 0" should have
been in either of the respective "if statement" branches.

Simplified the code by merging with the result signature parameter
update.

Fixes #61932.

Change-Id: I5d39bc8bbc4dd85c7c985055d29532b4b176955e
Reviewed-on: https://go-review.googlesource.com/c/go/+/519456
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/519417
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-17 21:05:04 +00:00
Robert Griesemer
7437db1085 [release-branch.go1.21] go/types, types2: use exact unification when comparing interface methods
Irrespective of whether unification is exact or inexact, method
signatures of interfaces must always match exactly: a type never
satisfies/implements an interface if relevant method signatures
are different (i.e., not identical, possibly after substitution).

Fixes #61959.

Change-Id: I20c0aa28ac86e2edec615b40f2269938e4a96938
Reviewed-on: https://go-review.googlesource.com/c/go/+/519435
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/519416
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-17 21:05:01 +00:00
Russ Cox
ed527ecfb2 [release-branch.go1.21] cmd/api: rename api.go to main_test.go
This makes cmd/api no longer an importable package.
In CL 453258 I forgot that there was no direct prohibition
on importing packages from cmd - we just rely on the
fact that cmd/* is all package main and everything else
is cmd/internal.

Fixes #62069.
Fixes #62071.

Change-Id: Ifed738d333b40663f85eca8f83025fcea5df89a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/520038
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/520061
2023-08-17 01:32:58 +00:00
Roland Shoemaker
b78e8cc145 [release-branch.go1.21] crypto/tls: add GODEBUG to control max RSA key size
Add a new GODEBUG setting, tlsmaxrsasize, which allows controlling the
maximum RSA key size we will accept during TLS handshakes.

Fixes #61967

Change-Id: I52f060be132014d219f4cd438f59990011a35c96
Reviewed-on: https://go-review.googlesource.com/c/go/+/517495
Auto-Submit: Roland Shoemaker <roland@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/518535
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-16 17:16:32 +00:00
Michael Matloob
3475e6af4c [release-branch.go1.21] cmd/go: fix missing case checking for empty slice
When we were comparing the first element of import stacks when sorting
depserrors we checked if the first stack was non empty, but not the
second one. Do the check for both stacks.

Fixes #61818
Updates #61816
For #59905

Change-Id: Id5c11c2b1104eec93196a08c53372ee2ba97c701
Reviewed-on: https://go-review.googlesource.com/c/go/+/516739
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
(cherry picked from commit 58447d757c)
Reviewed-on: https://go-review.googlesource.com/c/go/+/519658
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-15 19:04:26 +00:00
Damien Neil
179821c9e1 [release-branch.go1.21] net/http: permit requests with invalid Host headers
Historically, the Transport has silently truncated invalid
Host headers at the first '/' or ' ' character. CL 506996 changed
this behavior to reject invalid Host headers entirely.
Unfortunately, Docker appears to rely on the previous behavior.

When sending a HTTP/1 request with an invalid Host, send an empty
Host header. This is safer than truncation: If you care about the
Host, then you should get the one you set; if you don't care,
then an empty Host should be fine.

Continue to fully validate Host headers sent to a proxy,
since proxies generally can't productively forward requests
without a Host.

For #60374
Fixes #61431
Fixes #61904

Change-Id: If170c7dd860aa20eb58fe32990fc93af832742b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/511155
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
(cherry picked from commit b9153f6ef3)
Reviewed-on: https://go-review.googlesource.com/c/go/+/518856
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Roland Shoemaker <roland@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2023-08-14 21:55:15 +00:00
Russ Cox
9398951479 [release-branch.go1.21] cmd/distpack: include directory entries in tar files
Various tools expect tar files to contain entries for directories.
I dropped them when writing cmd/distpack because they're not
strictly necessary and omitting them saves space, but it also
turns out to break some things, so add them back.

We will backport this to release-branch.go1.21 so that Go 1.21.1
will include the directory entries. We can't do anything about
Go 1.21.0 retroactively.

% tar tzvf go1.22rsc1.src.tar.gz | sed 10q
drwxr-xr-x  0 0      0           0 Aug 10 10:07 go/
-rw-r--r--  0 0      0        1337 Aug 10 10:07 go/CONTRIBUTING.md
-rw-r--r--  0 0      0        1479 Aug 10 10:07 go/LICENSE
-rw-r--r--  0 0      0        1303 Aug 10 10:07 go/PATENTS
-rw-r--r--  0 0      0        1455 Aug 10 10:07 go/README.md
-rw-r--r--  0 0      0         419 Aug 10 10:07 go/SECURITY.md
-rw-r--r--  0 0      0          42 Aug 10 10:07 go/VERSION
drwxr-xr-x  0 0      0           0 Aug 10 10:07 go/api/
-rw-r--r--  0 0      0        1142 Aug 10 10:07 go/api/README
-rw-r--r--  0 0      0       35424 Aug 10 10:07 go/api/except.txt
% tar tzvf go1.22rsc1.darwin-amd64.tar.gz | sed 10q
drwxr-xr-x  0 0      0           0 Aug 10 10:07 go/
-rw-r--r--  0 0      0        1337 Aug 10 10:07 go/CONTRIBUTING.md
-rw-r--r--  0 0      0        1479 Aug 10 10:07 go/LICENSE
-rw-r--r--  0 0      0        1303 Aug 10 10:07 go/PATENTS
-rw-r--r--  0 0      0        1455 Aug 10 10:07 go/README.md
-rw-r--r--  0 0      0         419 Aug 10 10:07 go/SECURITY.md
-rw-r--r--  0 0      0          42 Aug 10 10:07 go/VERSION
drwxr-xr-x  0 0      0           0 Aug 10 10:07 go/api/
-rw-r--r--  0 0      0        1142 Aug 10 10:07 go/api/README
-rw-r--r--  0 0      0       35424 Aug 10 10:07 go/api/except.txt
%

Fixes #61862.
Fixes #61927.

Change-Id: Iecd9ba893015295e88715b031b79a104236b9ced
Reviewed-on: https://go-review.googlesource.com/c/go/+/518335
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/518835
Auto-Submit: Russ Cox <rsc@golang.org>
2023-08-11 17:56:21 +00:00
Bryan C. Mills
75d8be5fb4 [release-branch.go1.21] cmd/go/internal/web: release the net token when an HTTP request fails due to CheckRedirect
Updates #61877.
Fixes #61905.

Change-Id: I38c63565aaf9dc9b0c8085974521daccfbcbc790
Reviewed-on: https://go-review.googlesource.com/c/go/+/518015
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit 8cb5c55118)
Reviewed-on: https://go-review.googlesource.com/c/go/+/518395
2023-08-11 14:53:49 +00:00
Cuong Manh Le
1755d14559 [release-branch.go1.21] cmd/compile: fix missing init nodes for len(string([]byte)) optimization
CL 497276 added optimization for len(string([]byte)) by avoiding call to
slicebytetostring. However, the bytes to string expression may contain
init nodes, which need to be preserved. Otherwise, it would make the
liveness analysis confusing about the lifetime of temporary variables
created by init nodes.

Fixes #61781

Change-Id: I6d1280a7d61bcc75f11132af41bda086f084ab54
Reviewed-on: https://go-review.googlesource.com/c/go/+/516375
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/516535
2023-08-11 14:53:01 +00:00
Gopher Robot
c19c4c566c [release-branch.go1.21] go1.21.0
Change-Id: Iffac2ce43cd3a48ba420594adc3eab5aa9bcf113
Reviewed-on: https://go-review.googlesource.com/c/go/+/517055
Auto-Submit: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2023-08-08 15:00:52 +00:00
Damien Neil
e973d24261 [release-branch.go1.21] Revert "net/http: use Copy in ServeContent if CopyN not needed"
This reverts CL 446276.

Reason for revert: Causing surprising performance regression.

Fixes #61530

Change-Id: Ic970f2e05d875b606ce274ea621f7e4c8c337481
Reviewed-on: https://go-review.googlesource.com/c/go/+/512615
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit df0a129789)
Reviewed-on: https://go-review.googlesource.com/c/go/+/515795
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-08-04 16:24:02 +00:00
Robert Griesemer
2e6276df34 [release-branch.go1.21] spec: remove unnecessary sentence
Change-Id: I06345199ff16c80be83c345d734caef1714ec089
Reviewed-on: https://go-review.googlesource.com/c/go/+/515338
TryBot-Bypass: Robert Griesemer <gri@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/515715
2023-08-03 23:35:21 +00:00
Johan Brandhorst-Satzkorn
aeef93cd64 [release-branch.go1.21] runtime/internal: switch GOWASIRUNTIME default
CL 513235 switched the default wasip1 runtime in the misc/wasm
executable script, but it missed this use of the GOWASIRUNTIME
environment variable. Update this instance to make the default runtime
choice consistent.

Change-Id: Iff7f96231422747a38d65d13a940f6e9d04d835d
Reviewed-on: https://go-review.googlesource.com/c/go/+/515116
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
2023-08-03 22:16:40 +00:00
Damien Neil
35de5f2b0e [release-branch.go1.21] crypto/tls: change SendSessionTicket to take an options struct
To allow for future evolution of the API, make
QUICConn.SendSessionTicket take a QUICSessionTicketOptions
rather than a single bool.

For #60107

Change-Id: I798fd0feec5c7581e3c3574e2de99611c81df47f
Reviewed-on: https://go-review.googlesource.com/c/go/+/514997
Reviewed-by: Roland Shoemaker <roland@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Marten Seemann <martenseemann@gmail.com>
(cherry picked from commit a915b999c9)
Reviewed-on: https://go-review.googlesource.com/c/go/+/515335
Auto-Submit: Damien Neil <dneil@google.com>
2023-08-02 17:43:27 +00:00
Cherry Mui
a3b092d65e [release-branch.go1.21] cmd/link: use symbol-targeted relocation for initializers on Mach-O
Apple's new linker, ld-prime from Xcode 15 beta, when handling
initializers in __mod_init_func, drops the offset in the data,
resolving the relocation to the beginning of the section. The
latest version of ld-prime rejects non-zero addend. We need to use
symbol-targeted "external" relocations, so that it doesn't need
an addend and can be resolved correctly. This also works fine with
ld64.

Fixes #60694.
For #61229.

Change-Id: Ida2be6aa4c91bfcd142b755e2ec63aabfbbd77a6
Reviewed-on: https://go-review.googlesource.com/c/go/+/502616
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit bad9ca8a61)
Reviewed-on: https://go-review.googlesource.com/c/go/+/514535
2023-08-02 17:38:41 +00:00
Mauri de Souza Meneguzzo
07c72a0915 [release-branch.go1.21] cmd/go: missing name in failed command error
Fixed the error reporting for an unknown command to
preserve the name when displaying the error message.

Fixes #61604

Change-Id: I13defb84e61265ab48ab514e9d4f1626a4a3f758
GitHub-Last-Rev: 5d2889c60c
GitHub-Pull-Request: golang/go#61607
Reviewed-on: https://go-review.googlesource.com/c/go/+/513555
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/515278
Auto-Submit: David Chase <drchase@google.com>
2023-08-02 15:53:43 +00:00
Gopher Robot
041dd5ce05 [release-branch.go1.21] go1.21rc4
Change-Id: I4a166f3d04747703f2c0b3c3f245d9cde44f3068
Reviewed-on: https://go-review.googlesource.com/c/go/+/514777
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Gopher Robot <gobot@golang.org>
Auto-Submit: Gopher Robot <gobot@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2023-08-02 13:51:49 +00:00
Roland Shoemaker
a51957fb0b [release-branch.go1.21] crypto/tls: restrict RSA keys in certificates to <= 8192 bits
Extremely large RSA keys in certificate chains can cause a client/server
to expend significant CPU time verifying signatures. Limit this by
restricting the size of RSA keys transmitted during handshakes to <=
8192 bits.

Based on a survey of publicly trusted RSA keys, there are currently only
three certificates in circulation with keys larger than this, and all
three appear to be test certificates that are not actively deployed. It
is possible there are larger keys in use in private PKIs, but we target
the web PKI, so causing breakage here in the interests of increasing the
default safety of users of crypto/tls seems reasonable.

Thanks to Mateusz Poliwczak for reporting this issue.

Updates #61460
Fixes CVE-2023-29409

Change-Id: Ie35038515a649199a36a12fc2c5df3af855dca6c
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1912161
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Run-TryBot: Roland Shoemaker <bracewell@google.com>
(cherry picked from commit d865c715d92887361e4bd5596e19e513f27781b7)
Reviewed-on: https://go-review.googlesource.com/c/go/+/515056
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-08-01 21:58:27 +00:00
Russ Cox
363f2594aa [release-branch.go1.21] cmd/go: make go list -m -u all not complain about missing checksums
This is a band-aid of a fix for Go 1.21, to create space to work on
a real fix for Go 1.22, if in fact the real fix is different. It simply
disables the go.sum update check during go list -m -u.
I don't have a self-contained test for the breakage. See #61605.
All existing tests continue to pass.

For #61605.
After merging into the Go 1.21 branch we can move #61605 to the Go 1.22 milestone.

Change-Id: Ib155710092003f08d2a6ce0aefa8e0270cad5a5c
Reviewed-on: https://go-review.googlesource.com/c/go/+/514899
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-08-01 18:40:48 +00:00
Johan Brandhorst-Satzkorn
9b53b9b585 [release-branch.go1.21] misc/wasm: switch default WASI runtime
The default WASI runtime was originally set to Wazero, because it was
the first runtime used to test the Go implementation and because we
could easily find and fix issues in our implementation and theirs.

In CL 498675 we switched the default wasip1 runner to Wasmtime as it
runs faster and is a more established and mature runtime. We should
switch the default runtime to Wasmtime to consistently promote
Wasmtime as the primary tested and approved runtime.

Change-Id: Ic6c064142321af90f015e02b7fe0e71444d8842c
Reviewed-on: https://go-review.googlesource.com/c/go/+/513235
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Eli Bendersky <eliben@google.com>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
(cherry picked from commit 4918490962)
Reviewed-on: https://go-review.googlesource.com/c/go/+/514155
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2023-07-31 18:16:13 +00:00
Robert Griesemer
4a14d9c9af [release-branch.go1.21] spec: update spec to version at tip
This updates the spec by copying over several recent CLs
describing the new type inference mechanisms.

Fixes #61659.

Change-Id: I750c901e73e0404f782a3632f5cd936e3775ae13
Reviewed-on: https://go-review.googlesource.com/c/go/+/514435
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Bypass: Robert Griesemer <gri@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-07-31 17:13:23 +00:00
Cherry Mui
9786164333 [release-branch.go1.21 cmd/cgo/internal/test: don't pass -lm on darwin
On darwin, -lm is not necessary as the math functions are included
in libSystem. Passing -lm multiple times results in linker
warnings. Don't pass it on darwin.

For #61229.

Change-Id: I72d8dab1f0eead68cbeb176ac97b8ed1a0cfddab
Reviewed-on: https://go-review.googlesource.com/c/go/+/508697
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
(cherry picked from commit 333c80694c)
Reviewed-on: https://go-review.googlesource.com/c/go/+/513758
2023-07-31 16:12:29 +00:00
Dmitri Shuralyov
6df6e61cbb [release-branch.go1.21] cmd/dist: handle -json flag in runPending (minimal)
The -json flag is new to Go 1.21, but missed skips in runPending.
This CL adds minimal code to fix that. CL 512115 cleans up a bit.

For #37486.
Fixes #61557.

Change-Id: I53e426c9a5585b2703f0ff6661a0470e1993f960
Reviewed-on: https://go-review.googlesource.com/c/go/+/513761
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2023-07-28 17:57:22 +00:00
Ian Lance Taylor
b25266c58d [release-branch.go1.21] maps: remove Keys and Values
Preserve the names in case we want them to return an iterator.
Keep the efficient runtime implementations for now,
as we will probably want them under some name, perhaps KeysSlice
and ValuesSlice.

Fixes #61538

Change-Id: I6b03010bf071fb4531cb2f967dad46425962fcb8
Reviewed-on: https://go-review.googlesource.com/c/go/+/513476
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/513715
Run-TryBot: Russ Cox <rsc@golang.org>
2023-07-27 22:20:27 +00:00
Cherry Mui
1ea8d38517 [release-branch.go1.21] cmd/link: don't generate DYSYMTAB when external linking on Mach-O
When external linking, the external linker will generate it.

Updates #60694.
For #61229.

Change-Id: I086a7628dd9baa84b46315641746fc3640473f2b
Reviewed-on: https://go-review.googlesource.com/c/go/+/502617
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit f55e7e104b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/513757
2023-07-27 20:07:01 +00:00
zikaeroh
b2ffc23a82 [release-branch.go1.21] log/slog: fix comment above log levels
This extra newline causes pkg.go.dev and gopls to only show the bottom
half of this comment; I'm pretty sure this entire thing is meant to be
in the docs.

Change-Id: I5bbf081fb2072d9d773d5a995bc3693dc44f65ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/511855
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/512199
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2023-07-24 17:02:06 +00:00
Rob Findley
8472fcb62d [release-branch.go1.21] go/types, types2: update documentation for GoVersion
Update the documentation for Config.GoVersion to reflect the changes
made for #61175.

Change-Id: I9f3fbcf8ee88e52d6a5e7cf80dad3d2fb5313893
Reviewed-on: https://go-review.googlesource.com/c/go/+/511096
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
(cherry picked from commit 12f3d6858e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/511699
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
2023-07-24 02:02:15 +00:00
Dmitri Shuralyov
b36e5555dd [release-branch.go1.21] cmd/dist: apply timeout scale even if timeout isn't overridden
The timeout field is documented as being available so that it's possible
to override timeout by setting a non-zero value. If it's left at zero,
we don't need to override the default go test timeout, but we still need
to apply the timeout scale whenever it's something other than 1.

Fixes #61468.

Change-Id: I63634e9b3ef8c4ec7f334b5a6b4bf3cad121355c
Reviewed-on: https://go-review.googlesource.com/c/go/+/511976
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-07-21 14:59:06 +00:00
Meng Zhuo
ed977e2f47 [release-branch.go1.21] cmd/asm, cmd/internal/obj: generate proper atomic ops for riscv64
Go's memory model closely follows the approach C++ concurrency memory
model (https://go.dev/ref/mem) and Go atomic "has the same semantics as C++'s
sequentially consistent atomics".

Meanwhile according to RISCV manual A.6 "Mappings from C/C++ primitives to RISC-V primitives".
C/C++ atomic operations (memory_order_acq_rel) should be map to "amo<op>.{w|d}.aqrl"
LR/SC (memory_order_acq_rel) should map to "lr.{w|d}.aq; <op>; sc.{w|d}.rl"

goos: linux
goarch: riscv64
pkg: runtime/internal/atomic
                │ atomic.old.bench │          atomic.new.bench           │
                │      sec/op      │   sec/op     vs base                │
AtomicLoad64-4         4.216n ± 1%   4.202n ± 0%        ~ (p=0.127 n=10)
AtomicStore64-4        5.040n ± 0%   6.718n ± 0%  +33.30% (p=0.000 n=10)
AtomicLoad-4           4.217n ± 0%   4.213n ± 0%        ~ (p=0.145 n=10)
AtomicStore-4          5.040n ± 0%   6.718n ± 0%  +33.30% (p=0.000 n=10)
And8-4                 9.237n ± 0%   9.240n ± 0%        ~ (p=0.582 n=10)
And-4                  5.878n ± 0%   6.719n ± 0%  +14.31% (p=0.000 n=10)
And8Parallel-4         28.44n ± 0%   28.46n ± 0%   +0.07% (p=0.000 n=10)
AndParallel-4          28.40n ± 0%   28.43n ± 0%   +0.11% (p=0.000 n=10)
Or8-4                  8.399n ± 0%   8.398n ± 0%        ~ (p=0.357 n=10)
Or-4                   5.879n ± 0%   6.718n ± 0%  +14.27% (p=0.000 n=10)
Or8Parallel-4          28.43n ± 0%   28.45n ± 0%   +0.09% (p=0.000 n=10)
OrParallel-4           28.40n ± 0%   28.43n ± 0%   +0.11% (p=0.000 n=10)
Xadd-4                 30.05n ± 0%   30.10n ± 0%   +0.18% (p=0.000 n=10)
Xadd64-4               30.05n ± 0%   30.09n ± 0%   +0.12% (p=0.000 n=10)
Cas-4                  60.48n ± 0%   61.13n ± 0%   +1.08% (p=0.000 n=10)
Cas64-4                62.28n ± 0%   62.34n ± 0%        ~ (p=0.810 n=10)
Xchg-4                 30.05n ± 0%   30.09n ± 0%   +0.15% (p=0.000 n=10)
Xchg64-4               30.05n ± 0%   30.09n ± 0%   +0.13% (p=0.000 n=10)
geomean                15.42n        16.17n        +4.89%

Fixes #61295

Change-Id: I97b5325db50467eeec36fb079bded7b09a32330f
Reviewed-on: https://go-review.googlesource.com/c/go/+/508715
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
Run-TryBot: M Zhuo <mzh@golangcn.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit 890b96f7ab)
Reviewed-on: https://go-review.googlesource.com/c/go/+/511856
Auto-Submit: M Zhuo <mzh@golangcn.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: M Zhuo <mzh@golangcn.org>
2023-07-21 14:45:34 +00:00
Robert Griesemer
2fabb143d7 [release-branch.go1.21] go/types, types2: a min/max value argument must not be untyped
Fizes #61492.

Change-Id: I5770e238e44b724816894d914b3ea5dc78bc3ced
Reviewed-on: https://go-review.googlesource.com/c/go/+/511835
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/511857
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-07-21 03:06:31 +00:00
Cherry Mui
c9f01f0ec7 [release-branch.go1.21 cmd/go: attach PGO profile for test dependencies
When running "go test" including a main package which has a PGO
profile, we currently build the package being tested and its
dependencies with PGO, but we failed to attach the profile to
test-only dependencies. If a package is (transitively) imported
by both the package being tested and the test, the PGO version
and the non-PGO version of the package are both linked into the
binary, causing link-time error.

This CL fixes this by attaching the PGO profile to dependencies of
the test.

Fixes #61376.

Change-Id: I2559db9843c4cdab596b31e2025d8475ffbf58ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/510835
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit af0609b080)
Reviewed-on: https://go-review.googlesource.com/c/go/+/511698
2023-07-20 20:44:54 +00:00
Dmitri Shuralyov
252f20b2c1 [release-branch.go1.21] doc: delete go1.21.html copy
When the final Go 1.21.0 release is made, the release notes will lose
the draft status and begin to be visible at https://go.dev/doc/go1.21.
The source go1.21.html file will then move from the main Go repository
to x/website as part of it becoming a finished and eventually frozen
historical release notes page.

The https://tip.golang.org/doc/go1.21 page that currently serves the
current draft always uses the main repository's main branch, so this
copy on the release branch is not used by anything. Its existence on
this branch may attract backports, which are harmless but also not
very useful. Avoid that temptation by deleting it here sooner.

(Since the first RC comes with a mostly complete release notes draft,
maybe in the future we can consider moving this file to x/website
right before a release branch is cut, making the draft available at
https://go.dev/doc/go1.21 sooner so fewer links will need updating.)

For #58645.

Change-Id: I92c0200b748a5f255f9b8113b4952c122631c6d6
Reviewed-on: https://go-review.googlesource.com/c/go/+/511317
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-07-20 19:16:29 +00:00
qmuntal
7ee7a21ef2 [release-branch.go1.21] runtime: ensure stack is aligned in _rt0_amd64_windows_lib
The Windows DLL loader may call a DLL entry point, in our case
_rt0_amd64_windows_lib, with a stack that is
not 16-byte aligned. In theory, it shouldn't, but under some
circumstances, it does (see below how to reproduce it).

Having an unaligned stack can, and probably will, cause problems
down the line, for example if a movaps instruction tries to store
a value in an unaligned address it throws an Access Violation exception
(code 0xc0000005).

I managed to consistently reproduce this issue by loading a Go DLL into
a C program that has the Page Heap Verification diagnostic enabled [1].

Updates #54187 (and potentially fixes)

[1] https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/example-12---using-page-heap-verification-to-find-a-bug

Change-Id: Id0fea7f407e024c9b8cdce10ce4802d7535e7542
Reviewed-on: https://go-review.googlesource.com/c/go/+/510755
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
(cherry picked from commit 5fe3f0a265)
Reviewed-on: https://go-review.googlesource.com/c/go/+/511135
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
2023-07-20 18:24:32 +00:00
Heschi Kreinick
06a9034b60 [release-branch.go1.21] net: tolerate permission errors in interface tests
On our linux-arm64 builders, we're getting permission errors despite
running as root. Detect those errors and skip the test.

Fixes #61414.

Change-Id: I5d7c45789337bee3860b19335bbb9eb884c48986
Reviewed-on: https://go-review.googlesource.com/c/go/+/510737
Auto-Submit: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
(cherry picked from commit 8e1ec1cb93)
Reviewed-on: https://go-review.googlesource.com/c/go/+/511136
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2023-07-20 15:04:53 +00:00
Guoqi Chen
03c7e96be9 [release-branch.go1.21] doc/go1.21: add release notes for linux/loong64
Fixes #53301

Change-Id: Id447d57d43b12c3748267295928d45a089549340
Reviewed-on: https://go-review.googlesource.com/c/go/+/507815
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Bypass: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Meidan Li <limeidan@loongson.cn>
TryBot-Bypass: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
(cherry picked from commit 700727151f)
Reviewed-on: https://go-review.googlesource.com/c/go/+/510916
Reviewed-by: Heschi Kreinick <heschi@google.com>
2023-07-19 21:06:47 +00:00
Heschi Kreinick
c2de6836c1 [release-branch.go1.21] all: merge master (2639a17) into release-branch.go1.21
Change-Id: I316578aec6c2b618c840d43a65ef87b8d2168ac0
2023-07-18 13:25:02 -04:00
Gopher Robot
4aeac326b5 [release-branch.go1.21] go1.21rc3
Change-Id: Ibab6e2a998d5b57fcae923fa0627037846af1492
Reviewed-on: https://go-review.googlesource.com/c/go/+/509775
Reviewed-by: Eli Bendersky <eliben@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Gopher Robot <gobot@golang.org>
Auto-Submit: Gopher Robot <gobot@golang.org>
2023-07-14 15:07:26 +00:00
Cherry Mui
9480b4adf9 [release-branch.go1.21] all: merge master (b88bd91) into release-branch.go1.21
Merge List:

+ 2023-07-13 b88bd917b8 cmd/go/internal/modfetch: always allow Stat for the current toolchain to succeed
+ 2023-07-13 d4f0d896a6 net/http: revert stray edit to h2_bundle.go from CL 508996
+ 2023-07-13 4a0f51696e all: remove duplicate word and fix comment
+ 2023-07-12 49d42128fd all: fix typos and remove repeated words

Change-Id: Ie4f17f633483d5cd1d244b6e7cde6422dc246d9e
2023-07-13 12:46:26 -04:00
Cherry Mui
cc0cb3020d [release-branch.go1.21] all: merge master (6244b19) into release-branch.go1.21
Merge List:

+ 2023-07-12 6244b1946b all: update vendored dependencies
+ 2023-07-12 b4872ea187 cmd/go: fix go get go@badversion
+ 2023-07-12 87350393e6 cmd/go: fix GOTOOLCHAIN parsing for +auto
+ 2023-07-12 230e549142 doc/go1.21: add a release note for CL 463177
+ 2023-07-12 5e4000ad7f cmd/compile: on PPC64, fix sign/zero extension when masking
+ 2023-07-11 af8f94e3c5 src/README.vendor: s/latest/master/
+ 2023-07-11 3eaee3d5dd testing/slogtest: check for no group with empty record
+ 2023-07-11 167c8b73bf net/http/fcgi: eliminate goroutine leaks in tests
+ 2023-07-11 cb7a091d72 os/exec: ignore context.Canceled errors in TestConcurrentExec
+ 2023-07-11 651869716a log/slog: replace nil contexts with context.Background()
+ 2023-07-11 3f8b04bfb5 log/slog: change XXXCtx functions to XXXContext
+ 2023-07-11 4c58d6bf52 time: increase arbitrary upper bound in TestReset to 10s
+ 2023-07-11 6a063b01b0 net: mptcp: force using MPTCP with GODEBUG
+ 2023-07-10 07ede7a543 syscall: serialize locks on ForkLock on platforms where forkExecPipe is not atomic
+ 2023-07-10 7dc62f3bda test: add test cases for index value with range array clear
+ 2023-07-10 0b65b02ba5 cmd/compile: fix clear on slice with zero size elem
+ 2023-07-10 c4db811e44 cmd/compile: don't ICE on unaligned offsets for pointer writes
+ 2023-07-07 5c15498609 os: support reading empty root directories on Windows
+ 2023-07-07 894d24d617 src/database/sql: run gofmt
+ 2023-07-07 ac9137f8d3 go/types, types2: do not mutate arguments in NewChecker
+ 2023-07-07 158d11196f math: add test that covers riscv64 fnm{add,sub} codegen
+ 2023-07-07 d3d78b4bcc log/slog: handle recursively empty groups
+ 2023-07-06 39c5070712 os: do not skip directory entries with zero inodes on wasip1
+ 2023-07-06 e6ec2a34dc runtime: print output on failure in TestMemPprof
+ 2023-07-06 449ef3795d net: only build cgo_stub.go on unix or wasip1
+ 2023-07-06 6305d7fdd3 cmd/go: pass GoVersion in vet config
+ 2023-07-06 b490bdc27d go/types: record Config.GoVersion for reporting in Package.GoVersion method
+ 2023-07-06 36ea4f9680 log/slog: fix faulty test
+ 2023-07-05 3fce111535 cmd/compile: fix FMA negative commutativity of riscv64
+ 2023-07-05 c8dad424bf math: fix portable FMA when x*y < 0 and x*y == -z
+ 2023-07-05 cd6676126b database/sql: prevent internal context error from being returned from Rows.Err()
+ 2023-07-05 b2215e49c7 runtime,runtime/metrics: clarify OS stack metrics
+ 2023-07-04 e126572f8a runtime: have ReadMemStats do a nil check before switching stacks
+ 2023-07-04 e4ed92a355 os, syscall: update unreachable link about =C: envs
+ 2023-07-04 5c1a15df41 test/codegen: enable Mul2 DivPow2 test for riscv64
+ 2023-06-30 5b72f45dd1 runtime: check GOFLAGS not GCFLAGS
+ 2023-06-29 18e17e2cb1 net: enable pure Go resolver for wasip1
+ 2023-06-29 1e97c51536 syscall: stub Getrlimit on wasip1
+ 2023-06-29 683f51d307 cmd/asm/internal/lex: fix comment, remove the first "has"
+ 2023-06-29 2db31efdba cmd/compile/internal/types2: make TestIssue43124 match the go/types version
+ 2023-06-29 499458f7ca net/http: validate Host header before sending
+ 2023-06-29 fe73c186eb cmd/{go,compile}: run gofmt
+ 2023-06-29 da5d8fdd0c runtime: run wasip1 tests with wazero
+ 2023-06-29 411c99671a slices, maps: add examples; doc comment fixes
+ 2023-06-29 03cd8a7b0e internal/bytealg: fix alignment code in equal_riscv64.s
+ 2023-06-28 8b5fe5980c cmd/compile: handle min/max correctly in mayCall
+ 2023-06-28 79d4defa75 cmd/compile/internal/ssagen: fix min/max codegen, again
+ 2023-06-28 a3093eca64 cmd/go: enable slog vet check during 'go test'
+ 2023-06-27 6691f438c3 cmd/dist, internal/abi: support bootstrapping with gccgo
+ 2023-06-27 942c1c12d8 runtime: fix trace.Stop deadlock when built with faketime
+ 2023-06-27 4ad4128d3c cmd/compile: fix bad order of evaluation for min/max builtin
+ 2023-06-27 4f36f7e4dd encoding: document that base32 and base64 do not use UTF-8
+ 2023-06-27 8008c0840f syscall: clarify which handles are affected by SysProcAttr.NoInheritHandles
+ 2023-06-27 13529cc5f4 syscall: try non-blocking stdio on wasip1
+ 2023-06-26 1dbbafc70f go/types, types2: replace TODO with clarifying comment
+ 2023-06-26 7cc0740596 slices: add godoc links
+ 2023-06-26 d49017a7c3 go/types, types2: fix interface unification
+ 2023-06-26 b3ca8d2b3c types2, go/types: record final type for min/max arguments
+ 2023-06-26 ee361ce66c go/types, types2: more readable inference trace
+ 2023-06-26 9f6e87ff74 doc/go1.21: document changes in crypto/x509
+ 2023-06-26 f5015b5164 doc/go1.21: context.Background and TODO may now appear equal
+ 2023-06-24 a031f4ef83 cmd/compile: fix min/max builtin code generation
+ 2023-06-24 ea927e560d slices: clarify MinFunc/MaxFunc result for equal elements
+ 2023-06-23 3619255777 crypto/x509: rename duplicated test
+ 2023-06-23 48dbb6227a runtime: set raceignore to zero when starting a new goroutine
+ 2023-06-23 3adcce5ae7 crypto: document non-determinism of GenerateKey
+ 2023-06-23 6dce882b3a cmd/compile: scanning closures body when visiting wrapper function
+ 2023-06-23 164aceea08 log/slog: fix broken link to AnyValue in comment
+ 2023-06-23 82e17c4d11 log/slog: fix broken link to Record.Clone in package docs
+ 2023-06-22 25e46693a1 cmd/go: impersonate 'go tool dist list' if 'go tool dist' is not present
+ 2023-06-22 f8616b8484 internal/platform,cmd/dist: export the list of supported platforms
+ 2023-06-22 51885c1fa2 cmd/{go,cover}: enable response file args for cmd/cover
+ 2023-06-22 3479e1e543 internal/fuzz: fix typo in comment
+ 2023-06-22 20313660f5 crypto/x509: tolerate multiple matching chains in testVerify
+ 2023-06-21 78c3aba470 net/mail: permit more characters in mail headers
+ 2023-06-21 633b742ae0 test: add test that caused a gofrontend crash
+ 2023-06-21 ad2c517708 doc/go1.21: correct GOOS to GOARCH (another location)
+ 2023-06-21 b23cae8095 doc/go1.21: correct GOOS to GOARCH
+ 2023-06-21 b1f1fb2950 go/types, types2: avoid spurious "declared and not used" error
+ 2023-06-21 e45202f215 runtime: relate GODEBUG=gctrace output to runtime/metrics
+ 2023-06-21 36edde9d9f cmd/go: shorten longest 5 tests
+ 2023-06-21 413e6c0499 cmd/compile/internal/ir: typo
+ 2023-06-20 f3bf18117b log/slog: fix HandlerOptions.ReplaceAttr doc
+ 2023-06-20 e122ebabb6 encoding/binary: on invalid type return -1 from Size
+ 2023-06-20 8484f2fe02 cmd/go: add comment for intentional misspelling

Change-Id: Ie91c398dde7ca2a210b286c7865d3e7a905d6ea9
2023-07-12 17:23:13 -04:00
Gopher Robot
d8117459c5 [release-branch.go1.21] go1.21rc2
Change-Id: I4a2c753f3edf22290e0e71a90df6d380fc009202
Reviewed-on: https://go-review.googlesource.com/c/go/+/504876
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
Run-TryBot: Gopher Robot <gobot@golang.org>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2023-06-21 15:40:05 +00:00
Dmitri Shuralyov
ebbff91f59 [release-branch.go1.21] all: merge master (a7b1793) into release-branch.go1.21
Merge List:

+ 2023-06-20 a7b1793701 cmd/go: do not index std as a module in modcache
+ 2023-06-20 3d279283a4 cmd/go: restore go.mod files during toolchain selection
+ 2023-06-20 3b4b7b84de cmd/distpack: rename go.mod to _go.mod in toolchain modules
+ 2023-06-20 6459494014 cmd/go: disable sumdb less often for toolchain downloads
+ 2023-06-20 02789816c4 internal/bisect: add 'q' hash option for quiet hash behavior switching
+ 2023-06-20 98617fd23f runtime/trace: add godoc links
+ 2023-06-19 bc21d6a4fc cmd/go/internal/modfetch: fix retractions slice initial length not zero
+ 2023-06-17 261e267618 os/exec: document a method to check if a process is alive
+ 2023-06-16 dbf9bf2c39 cmd/internal/moddeps: allow the "misc" module to be missing from GOROOT
+ 2023-06-16 0183c1aa02 cmd/compile/internal/syntax: skip GOROOT/misc in TestStdLib if it doesn't exist
+ 2023-06-16 199fbd4b59 cmd/internal/testdir: skip Test if GOROOT/test does not exist
+ 2023-06-16 a48f9c26d5 go/types: skip tests that require GOROOT/test if it is not present
+ 2023-06-16 3891ecbd35 go/internal/gcimporter: skip TestImportTypeparamTests if GOROOT/test is missing
+ 2023-06-16 60876717b4 cmd/go/internal/test: don't wait for previous test actions when interrupted
+ 2023-06-16 c1bc44642d path/filepath: avoid assuming that GOROOT/test is present
+ 2023-06-16 9ece9a7ac9 cmd/cgo/internal/testshared: disable gccgo tests on PPC64
+ 2023-06-16 23c5e48c4a cmd/cgo/internal/testshared: strip newline from gccgo -dumpversion
+ 2023-06-16 cf7ae4f136 compress/bzip2: fix typo
+ 2023-06-16 3c8b7a9551 net/http: check RemoteAddr isn't nil before dereferencing
+ 2023-06-16 548790e64a net/http: close req.Body only when it's non-nil on js
+ 2023-06-16 6dc2d2aa6b testing/fstest: fix the Glob test when dir entries are out of order
+ 2023-06-16 2b0ff4b629 reflect: fix ArenaNew to match documentation
+ 2023-06-16 4eceefa338 cmd/distpack: make go_$GOOS_$GOARCH_exec programs executable
+ 2023-06-16 1a7709d6af runtime: use 1-byte load for address checking in racecallatomic
+ 2023-06-15 3e7ec13166 cmd/go: fix build config for 'go list -cover'
+ 2023-06-15 30b17f4f97 net/http: only disable Fetch API in tests
+ 2023-06-15 65db95d0ed math: document that Min/Max differ from min/max
+ 2023-06-15 60e6afb689 cmd/compile: do not report division by error during typecheck
+ 2023-06-15 f6e0dcc474 slices: add sort benchmark for sorted strings

Change-Id: If342a000b719335fbbb421f027a8b253b07c1cab
2023-06-20 13:19:35 -04:00
Gopher Robot
1c1c82432a [release-branch.go1.21] go1.21rc1
time 2023-06-15T19:02:18Z

Change-Id: If39aa20f4858a5a794f30e3564b0165e6d096b70
Reviewed-on: https://go-review.googlesource.com/c/go/+/504015
Auto-Submit: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Bypass: Michael Pratt <mpratt@google.com>
Run-TryBot: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: David Chase <drchase@google.com>
2023-06-16 14:21:46 +00:00
Michael Pratt
b4a0665266 [release-branch.go1.21] all: merge master (b7e7467) into release-branch.go1.21
Merge List:

+ 2023-06-15 b7e7467865 test/codegen: add fsqrt test for riscv64
+ 2023-06-15 befec5ddbb text/template: set variables correctly in range assignment
+ 2023-06-15 c5463218a2 cmd/api: skip TestIssue29837 when -short is set
+ 2023-06-15 9fc84363d1 cmd/asm: fix encoding errors for FMOVD and FMOVS instructions on arm64
+ 2023-06-14 da94586aa3 cmd/go: check for errors reading gccgo package list
+ 2023-06-14 b01cd41b46 cmd/go: use gover.Local for $goversion in TestScript
+ 2023-06-14 3aea422e2c crypto/x509: use synthetic root for platform testing

Change-Id: Icec55130749c52eef75c0e0325d889ff18b067f6
2023-06-15 11:36:34 -04:00
Michael Pratt
577e7b9bb9 [release-branch.go1.21] update codereview.cfg for release-branch.go1.21
Change-Id: Ib853d63f300b75ae399529dadd9baa03693b6f60
Reviewed-on: https://go-review.googlesource.com/c/go/+/503435
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-06-14 20:26:29 +00:00
6167 changed files with 140642 additions and 554066 deletions

45
.github/ISSUE_TEMPLATE/00-bug.md vendored Normal file
View File

@@ -0,0 +1,45 @@
---
name: Bugs
about: The go command, standard library, or anything else
title: "affected/package: "
---
<!--
Please answer these questions before submitting your issue. Thanks!
-->
### What version of Go are you using (`go version`)?
<pre>
$ go version
</pre>
### Does this issue reproduce with the latest release?
### What operating system and processor architecture are you using (`go env`)?
<details><summary><code>go env</code> Output</summary><br><pre>
$ go env
</pre></details>
### What did you do?
<!--
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on go.dev/play is best.
-->
### What did you expect to see?
### What did you see instead?

View File

@@ -1,94 +0,0 @@
# https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository#creating-issue-forms
# https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema
name: Bugs
description: The go command, standard library, or anything else
title: "import/path: issue title"
body:
- type: markdown
attributes:
value: |
Thanks for helping us improve! 🙏 Please answer these questions and provide as much information as possible about your problem.
- type: input
id: go-version
attributes:
label: Go version
description: |
What version of Go are you using (`go version`)?
Note: we only [support](https://go.dev/doc/devel/release#policy) the two most recent major releases.
placeholder: ex. go version go1.20.7 darwin/arm64
validations:
required: true
- type: textarea
id: go-env
attributes:
label: "Output of `go env` in your module/workspace:"
placeholder: |
GO111MODULE=""
GOARCH="arm64"
GOBIN="/Users/gopher/go/bin"
GOCACHE="/Users/gopher/go/cache"
GOENV="/Users/gopher/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/gopher/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/gopher/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.20.7"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/44/nbbyll_10jd0z8rj_qxm43740000gn/T/go-build2331607515=/tmp/go-build -gno-record-gcc-switches -fno-common"
render: shell
validations:
required: true
- type: textarea
id: what-did-you-do
attributes:
label: "What did you do?"
description: "If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on [go.dev/play](https://go.dev/play) is best."
validations:
required: true
- type: textarea
id: actual-behavior
attributes:
label: "What did you see happen?"
description: Command invocations and their associated output, functions with their arguments and return results, full stacktraces for panics (upload a file if it is very long), etc. Prefer copying text output over using screenshots.
validations:
required: true
- type: textarea
id: expected-behavior
attributes:
label: "What did you expect to see?"
description: Why is the current output incorrect, and any additional context we may need to understand the issue.
validations:
required: true

49
.github/ISSUE_TEMPLATE/01-pkgsite.md vendored Normal file
View File

@@ -0,0 +1,49 @@
---
name: Pkg.go.dev bugs or feature requests
about: Issues or feature requests for the documentation site
title: "x/pkgsite: "
labels: pkgsite
---
<!--
Please answer these questions before submitting your issue. Thanks!
-->
### What is the URL of the page with the issue?
### What is your user agent?
<!--
You can find your user agent here:
https://www.google.com/search?q=what+is+my+user+agent
-->
### Screenshot
<!--
Please paste a screenshot of the page.
-->
### What did you do?
<!--
If possible, provide a recipe for reproducing the error.
Starting with a Private/Incognito tab/window may help rule out problematic browser extensions.
-->
### What did you expect to see?
### What did you see instead?

View File

@@ -1,47 +0,0 @@
name: Pkg.go.dev bugs or feature requests
description: Issues or feature requests for the documentation site
title: "x/pkgsite: issue title"
labels: ["pkgsite"]
body:
- type: markdown
attributes:
value: "Please answer these questions before submitting your issue. Thanks!"
- type: input
id: url
attributes:
label: "What is the URL of the page with the issue?"
validations:
required: true
- type: input
id: user-agent
attributes:
label: "What is your user agent?"
description: "You can find your user agent here: https://www.google.com/search?q=what+is+my+user+agent"
validations:
required: true
- type: textarea
id: screenshot
attributes:
label: "Screenshot"
description: "Please paste a screenshot of the page."
validations:
required: false
- type: textarea
id: what-did-you-do
attributes:
label: "What did you do?"
description: "If possible, provide a recipe for reproducing the error. Starting with a Private/Incognito tab/window may help rule out problematic browser extensions."
validations:
required: true
- type: textarea
id: actual-behavior
attributes:
label: "What did you see happen?"
validations:
required: true
- type: textarea
id: expected-behavior
attributes:
label: "What did you expect to see?"
validations:
required: true

View File

@@ -0,0 +1,39 @@
---
name: Pkg.go.dev package removal request
about: Request a package be removed from the documentation site (pkg.go.dev)
title: "x/pkgsite: package removal request for [type path here]"
labels: pkgsite/package-removal
---
<!--
Please answer these questions before submitting your issue. Thanks!
-->
### What is the path of the package that you would like to have removed?
<!---
We can remove packages with a shared path prefix.
For example, a request for "github.com/author" would remove all pkg.go.dev pages with that package path prefix.
--->
### Are you the owner of this package?
<!---
Only the package owners can request to have their packages removed from pkg.go.dev.
--->
### What is the reason that you could not retract this package instead?
<!---
If you would like to have your module removed from pkg.go.dev, we recommend that you retract them, so that they can be removed from the go command and proxy.golang.org as well.
Retracting a module version involves adding a retract directive to your go.mod file and publishing a new version. For example: https://github.com/jba/retract-demo/blob/main/go.mod#L5-L8
See https://pkg.go.dev/about#removing-a-package for additional tips on retractions.
--->

View File

@@ -1,42 +0,0 @@
name: Pkg.go.dev package removal request
description: Request a package be removed from the documentation site (pkg.go.dev)
title: "x/pkgsite: package removal request for [type path here]"
labels: ["pkgsite/package-removal"]
body:
- type: markdown
attributes:
value: "Please answer these questions before submitting your issue. Thanks!"
- type: input
id: package-path
attributes:
label: "What is the path of the package that you would like to have removed?"
description: |
We can remove packages with a shared path prefix.
For example, a request for 'github.com/author' would remove all pkg.go.dev pages with that package path prefix.
validations:
required: true
- type: textarea
id: package-owner
attributes:
label: "Are you the owner of this package?"
description: |
Only the package owners can request to have their packages removed from pkg.go.dev.
If the package path doesn't include your github username, please provide some other form of proof of ownership.
validations:
required: true
- type: textarea
id: retraction-reason
attributes:
label: "What is the reason that you could not retract this package instead?"
description: |
Requesting we remove a module here only hides the generated documentation on pkg.go.dev.
It does not affect the behaviour of proxy.golang.org or the go command.
Instead we recommend using the retract directive which will be processed by all 3 of the above.
If you have deleted your repo, please recreate it and publish a retraction.
Retracting a module version involves adding a retract directive to your go.mod file and publishing a new version.
For example: https://github.com/jba/retract-demo/blob/main/go.mod#L5-L8.
See https://pkg.go.dev/about#removing-a-package for additional tips on retractions.
validations:
required: true

61
.github/ISSUE_TEMPLATE/03-gopls.md vendored Normal file
View File

@@ -0,0 +1,61 @@
---
name: Gopls bugs or feature requests
about: Issues or feature requests for the Go language server (gopls)
title: "x/tools/gopls: "
labels: gopls Tools
---
<!--
Please answer these questions before submitting your issue. Thanks!
-->
### gopls version
<!--
Output of `gopls -v version` on the command line
-->
### go env
<!--
Output of `go env` on the command line in your workspace directory
-->
### What did you do?
<!--
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on go.dev/play is better.
A failing unit test is the best.
-->
### What did you expect to see?
### What did you see instead?
### Editor and settings
<!--
Your editor and any settings you have configured (for example, your VSCode settings.json file)
-->
### Logs
<!--
If possible please include gopls logs. Instructions for capturing them can be found here:
https://github.com/golang/tools/blob/master/gopls/doc/troubleshooting.md#capture-logs
-->

View File

@@ -1,56 +0,0 @@
name: Gopls bugs or feature requests
description: Issues or feature requests for the Go language server (gopls)
title: "x/tools/gopls: issue title"
labels: ["gopls", "Tools"]
body:
- type: markdown
attributes:
value: "Please answer these questions before submitting your issue. Thanks!"
- type: textarea
id: gopls-version
attributes:
label: "gopls version"
description: "Output of `gopls -v version` on the command line"
validations:
required: true
- type: textarea
id: go-env
attributes:
label: "go env"
description: "Output of `go env` on the command line in your workspace directory"
render: shell
validations:
required: true
- type: textarea
id: what-did-you-do
attributes:
label: "What did you do?"
description: "If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on [go.dev/play](https://go.dev/play) is better. A failing unit test is the best."
validations:
required: true
- type: textarea
id: actual-behavior
attributes:
label: "What did you see happen?"
validations:
required: true
- type: textarea
id: expected-behavior
attributes:
label: "What did you expect to see?"
validations:
required: true
- type: textarea
id: editor-and-settings
attributes:
label: "Editor and settings"
description: "Your editor and any settings you have configured (for example, your VSCode settings.json file)"
validations:
required: false
- type: textarea
id: logs
attributes:
label: "Logs"
description: "If possible please include gopls logs. Instructions for capturing them can be found here: https://github.com/golang/tools/blob/master/gopls/doc/troubleshooting.md#capture-logs"
validations:
required: false

51
.github/ISSUE_TEMPLATE/04-vuln.md vendored Normal file
View File

@@ -0,0 +1,51 @@
---
name: Go vulnerability management - bugs and feature requests
about: Issues or feature requests about Go vulnerability management
title: "x/vuln: "
labels: "vulncheck or vulndb"
---
<!--
Please answer these questions before submitting your issue. Thanks!
To add a new vulnerability to the Go vulnerability database
(https://vuln.go.dev), see https://go.dev/s/vulndb-report-new.
To report an issue about a report, see https://go.dev/s/vulndb-report-feedback.
-->
### What version of Go are you using (`go version`)?
<pre>
$ go version
</pre>
### Does this issue reproduce at the latest version of golang.org/x/vuln?
### What operating system and processor architecture are you using (`go env`)?
<details><summary><code>go env</code> Output</summary><br><pre>
$ go env
</pre></details>
### What did you do?
<!--
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on go.dev/play is best.
-->
### What did you expect to see?
### What did you see instead?

View File

@@ -1,52 +0,0 @@
name: Go vulnerability management - bugs and feature requests
description: Issues or feature requests about Go vulnerability management
title: "x/vuln: issue title"
labels: ["vulncheck or vulndb"]
body:
- type: markdown
attributes:
value: "Please answer these questions before submitting your issue. Thanks! To add a new vulnerability to the Go vulnerability database (https://vuln.go.dev), see https://go.dev/s/vulndb-report-new. To report an issue about a report, see https://go.dev/s/vulndb-report-feedback."
- type: textarea
id: govulncheck-version
attributes:
label: govulncheck version
description: What version of govulncheck are you using (`govulncheck -version`)?
placeholder: |
Go: devel go1.22-0262ea1ff9 Thu Oct 26 18:46:50 2023 +0000
Scanner: govulncheck@v1.0.2-0.20231108200754-fcf7dff7b242
DB: https://vuln.go.dev
DB updated: 2023-11-21 15:39:17 +0000 UTC
validations:
required: true
- type: textarea
id: reproduce-latest-version
attributes:
label: "Does this issue reproduce at the latest version of golang.org/x/vuln?"
validations:
required: true
- type: textarea
id: go-env
attributes:
label: "Output of `go env` in your module/workspace:"
render: shell
validations:
required: true
- type: textarea
id: what-did-you-do
attributes:
label: "What did you do?"
description: "If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on [go.dev/play](https://go.dev/play) is best."
validations:
required: true
- type: textarea
id: actual-behavior
attributes:
label: "What did you see happen?"
validations:
required: true
- type: textarea
id: expected-behavior
attributes:
label: "What did you expect to see?"
validations:
required: true

13
.github/ISSUE_TEMPLATE/10-proposal.md vendored Normal file
View File

@@ -0,0 +1,13 @@
---
name: Proposals
about: New external API or other notable changes
title: "proposal: affected/package: "
labels: Proposal
---
<!--
Our proposal process is documented here:
https://go.dev/s/proposal-process
-->

View File

@@ -1,15 +0,0 @@
name: Proposals
description: New external API or other notable changes
title: "proposal: import/path: proposal title"
labels: ["Proposal"]
body:
- type: markdown
attributes:
value: "Our proposal process is documented here: https://go.dev/s/proposal-process"
- type: textarea
id: proposal-details
attributes:
label: "Proposal Details"
description: "Please provide the details of your proposal here."
validations:
required: true

View File

@@ -0,0 +1,52 @@
---
name: Language Change Proposals
about: Changes to the language
title: "proposal: Go 2: "
labels: Proposal Go2 LanguageChange
---
<!--
Our process for evaluating language changes can be found here:
https://go.googlesource.com/proposal/+/refs/heads/master#language-changes
-->
### Author background
- **Would you consider yourself a novice, intermediate, or experienced Go programmer?**
- **What other languages do you have experience with?**
### Related proposals
- **Has this idea, or one like it, been proposed before?**
- **If so, how does this proposal differ?**
- **Does this affect error handling?**
- **If so, how does this differ from previous error handling proposals?**
- **Is this about generics?**
- **If so, how does this relate to the accepted design and other generics proposals?**
### Proposal
- **What is the proposed change?**
- **Who does this proposal help, and why?**
- **Please describe as precisely as possible the change to the language.**
- **What would change in the language spec?**
- **Please also describe the change informally, as in a class teaching Go.**
- **Is this change backward compatible?**
- Breaking the Go 1 compatibility guarantee is a large cost and requires a large benefit.
Show example code before and after the change.
- **Before**
- **After**
- **Orthogonality: how does this change interact or overlap with existing features?**
- **Is the goal of this change a performance improvement?**
- **If so, what quantifiable improvement should we expect?**
- **How would we measure it?**
### Costs
- **Would this change make Go easier or harder to learn, and why?**
- **What is the cost of this proposal? (Every language change has a cost).**
- **How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected?**
- **What is the compile time cost?**
- **What is the run time cost?**
- **Can you describe a possible implementation?**
- **Do you have a prototype? (This is not required.)**

View File

@@ -1,165 +0,0 @@
name: Language Change Proposals
description: Changes to the language
labels: ["Proposal", "LanguageChange", "LanguageChangeReview"]
title: "proposal: spec: proposal title"
body:
- type: markdown
attributes:
value: |
## Our process for evaluating language changes can be found [here](https://go.googlesource.com/proposal/+/refs/heads/master#language-changes)
- type: dropdown
id: author-go-experience
attributes:
label: "Go Programming Experience"
description: "Would you consider yourself a novice, intermediate, or experienced Go programmer?"
options:
- "Novice"
- "Intermediate"
- "Experienced"
default: 1
- type: input
id: author-other-languages-experience
attributes:
label: "Other Languages Experience"
description: "What other languages do you have experience with?"
placeholder: "Go, Python, JS, Rust"
validations:
required: false
- type: checkboxes
id: related-idea
attributes:
label: "Related Idea"
options:
- label: "Has this idea, or one like it, been proposed before?"
- label: "Does this affect error handling?"
- label: "Is this about generics?"
- label: "Is this change backward compatible? Breaking the Go 1 compatibility guarantee is a large cost and requires a large benefit"
- type: textarea
id: related-proposals
attributes:
label: Has this idea, or one like it, been proposed before?
description: If so, how does this proposal differ?
placeholder: |
Yes or No
If yes,
1. Mention the related proposals
2. then describe how this proposal differs
validations:
required: true
- type: textarea
id: error-handling-proposal
attributes:
label: Does this affect error handling?
description: If so, how does this differ from previous error handling proposals?
placeholder: |
Yes or No
If yes,
1.how does this differ from previous error handling proposals?
validations:
required: true
- type: textarea
id: generics-proposal
attributes:
label: Is this about generics?
description: If so, how does this relate to the accepted design and other generics proposals?
placeholder: |
Yes or No
If yes,
1. how does this relate to the accepted design and other generics proposals?
validations:
required: true
- type: textarea
id: proposal
attributes:
label: "Proposal"
description: "What is the proposed change? Who does this proposal help, and why? Please describe as precisely as possible the change to the language."
validations:
required: true
- type: textarea
id: language-spec-changes
attributes:
label: "Language Spec Changes"
description: "What would change in the language spec?"
validations:
required: false
- type: textarea
id: informal-change
attributes:
label: "Informal Change"
description: "Please also describe the change informally, as in a class teaching Go."
validations:
required: false
- type: textarea
id: go-backwards-compatiblity
attributes:
label: Is this change backward compatible?
description: Breaking the Go 1 compatibility guarantee is a large cost and requires a large benefit.
placeholder: |
Yes or No
If yes,
1. Show example code before and after the change.
validations:
required: true
- type: textarea
id: orthogonality
attributes:
label: "Orthogonality: How does this change interact or overlap with existing features?"
description: "Is the goal of this change a performance improvement? If so, what quantifiable improvement should we expect? How would we measure it?"
validations:
required: false
- type: textarea
id: learning-curve
attributes:
label: "Would this change make Go easier or harder to learn, and why?"
- type: textarea
id: cost-description
attributes:
label: "Cost Description"
description: "What is the cost of this proposal? (Every language change has a cost)"
- type: input
id: go-toolchain
attributes:
label: Changes to Go ToolChain
description: "How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected? "
validations:
required: false
- type: input
id: perf-costs
attributes:
label: Performance Costs
description: "What is the compile time cost? What is the run time cost? "
validations:
required: false
- type: textarea
id: prototype
attributes:
label: "Prototype"
description: "Can you describe a possible implementation?"
validations:
required: false

View File

@@ -1,30 +0,0 @@
name: Go Telemetry Proposals
description: Changes to the telemetry upload configuration
title: "x/telemetry/config: proposal title"
labels: ["Telemetry-Proposal"]
projects: ["golang/29"]
body:
- type: textarea
attributes:
label: Summary
description: >
What change are you proposing to the upload configuration, and why?
For new upload configuration, which new counters will be collected, what
do they measure, and why is it important to collect them?
Note that uploaded data must not carry sensitive user information.
See [go.dev/doc/telemetry#proposals](https://go.dev/doc/telemetry#proposals)
for more details on telemetry proposals.
validations:
required: true
- type: input
attributes:
label: Proposed Config Change
description: >
A CL containing proposed changes to the
[config.txt](https://go.googlesource.com/telemetry/+/master/internal/chartconfig/config.txt)
chart configuration.
See the [chartconfig](https://pkg.go.dev/golang.org/x/telemetry/internal/chartconfig)
package for an explanation of the chart config format.
For an example change, see [CL 564619](https://go.dev/cl/564619).
validations:
required: true

View File

@@ -1,4 +1,4 @@
blank_issues_enabled: true
blank_issues_enabled: false
contact_links:
- name: Questions
about: Please use one of the forums for questions or general discussions

2
.gitignore vendored
View File

@@ -37,7 +37,7 @@ _testmain.go
/src/go/build/zcgo.go
/src/go/doc/headscan
/src/internal/buildcfg/zbootstrap.go
/src/internal/runtime/sys/zversion.go
/src/runtime/internal/sys/zversion.go
/src/unicode/maketables
/src/time/tzdata/zzipdata.go
/test.out

View File

@@ -1,4 +1,4 @@
Copyright 2009 The Go Authors.
Copyright (c) 2009 The Go Authors. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
@@ -10,7 +10,7 @@ notice, this list of conditions and the following disclaimer.
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google LLC nor the names of its
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

View File

@@ -4,7 +4,7 @@ Go is an open source programming language that makes it easy to build simple,
reliable, and efficient software.
![Gopher image](https://golang.org/doc/gopher/fiveyears.jpg)
*Gopher image by [Renee French][rf], licensed under [Creative Commons 4.0 Attribution license][cc4-by].*
*Gopher image by [Renee French][rf], licensed under [Creative Commons 4.0 Attributions license][cc4-by].*
Our canonical Git repository is located at https://go.googlesource.com/go.
There is a mirror of the repository at https://github.com/golang/go.

View File

@@ -10,4 +10,4 @@ part of that page.
## Reporting a Vulnerability
See https://go.dev/security/policy for how to report a vulnerability.
See https://go.dev/security for how to report a vulnerability.

2
VERSION Normal file
View File

@@ -0,0 +1,2 @@
go1.21.9
time 2024-03-29T15:27:02Z

View File

@@ -21,6 +21,3 @@ warning output from the go api tool. Each file should be named
nnnnn.txt, after the issue number for the accepted proposal.
(The #nnnnn suffix must also appear at the end of each line in the file;
that will be preserved when next/*.txt is concatenated into go1.XX.txt.)
When you add a file to the api/next directory, you must add at least one file
under doc/next. See doc/README.md for details.

View File

@@ -598,7 +598,3 @@ pkg syscall (freebsd-arm64-cgo), const SYS_MKNODAT = 498
pkg syscall (freebsd-arm64-cgo), const SYS_STAT = 188
pkg syscall (freebsd-arm64-cgo), const SYS_STAT ideal-int
pkg syscall (freebsd-arm64-cgo), const SYS_STATFS = 396
pkg syscall (openbsd-386), const ELAST = 91
pkg syscall (openbsd-386-cgo), const ELAST = 91
pkg syscall (openbsd-amd64), const ELAST = 91
pkg syscall (openbsd-amd64-cgo), const ELAST = 91

View File

@@ -167,12 +167,7 @@ pkg errors, var ErrUnsupported error #41198
pkg flag, func BoolFunc(string, string, func(string) error) #53747
pkg flag, method (*FlagSet) BoolFunc(string, string, func(string) error) #53747
pkg go/ast, func IsGenerated(*File) bool #28089
pkg go/ast, func NewPackage //deprecated #52463
pkg go/ast, type File struct, GoVersion string #59033
pkg go/ast, type Importer //deprecated #52463
pkg go/ast, type Object //deprecated #52463
pkg go/ast, type Package //deprecated #52463
pkg go/ast, type Scope //deprecated #52463
pkg go/build/constraint, func GoVersion(Expr) string #59033
pkg go/build, type Directive struct #56986
pkg go/build, type Directive struct, Pos token.Position #56986

View File

@@ -1,135 +0,0 @@
pkg archive/tar, method (*Writer) AddFS(fs.FS) error #58000
pkg archive/zip, method (*Writer) AddFS(fs.FS) error #54898
pkg cmp, func Or[$0 comparable](...$0) $0 #60204
pkg crypto/x509, func OIDFromInts([]uint64) (OID, error) #60665
pkg crypto/x509, method (*CertPool) AddCertWithConstraint(*Certificate, func([]*Certificate) error) #57178
pkg crypto/x509, method (OID) Equal(OID) bool #60665
pkg crypto/x509, method (OID) EqualASN1OID(asn1.ObjectIdentifier) bool #60665
pkg crypto/x509, method (OID) String() string #60665
pkg crypto/x509, type Certificate struct, Policies []OID #60665
pkg crypto/x509, type OID struct #60665
pkg database/sql, method (*Null[$0]) Scan(interface{}) error #60370
pkg database/sql, method (Null[$0]) Value() (driver.Value, error) #60370
pkg database/sql, type Null[$0 interface{}] struct #60370
pkg database/sql, type Null[$0 interface{}] struct, V $0 #60370
pkg database/sql, type Null[$0 interface{}] struct, Valid bool #60370
pkg debug/elf, const R_LARCH_64_PCREL = 109 #63725
pkg debug/elf, const R_LARCH_64_PCREL R_LARCH #63725
pkg debug/elf, const R_LARCH_ADD6 = 105 #63725
pkg debug/elf, const R_LARCH_ADD6 R_LARCH #63725
pkg debug/elf, const R_LARCH_ADD_ULEB128 = 107 #63725
pkg debug/elf, const R_LARCH_ADD_ULEB128 R_LARCH #63725
pkg debug/elf, const R_LARCH_ALIGN = 102 #63725
pkg debug/elf, const R_LARCH_ALIGN R_LARCH #63725
pkg debug/elf, const R_LARCH_CFA = 104 #63725
pkg debug/elf, const R_LARCH_CFA R_LARCH #63725
pkg debug/elf, const R_LARCH_DELETE = 101 #63725
pkg debug/elf, const R_LARCH_DELETE R_LARCH #63725
pkg debug/elf, const R_LARCH_PCREL20_S2 = 103 #63725
pkg debug/elf, const R_LARCH_PCREL20_S2 R_LARCH #63725
pkg debug/elf, const R_LARCH_SUB6 = 106 #63725
pkg debug/elf, const R_LARCH_SUB6 R_LARCH #63725
pkg debug/elf, const R_LARCH_SUB_ULEB128 = 108 #63725
pkg debug/elf, const R_LARCH_SUB_ULEB128 R_LARCH #63725
pkg debug/elf, const R_MIPS_PC32 = 248 #61974
pkg debug/elf, const R_MIPS_PC32 R_MIPS #61974
pkg encoding/base32, method (*Encoding) AppendDecode([]uint8, []uint8) ([]uint8, error) #53693
pkg encoding/base32, method (*Encoding) AppendEncode([]uint8, []uint8) []uint8 #53693
pkg encoding/base64, method (*Encoding) AppendDecode([]uint8, []uint8) ([]uint8, error) #53693
pkg encoding/base64, method (*Encoding) AppendEncode([]uint8, []uint8) []uint8 #53693
pkg encoding/hex, func AppendDecode([]uint8, []uint8) ([]uint8, error) #53693
pkg encoding/hex, func AppendEncode([]uint8, []uint8) []uint8 #53693
pkg go/ast, func NewPackage //deprecated #52463
pkg go/ast, func Unparen(Expr) Expr #60061
pkg go/ast, type Importer //deprecated #52463
pkg go/ast, type Object //deprecated #52463
pkg go/ast, type Package //deprecated #52463
pkg go/ast, type Scope //deprecated #52463
pkg go/types, func NewAlias(*TypeName, Type) *Alias #63223
pkg go/types, func Unalias(Type) Type #63223
pkg go/types, method (*Alias) Obj() *TypeName #63223
pkg go/types, method (*Alias) String() string #63223
pkg go/types, method (*Alias) Underlying() Type #63223
pkg go/types, method (*Info) PkgNameOf(*ast.ImportSpec) *PkgName #62037
pkg go/types, method (Checker) PkgNameOf(*ast.ImportSpec) *PkgName #62037
pkg go/types, type Alias struct #63223
pkg go/types, type Info struct, FileVersions map[*ast.File]string #62605
pkg go/version, func Compare(string, string) int #62039
pkg go/version, func IsValid(string) bool #62039
pkg go/version, func Lang(string) string #62039
pkg html/template, const ErrJSTemplate //deprecated #61619
pkg io, method (*SectionReader) Outer() (ReaderAt, int64, int64) #61870
pkg log/slog, func SetLogLoggerLevel(Level) Level #62418
pkg math/big, method (*Rat) FloatPrec() (int, bool) #50489
pkg math/rand/v2, func ExpFloat64() float64 #61716
pkg math/rand/v2, func Float32() float32 #61716
pkg math/rand/v2, func Float64() float64 #61716
pkg math/rand/v2, func Int() int #61716
pkg math/rand/v2, func Int32() int32 #61716
pkg math/rand/v2, func Int32N(int32) int32 #61716
pkg math/rand/v2, func Int64() int64 #61716
pkg math/rand/v2, func Int64N(int64) int64 #61716
pkg math/rand/v2, func IntN(int) int #61716
pkg math/rand/v2, func N[$0 intType]($0) $0 #61716
pkg math/rand/v2, func New(Source) *Rand #61716
pkg math/rand/v2, func NewChaCha8([32]uint8) *ChaCha8 #61716
pkg math/rand/v2, func NewPCG(uint64, uint64) *PCG #61716
pkg math/rand/v2, func NewZipf(*Rand, float64, float64, uint64) *Zipf #61716
pkg math/rand/v2, func NormFloat64() float64 #61716
pkg math/rand/v2, func Perm(int) []int #61716
pkg math/rand/v2, func Shuffle(int, func(int, int)) #61716
pkg math/rand/v2, func Uint32() uint32 #61716
pkg math/rand/v2, func Uint32N(uint32) uint32 #61716
pkg math/rand/v2, func Uint64() uint64 #61716
pkg math/rand/v2, func Uint64N(uint64) uint64 #61716
pkg math/rand/v2, func UintN(uint) uint #61716
pkg math/rand/v2, method (*ChaCha8) MarshalBinary() ([]uint8, error) #61716
pkg math/rand/v2, method (*ChaCha8) Seed([32]uint8) #61716
pkg math/rand/v2, method (*ChaCha8) Uint64() uint64 #61716
pkg math/rand/v2, method (*ChaCha8) UnmarshalBinary([]uint8) error #61716
pkg math/rand/v2, method (*PCG) MarshalBinary() ([]uint8, error) #61716
pkg math/rand/v2, method (*PCG) Seed(uint64, uint64) #61716
pkg math/rand/v2, method (*PCG) Uint64() uint64 #61716
pkg math/rand/v2, method (*PCG) UnmarshalBinary([]uint8) error #61716
pkg math/rand/v2, method (*Rand) ExpFloat64() float64 #61716
pkg math/rand/v2, method (*Rand) Float32() float32 #61716
pkg math/rand/v2, method (*Rand) Float64() float64 #61716
pkg math/rand/v2, method (*Rand) Int() int #61716
pkg math/rand/v2, method (*Rand) Int32() int32 #61716
pkg math/rand/v2, method (*Rand) Int32N(int32) int32 #61716
pkg math/rand/v2, method (*Rand) Int64() int64 #61716
pkg math/rand/v2, method (*Rand) Int64N(int64) int64 #61716
pkg math/rand/v2, method (*Rand) IntN(int) int #61716
pkg math/rand/v2, method (*Rand) NormFloat64() float64 #61716
pkg math/rand/v2, method (*Rand) Perm(int) []int #61716
pkg math/rand/v2, method (*Rand) Shuffle(int, func(int, int)) #61716
pkg math/rand/v2, method (*Rand) Uint32() uint32 #61716
pkg math/rand/v2, method (*Rand) Uint32N(uint32) uint32 #61716
pkg math/rand/v2, method (*Rand) Uint64() uint64 #61716
pkg math/rand/v2, method (*Rand) Uint64N(uint64) uint64 #61716
pkg math/rand/v2, method (*Rand) UintN(uint) uint #61716
pkg math/rand/v2, method (*Zipf) Uint64() uint64 #61716
pkg math/rand/v2, type ChaCha8 struct #61716
pkg math/rand/v2, type PCG struct #61716
pkg math/rand/v2, type Rand struct #61716
pkg math/rand/v2, type Source interface { Uint64 } #61716
pkg math/rand/v2, type Source interface, Uint64() uint64 #61716
pkg math/rand/v2, type Zipf struct #61716
pkg net, method (*TCPConn) WriteTo(io.Writer) (int64, error) #58808
pkg net/http, func FileServerFS(fs.FS) Handler #51971
pkg net/http, func NewFileTransportFS(fs.FS) RoundTripper #51971
pkg net/http, func ServeFileFS(ResponseWriter, *Request, fs.FS, string) #51971
pkg net/http, method (*Request) PathValue(string) string #61410
pkg net/http, method (*Request) SetPathValue(string, string) #61410
pkg net/netip, method (AddrPort) Compare(AddrPort) int #61642
pkg os, method (*File) WriteTo(io.Writer) (int64, error) #58808
pkg reflect, func PtrTo //deprecated #59599
pkg reflect, func TypeFor[$0 interface{}]() Type #60088
pkg slices, func Concat[$0 interface{ ~[]$1 }, $1 interface{}](...$0) $0 #56353
pkg syscall (linux-386), type SysProcAttr struct, PidFD *int #51246
pkg syscall (linux-386-cgo), type SysProcAttr struct, PidFD *int #51246
pkg syscall (linux-amd64), type SysProcAttr struct, PidFD *int #51246
pkg syscall (linux-amd64-cgo), type SysProcAttr struct, PidFD *int #51246
pkg syscall (linux-arm), type SysProcAttr struct, PidFD *int #51246
pkg syscall (linux-arm-cgo), type SysProcAttr struct, PidFD *int #51246
pkg testing/slogtest, func Run(*testing.T, func(*testing.T) slog.Handler, func(*testing.T) map[string]interface{}) #61758

View File

@@ -1,158 +0,0 @@
pkg archive/tar, type FileInfoNames interface { Gname, IsDir, ModTime, Mode, Name, Size, Sys, Uname } #50102
pkg archive/tar, type FileInfoNames interface, Gname() (string, error) #50102
pkg archive/tar, type FileInfoNames interface, IsDir() bool #50102
pkg archive/tar, type FileInfoNames interface, ModTime() time.Time #50102
pkg archive/tar, type FileInfoNames interface, Mode() fs.FileMode #50102
pkg archive/tar, type FileInfoNames interface, Name() string #50102
pkg archive/tar, type FileInfoNames interface, Size() int64 #50102
pkg archive/tar, type FileInfoNames interface, Sys() interface{} #50102
pkg archive/tar, type FileInfoNames interface, Uname() (string, error) #50102
pkg crypto/tls, const QUICResumeSession = 8 #63691
pkg crypto/tls, const QUICResumeSession QUICEventKind #63691
pkg crypto/tls, const QUICStoreSession = 9 #63691
pkg crypto/tls, const QUICStoreSession QUICEventKind #63691
pkg crypto/tls, method (*ECHRejectionError) Error() string #63369
pkg crypto/tls, method (*QUICConn) StoreSession(*SessionState) error #63691
pkg crypto/tls, type Config struct, EncryptedClientHelloConfigList []uint8 #63369
pkg crypto/tls, type Config struct, EncryptedClientHelloRejectionVerify func(ConnectionState) error #63369
pkg crypto/tls, type ConnectionState struct, ECHAccepted bool #63369
pkg crypto/tls, type ECHRejectionError struct #63369
pkg crypto/tls, type ECHRejectionError struct, RetryConfigList []uint8 #63369
pkg crypto/tls, type QUICConfig struct, EnableSessionEvents bool #63691
pkg crypto/tls, type QUICEvent struct, SessionState *SessionState #63691
pkg crypto/tls, type QUICSessionTicketOptions struct, Extra [][]uint8 #63691
pkg crypto/x509, func ParseOID(string) (OID, error) #66249
pkg crypto/x509, method (*OID) UnmarshalBinary([]uint8) error #66249
pkg crypto/x509, method (*OID) UnmarshalText([]uint8) error #66249
pkg crypto/x509, method (OID) MarshalBinary() ([]uint8, error) #66249
pkg crypto/x509, method (OID) MarshalText() ([]uint8, error) #66249
pkg debug/elf, const PT_OPENBSD_NOBTCFI = 1705237480 #66054
pkg debug/elf, const PT_OPENBSD_NOBTCFI ProgType #66054
pkg debug/elf, const STT_GNU_IFUNC = 10 #66836
pkg debug/elf, const STT_GNU_IFUNC SymType #66836
pkg debug/elf, const STT_RELC = 8 #66836
pkg debug/elf, const STT_RELC SymType #66836
pkg debug/elf, const STT_SRELC = 9 #66836
pkg debug/elf, const STT_SRELC SymType #66836
pkg encoding/binary, func Append([]uint8, ByteOrder, interface{}) ([]uint8, error) #60023
pkg encoding/binary, func Decode([]uint8, ByteOrder, interface{}) (int, error) #60023
pkg encoding/binary, func Encode([]uint8, ByteOrder, interface{}) (int, error) #60023
pkg go/ast, func Preorder(Node) iter.Seq[Node] #66339
pkg go/types, method (*Alias) Origin() *Alias #67143
pkg go/types, method (*Alias) Rhs() Type #66559
pkg go/types, method (*Alias) SetTypeParams([]*TypeParam) #67143
pkg go/types, method (*Alias) TypeArgs() *TypeList #67143
pkg go/types, method (*Alias) TypeParams() *TypeParamList #67143
pkg go/types, method (*Func) Signature() *Signature #65772
pkg iter, func Pull2[$0 interface{}, $1 interface{}](Seq2[$0, $1]) (func() ($0, $1, bool), func()) #61897
pkg iter, func Pull[$0 interface{}](Seq[$0]) (func() ($0, bool), func()) #61897
pkg iter, type Seq2[$0 interface{}, $1 interface{}] func(func($0, $1) bool) #61897
pkg iter, type Seq[$0 interface{}] func(func($0) bool) #61897
pkg maps, func All[$0 interface{ ~map[$1]$2 }, $1 comparable, $2 interface{}]($0) iter.Seq2[$1, $2] #61900
pkg maps, func Collect[$0 comparable, $1 interface{}](iter.Seq2[$0, $1]) map[$0]$1 #61900
pkg maps, func Insert[$0 interface{ ~map[$1]$2 }, $1 comparable, $2 interface{}]($0, iter.Seq2[$1, $2]) #61900
pkg maps, func Keys[$0 interface{ ~map[$1]$2 }, $1 comparable, $2 interface{}]($0) iter.Seq[$1] #61900
pkg maps, func Values[$0 interface{ ~map[$1]$2 }, $1 comparable, $2 interface{}]($0) iter.Seq[$2] #61900
pkg math/rand/v2, func Uint() uint #61716
pkg math/rand/v2, method (*ChaCha8) Read([]uint8) (int, error) #67059
pkg math/rand/v2, method (*Rand) Uint() uint #61716
pkg net, method (*DNSError) Unwrap() error #63116
pkg net, method (*TCPConn) SetKeepAliveConfig(KeepAliveConfig) error #62254
pkg net, type DNSError struct, UnwrapErr error #63116
pkg net, type Dialer struct, KeepAliveConfig KeepAliveConfig #62254
pkg net, type KeepAliveConfig struct #62254
pkg net, type KeepAliveConfig struct, Count int #62254
pkg net, type KeepAliveConfig struct, Enable bool #62254
pkg net, type KeepAliveConfig struct, Idle time.Duration #62254
pkg net, type KeepAliveConfig struct, Interval time.Duration #62254
pkg net, type ListenConfig struct, KeepAliveConfig KeepAliveConfig #62254
pkg net/http, func ParseCookie(string) ([]*Cookie, error) #66008
pkg net/http, func ParseSetCookie(string) (*Cookie, error) #66008
pkg net/http, method (*Request) CookiesNamed(string) []*Cookie #61472
pkg net/http, type Cookie struct, Partitioned bool #62490
pkg net/http, type Cookie struct, Quoted bool #46443
pkg net/http, type Request struct, Pattern string #66405
pkg net/http/httptest, func NewRequestWithContext(context.Context, string, string, io.Reader) *http.Request #59473
pkg os, func CopyFS(string, fs.FS) error #62484
pkg path/filepath, func Localize(string) (string, error) #57151
pkg reflect, func SliceAt(Type, unsafe.Pointer, int) Value #61308
pkg reflect, method (Value) Seq() iter.Seq[Value] #66056
pkg reflect, method (Value) Seq2() iter.Seq2[Value, Value] #66056
pkg reflect, type Type interface, CanSeq() bool #66056
pkg reflect, type Type interface, CanSeq2() bool #66056
pkg reflect, type Type interface, OverflowComplex(complex128) bool #60427
pkg reflect, type Type interface, OverflowFloat(float64) bool #60427
pkg reflect, type Type interface, OverflowInt(int64) bool #60427
pkg reflect, type Type interface, OverflowUint(uint64) bool #60427
pkg runtime/debug, func SetCrashOutput(*os.File, CrashOptions) error #42888
pkg runtime/debug, type CrashOptions struct #67182
pkg slices, func All[$0 interface{ ~[]$1 }, $1 interface{}]($0) iter.Seq2[int, $1] #61899
pkg slices, func AppendSeq[$0 interface{ ~[]$1 }, $1 interface{}]($0, iter.Seq[$1]) $0 #61899
pkg slices, func Backward[$0 interface{ ~[]$1 }, $1 interface{}]($0) iter.Seq2[int, $1] #61899
pkg slices, func Chunk[$0 interface{ ~[]$1 }, $1 interface{}]($0, int) iter.Seq[$0] #53987
pkg slices, func Collect[$0 interface{}](iter.Seq[$0]) []$0 #61899
pkg slices, func Repeat[$0 interface{ ~[]$1 }, $1 interface{}]($0, int) $0 #65238
pkg slices, func SortedFunc[$0 interface{}](iter.Seq[$0], func($0, $0) int) []$0 #61899
pkg slices, func SortedStableFunc[$0 interface{}](iter.Seq[$0], func($0, $0) int) []$0 #61899
pkg slices, func Sorted[$0 cmp.Ordered](iter.Seq[$0]) []$0 #61899
pkg slices, func Values[$0 interface{ ~[]$1 }, $1 interface{}]($0) iter.Seq[$1] #61899
pkg structs, type HostLayout struct #66408
pkg sync, method (*Map) Clear() #61696
pkg sync/atomic, func AndInt32(*int32, int32) int32 #61395
pkg sync/atomic, func AndInt64(*int64, int64) int64 #61395
pkg sync/atomic, func AndUint32(*uint32, uint32) uint32 #61395
pkg sync/atomic, func AndUint64(*uint64, uint64) uint64 #61395
pkg sync/atomic, func AndUintptr(*uintptr, uintptr) uintptr #61395
pkg sync/atomic, func OrInt32(*int32, int32) int32 #61395
pkg sync/atomic, func OrInt64(*int64, int64) int64 #61395
pkg sync/atomic, func OrUint32(*uint32, uint32) uint32 #61395
pkg sync/atomic, func OrUint64(*uint64, uint64) uint64 #61395
pkg sync/atomic, func OrUintptr(*uintptr, uintptr) uintptr #61395
pkg sync/atomic, method (*Int32) And(int32) int32 #61395
pkg sync/atomic, method (*Int32) Or(int32) int32 #61395
pkg sync/atomic, method (*Int64) And(int64) int64 #61395
pkg sync/atomic, method (*Int64) Or(int64) int64 #61395
pkg sync/atomic, method (*Uint32) And(uint32) uint32 #61395
pkg sync/atomic, method (*Uint32) Or(uint32) uint32 #61395
pkg sync/atomic, method (*Uint64) And(uint64) uint64 #61395
pkg sync/atomic, method (*Uint64) Or(uint64) uint64 #61395
pkg sync/atomic, method (*Uintptr) And(uintptr) uintptr #61395
pkg sync/atomic, method (*Uintptr) Or(uintptr) uintptr #61395
pkg syscall (openbsd-386), const EBADMSG = 92 #67998
pkg syscall (openbsd-386), const ELAST = 95 #67998
pkg syscall (openbsd-386), const ENOTRECOVERABLE = 93 #67998
pkg syscall (openbsd-386), const ENOTRECOVERABLE Errno #67998
pkg syscall (openbsd-386), const EOWNERDEAD = 94 #67998
pkg syscall (openbsd-386), const EOWNERDEAD Errno #67998
pkg syscall (openbsd-386), const EPROTO = 95 #67998
pkg syscall (openbsd-386-cgo), const EBADMSG = 92 #67998
pkg syscall (openbsd-386-cgo), const ELAST = 95 #67998
pkg syscall (openbsd-386-cgo), const ENOTRECOVERABLE = 93 #67998
pkg syscall (openbsd-386-cgo), const ENOTRECOVERABLE Errno #67998
pkg syscall (openbsd-386-cgo), const EOWNERDEAD = 94 #67998
pkg syscall (openbsd-386-cgo), const EOWNERDEAD Errno #67998
pkg syscall (openbsd-386-cgo), const EPROTO = 95 #67998
pkg syscall (openbsd-amd64), const EBADMSG = 92 #67998
pkg syscall (openbsd-amd64), const ELAST = 95 #67998
pkg syscall (openbsd-amd64), const ENOTRECOVERABLE = 93 #67998
pkg syscall (openbsd-amd64), const ENOTRECOVERABLE Errno #67998
pkg syscall (openbsd-amd64), const EOWNERDEAD = 94 #67998
pkg syscall (openbsd-amd64), const EOWNERDEAD Errno #67998
pkg syscall (openbsd-amd64), const EPROTO = 95 #67998
pkg syscall (openbsd-amd64-cgo), const EBADMSG = 92 #67998
pkg syscall (openbsd-amd64-cgo), const ELAST = 95 #67998
pkg syscall (openbsd-amd64-cgo), const ENOTRECOVERABLE = 93 #67998
pkg syscall (openbsd-amd64-cgo), const ENOTRECOVERABLE Errno #67998
pkg syscall (openbsd-amd64-cgo), const EOWNERDEAD = 94 #67998
pkg syscall (openbsd-amd64-cgo), const EOWNERDEAD Errno #67998
pkg syscall (openbsd-amd64-cgo), const EPROTO = 95 #67998
pkg syscall (windows-386), const WSAENOPROTOOPT = 10042 #62254
pkg syscall (windows-386), const WSAENOPROTOOPT Errno #62254
pkg syscall (windows-amd64), const WSAENOPROTOOPT = 10042 #62254
pkg syscall (windows-amd64), const WSAENOPROTOOPT Errno #62254
pkg syscall, const EBADMSG Errno #67998
pkg syscall, const EPROTO Errno #67998
pkg unicode/utf16, func RuneLen(int32) int #44940
pkg unique, func Make[$0 comparable]($0) Handle[$0] #62483
pkg unique, method (Handle[$0]) Value() $0 #62483
pkg unique, type Handle[$0 comparable] struct #62483

View File

@@ -1,230 +0,0 @@
pkg bytes, func FieldsFuncSeq([]uint8, func(int32) bool) iter.Seq[[]uint8] #61901
pkg bytes, func FieldsSeq([]uint8) iter.Seq[[]uint8] #61901
pkg bytes, func Lines([]uint8) iter.Seq[[]uint8] #61901
pkg bytes, func SplitAfterSeq([]uint8, []uint8) iter.Seq[[]uint8] #61901
pkg bytes, func SplitSeq([]uint8, []uint8) iter.Seq[[]uint8] #61901
pkg crypto/cipher, func NewCFBDecrypter //deprecated #69445
pkg crypto/cipher, func NewCFBEncrypter //deprecated #69445
pkg crypto/cipher, func NewGCMWithRandomNonce(Block) (AEAD, error) #69981
pkg crypto/cipher, func NewOFB //deprecated #69445
pkg crypto/fips140, func Enabled() bool #70123
pkg crypto/hkdf, func Expand[$0 hash.Hash](func() $0, []uint8, string, int) ([]uint8, error) #61477
pkg crypto/hkdf, func Extract[$0 hash.Hash](func() $0, []uint8, []uint8) ([]uint8, error) #61477
pkg crypto/hkdf, func Key[$0 hash.Hash](func() $0, []uint8, []uint8, string, int) ([]uint8, error) #61477
pkg crypto/mlkem, const CiphertextSize1024 = 1568 #70122
pkg crypto/mlkem, const CiphertextSize1024 ideal-int #70122
pkg crypto/mlkem, const CiphertextSize768 = 1088 #70122
pkg crypto/mlkem, const CiphertextSize768 ideal-int #70122
pkg crypto/mlkem, const EncapsulationKeySize1024 = 1568 #70122
pkg crypto/mlkem, const EncapsulationKeySize1024 ideal-int #70122
pkg crypto/mlkem, const EncapsulationKeySize768 = 1184 #70122
pkg crypto/mlkem, const EncapsulationKeySize768 ideal-int #70122
pkg crypto/mlkem, const SeedSize = 64 #70122
pkg crypto/mlkem, const SeedSize ideal-int #70122
pkg crypto/mlkem, const SharedKeySize = 32 #70122
pkg crypto/mlkem, const SharedKeySize ideal-int #70122
pkg crypto/mlkem, func GenerateKey1024() (*DecapsulationKey1024, error) #70122
pkg crypto/mlkem, func GenerateKey768() (*DecapsulationKey768, error) #70122
pkg crypto/mlkem, func NewDecapsulationKey1024([]uint8) (*DecapsulationKey1024, error) #70122
pkg crypto/mlkem, func NewDecapsulationKey768([]uint8) (*DecapsulationKey768, error) #70122
pkg crypto/mlkem, func NewEncapsulationKey1024([]uint8) (*EncapsulationKey1024, error) #70122
pkg crypto/mlkem, func NewEncapsulationKey768([]uint8) (*EncapsulationKey768, error) #70122
pkg crypto/mlkem, method (*DecapsulationKey1024) Bytes() []uint8 #70122
pkg crypto/mlkem, method (*DecapsulationKey1024) Decapsulate([]uint8) ([]uint8, error) #70122
pkg crypto/mlkem, method (*DecapsulationKey1024) EncapsulationKey() *EncapsulationKey1024 #70122
pkg crypto/mlkem, method (*DecapsulationKey768) Bytes() []uint8 #70122
pkg crypto/mlkem, method (*DecapsulationKey768) Decapsulate([]uint8) ([]uint8, error) #70122
pkg crypto/mlkem, method (*DecapsulationKey768) EncapsulationKey() *EncapsulationKey768 #70122
pkg crypto/mlkem, method (*EncapsulationKey1024) Bytes() []uint8 #70122
pkg crypto/mlkem, method (*EncapsulationKey1024) Encapsulate() ([]uint8, []uint8) #70122
pkg crypto/mlkem, method (*EncapsulationKey768) Bytes() []uint8 #70122
pkg crypto/mlkem, method (*EncapsulationKey768) Encapsulate() ([]uint8, []uint8) #70122
pkg crypto/mlkem, type DecapsulationKey1024 struct #70122
pkg crypto/mlkem, type DecapsulationKey768 struct #70122
pkg crypto/mlkem, type EncapsulationKey1024 struct #70122
pkg crypto/mlkem, type EncapsulationKey768 struct #70122
pkg crypto/pbkdf2, func Key[$0 hash.Hash](func() $0, string, []uint8, int, int) ([]uint8, error) #69488
pkg crypto/rand, func Text() string #67057
pkg crypto/sha3, func New224() *SHA3 #69982
pkg crypto/sha3, func New256() *SHA3 #69982
pkg crypto/sha3, func New384() *SHA3 #69982
pkg crypto/sha3, func New512() *SHA3 #69982
pkg crypto/sha3, func NewCSHAKE128([]uint8, []uint8) *SHAKE #69982
pkg crypto/sha3, func NewCSHAKE256([]uint8, []uint8) *SHAKE #69982
pkg crypto/sha3, func NewSHAKE128() *SHAKE #69982
pkg crypto/sha3, func NewSHAKE256() *SHAKE #69982
pkg crypto/sha3, func Sum224([]uint8) [28]uint8 #69982
pkg crypto/sha3, func Sum256([]uint8) [32]uint8 #69982
pkg crypto/sha3, func Sum384([]uint8) [48]uint8 #69982
pkg crypto/sha3, func Sum512([]uint8) [64]uint8 #69982
pkg crypto/sha3, func SumSHAKE128([]uint8, int) []uint8 #69982
pkg crypto/sha3, func SumSHAKE256([]uint8, int) []uint8 #69982
pkg crypto/sha3, method (*SHA3) AppendBinary([]uint8) ([]uint8, error) #69982
pkg crypto/sha3, method (*SHA3) BlockSize() int #69982
pkg crypto/sha3, method (*SHA3) MarshalBinary() ([]uint8, error) #69982
pkg crypto/sha3, method (*SHA3) Reset() #69982
pkg crypto/sha3, method (*SHA3) Size() int #69982
pkg crypto/sha3, method (*SHA3) Sum([]uint8) []uint8 #69982
pkg crypto/sha3, method (*SHA3) UnmarshalBinary([]uint8) error #69982
pkg crypto/sha3, method (*SHA3) Write([]uint8) (int, error) #69982
pkg crypto/sha3, method (*SHAKE) AppendBinary([]uint8) ([]uint8, error) #69982
pkg crypto/sha3, method (*SHAKE) BlockSize() int #69982
pkg crypto/sha3, method (*SHAKE) MarshalBinary() ([]uint8, error) #69982
pkg crypto/sha3, method (*SHAKE) Read([]uint8) (int, error) #69982
pkg crypto/sha3, method (*SHAKE) Reset() #69982
pkg crypto/sha3, method (*SHAKE) UnmarshalBinary([]uint8) error #69982
pkg crypto/sha3, method (*SHAKE) Write([]uint8) (int, error) #69982
pkg crypto/sha3, type SHA3 struct #69982
pkg crypto/sha3, type SHAKE struct #69982
pkg crypto/subtle, func WithDataIndependentTiming(func()) #66450
pkg crypto/tls, const X25519MLKEM768 = 4588 #69985
pkg crypto/tls, const X25519MLKEM768 CurveID #69985
pkg crypto/tls, type ClientHelloInfo struct, Extensions []uint16 #32936
pkg crypto/tls, type Config struct, EncryptedClientHelloKeys []EncryptedClientHelloKey #68500
pkg crypto/tls, type EncryptedClientHelloKey struct #68500
pkg crypto/tls, type EncryptedClientHelloKey struct, Config []uint8 #68500
pkg crypto/tls, type EncryptedClientHelloKey struct, PrivateKey []uint8 #68500
pkg crypto/tls, type EncryptedClientHelloKey struct, SendAsRetry bool #68500
pkg crypto/x509, const NoValidChains = 10 #68484
pkg crypto/x509, const NoValidChains InvalidReason #68484
pkg crypto/x509, method (OID) AppendBinary([]uint8) ([]uint8, error) #62384
pkg crypto/x509, method (OID) AppendText([]uint8) ([]uint8, error) #62384
pkg crypto/x509, type Certificate struct, InhibitAnyPolicy int #68484
pkg crypto/x509, type Certificate struct, InhibitAnyPolicyZero bool #68484
pkg crypto/x509, type Certificate struct, InhibitPolicyMapping int #68484
pkg crypto/x509, type Certificate struct, InhibitPolicyMappingZero bool #68484
pkg crypto/x509, type Certificate struct, PolicyMappings []PolicyMapping #68484
pkg crypto/x509, type Certificate struct, RequireExplicitPolicy int #68484
pkg crypto/x509, type Certificate struct, RequireExplicitPolicyZero bool #68484
pkg crypto/x509, type PolicyMapping struct #68484
pkg crypto/x509, type PolicyMapping struct, IssuerDomainPolicy OID #68484
pkg crypto/x509, type PolicyMapping struct, SubjectDomainPolicy OID #68484
pkg crypto/x509, type VerifyOptions struct, CertificatePolicies []OID #68484
pkg debug/elf, const VER_FLG_BASE = 1 #63952
pkg debug/elf, const VER_FLG_BASE DynamicVersionFlag #63952
pkg debug/elf, const VER_FLG_INFO = 4 #63952
pkg debug/elf, const VER_FLG_INFO DynamicVersionFlag #63952
pkg debug/elf, const VER_FLG_WEAK = 2 #63952
pkg debug/elf, const VER_FLG_WEAK DynamicVersionFlag #63952
pkg debug/elf, const VerFlagGlobal = 2 #63952
pkg debug/elf, const VerFlagGlobal SymbolVersionFlag #63952
pkg debug/elf, const VerFlagHidden = 4 #63952
pkg debug/elf, const VerFlagHidden SymbolVersionFlag #63952
pkg debug/elf, const VerFlagLocal = 1 #63952
pkg debug/elf, const VerFlagLocal SymbolVersionFlag #63952
pkg debug/elf, const VerFlagNone = 0 #63952
pkg debug/elf, const VerFlagNone SymbolVersionFlag #63952
pkg debug/elf, method (*File) DynamicVersionNeeds() ([]DynamicVersionNeed, error) #63952
pkg debug/elf, method (*File) DynamicVersions() ([]DynamicVersion, error) #63952
pkg debug/elf, type DynamicVersion struct #63952
pkg debug/elf, type DynamicVersion struct, Deps []string #63952
pkg debug/elf, type DynamicVersion struct, Flags DynamicVersionFlag #63952
pkg debug/elf, type DynamicVersion struct, Index uint16 #63952
pkg debug/elf, type DynamicVersion struct, Version uint16 #63952
pkg debug/elf, type DynamicVersionDep struct #63952
pkg debug/elf, type DynamicVersionDep struct, Dep string #63952
pkg debug/elf, type DynamicVersionDep struct, Flags DynamicVersionFlag #63952
pkg debug/elf, type DynamicVersionDep struct, Other uint16 #63952
pkg debug/elf, type DynamicVersionFlag uint16 #63952
pkg debug/elf, type DynamicVersionNeed struct #63952
pkg debug/elf, type DynamicVersionNeed struct, Name string #63952
pkg debug/elf, type DynamicVersionNeed struct, Needs []DynamicVersionDep #63952
pkg debug/elf, type DynamicVersionNeed struct, Version uint16 #63952
pkg debug/elf, type Symbol struct, VersionFlags SymbolVersionFlag #63952
pkg debug/elf, type Symbol struct, VersionIndex int16 #63952
pkg debug/elf, type SymbolVersionFlag uint8 #63952
pkg encoding, type BinaryAppender interface { AppendBinary } #62384
pkg encoding, type BinaryAppender interface, AppendBinary([]uint8) ([]uint8, error) #62384
pkg encoding, type TextAppender interface { AppendText } #62384
pkg encoding, type TextAppender interface, AppendText([]uint8) ([]uint8, error) #62384
pkg go/types, method (*Interface) EmbeddedTypes() iter.Seq[Type] #66626
pkg go/types, method (*Interface) ExplicitMethods() iter.Seq[*Func] #66626
pkg go/types, method (*Interface) Methods() iter.Seq[*Func] #66626
pkg go/types, method (*MethodSet) Methods() iter.Seq[*Selection] #66626
pkg go/types, method (*Named) Methods() iter.Seq[*Func] #66626
pkg go/types, method (*Scope) Children() iter.Seq[*Scope] #66626
pkg go/types, method (*Struct) Fields() iter.Seq[*Var] #66626
pkg go/types, method (*Tuple) Variables() iter.Seq[*Var] #66626
pkg go/types, method (*TypeList) Types() iter.Seq[Type] #66626
pkg go/types, method (*TypeParamList) TypeParams() iter.Seq[*TypeParam] #66626
pkg go/types, method (*Union) Terms() iter.Seq[*Term] #66626
pkg hash/maphash, func Comparable[$0 comparable](Seed, $0) uint64 #54670
pkg hash/maphash, func WriteComparable[$0 comparable](*Hash, $0) #54670
pkg log/slog, method (*LevelVar) AppendText([]uint8) ([]uint8, error) #62384
pkg log/slog, method (Level) AppendText([]uint8) ([]uint8, error) #62384
pkg log/slog, var DiscardHandler Handler #62005
pkg math/big, method (*Float) AppendText([]uint8) ([]uint8, error) #62384
pkg math/big, method (*Int) AppendText([]uint8) ([]uint8, error) #62384
pkg math/big, method (*Rat) AppendText([]uint8) ([]uint8, error) #62384
pkg math/rand/v2, method (*ChaCha8) AppendBinary([]uint8) ([]uint8, error) #62384
pkg math/rand/v2, method (*PCG) AppendBinary([]uint8) ([]uint8, error) #62384
pkg net, method (IP) AppendText([]uint8) ([]uint8, error) #62384
pkg net/http, method (*Protocols) SetHTTP1(bool) #67814
pkg net/http, method (*Protocols) SetHTTP2(bool) #67814
pkg net/http, method (*Protocols) SetUnencryptedHTTP2(bool) #67816
pkg net/http, method (Protocols) HTTP1() bool #67814
pkg net/http, method (Protocols) HTTP2() bool #67814
pkg net/http, method (Protocols) String() string #67814
pkg net/http, method (Protocols) UnencryptedHTTP2() bool #67816
pkg net/http, type HTTP2Config struct #67813
pkg net/http, type HTTP2Config struct, CountError func(string) #67813
pkg net/http, type HTTP2Config struct, MaxConcurrentStreams int #67813
pkg net/http, type HTTP2Config struct, MaxDecoderHeaderTableSize int #67813
pkg net/http, type HTTP2Config struct, MaxEncoderHeaderTableSize int #67813
pkg net/http, type HTTP2Config struct, MaxReadFrameSize int #67813
pkg net/http, type HTTP2Config struct, MaxReceiveBufferPerConnection int #67813
pkg net/http, type HTTP2Config struct, MaxReceiveBufferPerStream int #67813
pkg net/http, type HTTP2Config struct, PermitProhibitedCipherSuites bool #67813
pkg net/http, type HTTP2Config struct, PingTimeout time.Duration #67813
pkg net/http, type HTTP2Config struct, SendPingTimeout time.Duration #67813
pkg net/http, type HTTP2Config struct, WriteByteTimeout time.Duration #67813
pkg net/http, type Protocols struct #67814
pkg net/http, type Server struct, HTTP2 *HTTP2Config #67813
pkg net/http, type Server struct, Protocols *Protocols #67814
pkg net/http, type Transport struct, HTTP2 *HTTP2Config #67813
pkg net/http, type Transport struct, Protocols *Protocols #67814
pkg net/netip, method (Addr) AppendBinary([]uint8) ([]uint8, error) #62384
pkg net/netip, method (Addr) AppendText([]uint8) ([]uint8, error) #62384
pkg net/netip, method (AddrPort) AppendBinary([]uint8) ([]uint8, error) #62384
pkg net/netip, method (AddrPort) AppendText([]uint8) ([]uint8, error) #62384
pkg net/netip, method (Prefix) AppendBinary([]uint8) ([]uint8, error) #62384
pkg net/netip, method (Prefix) AppendText([]uint8) ([]uint8, error) #62384
pkg net/url, method (*URL) AppendBinary([]uint8) ([]uint8, error) #62384
pkg os, func OpenInRoot(string, string) (*File, error) #67002
pkg os, func OpenRoot(string) (*Root, error) #67002
pkg os, method (*Root) Close() error #67002
pkg os, method (*Root) Create(string) (*File, error) #67002
pkg os, method (*Root) FS() fs.FS #67002
pkg os, method (*Root) Lstat(string) (fs.FileInfo, error) #67002
pkg os, method (*Root) Mkdir(string, fs.FileMode) error #67002
pkg os, method (*Root) Name() string #67002
pkg os, method (*Root) Open(string) (*File, error) #67002
pkg os, method (*Root) OpenFile(string, int, fs.FileMode) (*File, error) #67002
pkg os, method (*Root) OpenRoot(string) (*Root, error) #67002
pkg os, method (*Root) Remove(string) error #67002
pkg os, method (*Root) Stat(string) (fs.FileInfo, error) #67002
pkg os, type Root struct #67002
pkg regexp, method (*Regexp) AppendText([]uint8) ([]uint8, error) #62384
pkg runtime, func AddCleanup[$0 interface{}, $1 interface{}](*$0, func($1), $1) Cleanup #67535
pkg runtime, func GOROOT //deprecated #51473
pkg runtime, method (Cleanup) Stop() #67535
pkg runtime, type Cleanup struct #67535
pkg strings, func FieldsFuncSeq(string, func(int32) bool) iter.Seq[string] #61901
pkg strings, func FieldsSeq(string) iter.Seq[string] #61901
pkg strings, func Lines(string) iter.Seq[string] #61901
pkg strings, func SplitAfterSeq(string, string) iter.Seq[string] #61901
pkg strings, func SplitSeq(string, string) iter.Seq[string] #61901
pkg testing, method (*B) Chdir(string) #62516
pkg testing, method (*B) Context() context.Context #36532
pkg testing, method (*B) Loop() bool #61515
pkg testing, method (*F) Chdir(string) #62516
pkg testing, method (*F) Context() context.Context #36532
pkg testing, method (*T) Chdir(string) #62516
pkg testing, method (*T) Context() context.Context #36532
pkg testing, type TB interface, Chdir(string) #62516
pkg testing, type TB interface, Context() context.Context #36532
pkg time, method (Time) AppendBinary([]uint8) ([]uint8, error) #62384
pkg time, method (Time) AppendText([]uint8) ([]uint8, error) #62384
pkg weak, func Make[$0 interface{}](*$0) Pointer[$0] #67552
pkg weak, method (Pointer[$0]) Value() *$0 #67552
pkg weak, type Pointer[$0 interface{}] struct #67552

View File

@@ -1 +1,2 @@
branch: master
branch: release-branch.go1.21
parent-branch: master

View File

@@ -1,75 +0,0 @@
# Release Notes
The `initial` and `next` subdirectories of this directory are for release notes.
## For developers
Release notes should be added to `next` by editing existing files or creating
new files. **Do not add RELNOTE=yes comments in CLs.** Instead, add a file to
the CL (or ask the author to do so).
At the end of the development cycle, the files will be merged by being
concatenated in sorted order by pathname. Files in the directory matching the
glob "*stdlib/*minor" are treated specially. They should be in subdirectories
corresponding to standard library package paths, and headings for those package
paths will be generated automatically.
Files in this repo's `api/next` directory must have corresponding files in
`doc/next/*stdlib/*minor`.
The files should be in the subdirectory for the package with the new
API, and should be named after the issue number of the API proposal.
For example, if the directory `6-stdlib/99-minor` is present,
then an `api/next` file with the line
pkg net/http, function F #12345
should have a corresponding file named `doc/next/6-stdlib/99-minor/net/http/12345.md`.
At a minimum, that file should contain either a full sentence or a TODO,
ideally referring to a person with the responsibility to complete the note.
If your CL addresses an accepted proposal, mention the proposal issue number in
your release note in the form `/issue/NUMBER`. A link to the issue in the text
will have this form (see below). If you don't want to mention the issue in the
text, add it as a comment:
```
<!-- go.dev/issue/12345 -->
```
If an accepted proposal is mentioned in a CL but not in the release notes, it will be
flagged as a TODO by the automated tooling. That is true even for proposals that add API.
Use the following forms in your markdown:
[http.Request] # symbol documentation; auto-linked as in Go doc strings
[Request] # short form, for symbols in the package being documented
[net/http] # package link
[#12345](/issue/12345) # GitHub issues
[CL 6789](/cl/6789) # Gerrit changelists
To preview `next` content in merged form using a local instance of the website, run:
```
go run golang.org/x/website/cmd/golangorg@latest -goroot=..
```
Then open http://localhost:6060/doc/next. Refresh the page to see your latest edits.
## For the release team
The `relnote` tool, at `golang.org/x/build/cmd/relnote`, operates on the files
in `doc/next`.
As a release cycle nears completion, run `relnote todo` to get a list of
unfinished release note work.
To prepare the release notes for a release, run `relnote generate`.
That will merge the `.md` files in `next` into a single file.
Atomically (as close to it as possible) add that file to `_content/doc` directory
of the website repository and remove the `doc/next` directory in this repository.
To begin the next release development cycle, populate the contents of `next`
with those of `initial`. From the repo root:
> cd doc
> cp -R initial/ next
Then edit `next/1-intro.md` to refer to the next version.

View File

@@ -464,23 +464,6 @@ Function is the outermost frame of the call stack. Traceback should stop at this
</li>
</ul>
<h3 id="special-instructions">Special instructions</h3>
<p>
The <code>PCALIGN</code> pseudo-instruction is used to indicate that the next instruction should be aligned
to a specified boundary by padding with no-op instructions.
</p>
<p>
It is currently supported on arm64, amd64, ppc64, loong64 and riscv64.
For example, the start of the <code>MOVD</code> instruction below is aligned to 32 bytes:
<pre>
PCALIGN $32
MOVD $2, R0
</pre>
</p>
<h3 id="data-offsets">Interacting with Go types and constants</h3>
<p>

View File

@@ -1,17 +1,14 @@
<!--{
"Title": "The Go Programming Language Specification",
"Subtitle": "Language version go1.17 (Oct 15, 2021)",
"Subtitle": "Version of Oct 15, 2021",
"Path": "/ref/spec"
}-->
<h2 id="Introduction">Introduction</h2>
<p>
This is the reference manual for the Go programming language as it was for
language version 1.17, in October 2021, before the introduction of generics.
It is provided for historical interest.
The current reference manual can be found <a href="/doc/go_spec.html">here</a>.
For more information and other documents, see <a href="/">go.dev</a>.
This is a reference manual for the Go programming language. For
more information and other documents, see <a href="/">golang.org</a>.
</p>
<p>
@@ -656,7 +653,7 @@ and are discussed in that section.
<p>
Numeric constants represent exact values of arbitrary precision and do not overflow.
Consequently, there are no constants denoting the IEEE 754 negative zero, infinity,
Consequently, there are no constants denoting the IEEE-754 negative zero, infinity,
and not-a-number values.
</p>
@@ -882,8 +879,8 @@ int16 the set of all signed 16-bit integers (-32768 to 32767)
int32 the set of all signed 32-bit integers (-2147483648 to 2147483647)
int64 the set of all signed 64-bit integers (-9223372036854775808 to 9223372036854775807)
float32 the set of all IEEE 754 32-bit floating-point numbers
float64 the set of all IEEE 754 64-bit floating-point numbers
float32 the set of all IEEE-754 32-bit floating-point numbers
float64 the set of all IEEE-754 64-bit floating-point numbers
complex64 the set of all complex numbers with float32 real and imaginary parts
complex128 the set of all complex numbers with float64 real and imaginary parts
@@ -917,7 +914,7 @@ are required when different numeric types are mixed in an expression
or assignment. For instance, <code>int32</code> and <code>int</code>
are not the same type even though they may have the same size on a
particular architecture.
</p>
<h3 id="String_types">String types</h3>
@@ -1454,7 +1451,6 @@ maps grow to accommodate the number of items
stored in them, with the exception of <code>nil</code> maps.
A <code>nil</code> map is equivalent to an empty map except that no elements
may be added.
</p>
<h3 id="Channel_types">Channel types</h3>
@@ -3645,8 +3641,6 @@ As the <code>++</code> and <code>--</code> operators form
statements, not expressions, they fall
outside the operator hierarchy.
As a consequence, statement <code>*p++</code> is the same as <code>(*p)++</code>.
</p>
<p>
There are five precedence levels for binary operators.
Multiplication operators bind strongest, followed by addition
@@ -3814,7 +3808,7 @@ For floating-point and complex numbers,
<code>+x</code> is the same as <code>x</code>,
while <code>-x</code> is the negation of <code>x</code>.
The result of a floating-point or complex division by zero is not specified beyond the
IEEE 754 standard; whether a <a href="#Run_time_panics">run-time panic</a>
IEEE-754 standard; whether a <a href="#Run_time_panics">run-time panic</a>
occurs is implementation-specific.
</p>
@@ -3904,7 +3898,7 @@ These terms and the result of the comparisons are defined as follows:
<li>
Floating-point values are comparable and ordered,
as defined by the IEEE 754 standard.
as defined by the IEEE-754 standard.
</li>
<li>
@@ -4252,7 +4246,7 @@ When converting an integer or floating-point number to a floating-point type,
or a complex number to another complex type, the result value is rounded
to the precision specified by the destination type.
For instance, the value of a variable <code>x</code> of type <code>float32</code>
may be stored using additional precision beyond that of an IEEE 754 32-bit number,
may be stored using additional precision beyond that of an IEEE-754 32-bit number,
but float32(x) represents the result of rounding <code>x</code>'s value to
32-bit precision. Similarly, <code>x + 0.1</code> may use more than 32 bits
of precision, but <code>float32(x + 0.1)</code> does not.

View File

@@ -82,7 +82,7 @@ while still insisting that races are errors and that tools can diagnose and repo
<p>
The following formal definition of Go's memory model closely follows
the approach presented by Hans-J. Boehm and Sarita V. Adve in
<a href="https://dl.acm.org/doi/10.1145/1375581.1375591">Foundations of the C++ Concurrency Memory Model</a>”,
<a href="https://www.hpl.hp.com/techreports/2008/HPL-2008-56.pdf">Foundations of the C++ Concurrency Memory Model</a>”,
published in PLDI 2008.
The definition of data-race-free programs and the guarantee of sequential consistency
for race-free programs are equivalent to the ones in that work.
@@ -98,12 +98,12 @@ which in turn are made up of memory operations.
A <i>memory operation</i> is modeled by four details:
</p>
<ul>
<li>its kind, indicating whether it is an ordinary data read, an ordinary data write,
or a <i>synchronizing operation</i> such as an atomic data access,
a mutex operation, or a channel operation,</li>
<li>its location in the program,</li>
<li>the memory location or variable being accessed, and</li>
<li>the values read or written by the operation.</li>
<li>its kind, indicating whether it is an ordinary data read, an ordinary data write,
or a <i>synchronizing operation</i> such as an atomic data access,
a mutex operation, or a channel operation,
<li>its location in the program,
<li>the memory location or variable being accessed, and
<li>the values read or written by the operation.
</ul>
<p>
Some memory operations are <i>read-like</i>, including read, atomic read, mutex lock, and channel receive.
@@ -159,11 +159,10 @@ union of the sequenced before and synchronized before relations.
For an ordinary (non-synchronizing) data read <i>r</i> on a memory location <i>x</i>,
<i>W</i>(<i>r</i>) must be a write <i>w</i> that is <i>visible</i> to <i>r</i>,
where visible means that both of the following hold:
</p>
<ol>
<li><i>w</i> happens before <i>r</i>.</li>
<li><i>w</i> does not happen before any other write <i>w'</i> (to <i>x</i>) that happens before <i>r</i>.</li>
<li><i>w</i> happens before <i>r</i>.
<li><i>w</i> does not happen before any other write <i>w'</i> (to <i>x</i>) that happens before <i>r</i>.
</ol>
<p>
@@ -222,7 +221,7 @@ for programs that do contain races.
</p>
<p>
Any implementation can, upon detecting a data race,
First, any implementation can, upon detecting a data race,
report the race and halt execution of the program.
Implementations using ThreadSanitizer
(accessed with “<code>go</code> <code>build</code> <code>-race</code>”)
@@ -230,18 +229,7 @@ do exactly this.
</p>
<p>
A read of an array, struct, or complex number
may by implemented as a read of each individual sub-value
(array element, struct field, or real/imaginary component),
in any order.
Similarly, a write of an array, struct, or complex number
may be implemented as a write of each individual sub-value,
in any order.
</p>
<p>
A read <i>r</i> of a memory location <i>x</i>
holding a value
Otherwise, a read <i>r</i> of a memory location <i>x</i>
that is not larger than a machine word must observe
some write <i>w</i> such that <i>r</i> does not happen before <i>w</i>
and there is no write <i>w'</i> such that <i>w</i> happens before <i>w'</i>

View File

@@ -1,6 +1,6 @@
<!--{
"Title": "The Go Programming Language Specification",
"Subtitle": "Language version go1.24 (Nov 20, 2024)",
"Subtitle": "Version of Aug 2, 2023",
"Path": "/ref/spec"
}-->
@@ -10,7 +10,7 @@
This is the reference manual for the Go programming language.
The pre-Go1.18 version, without generics, can be found
<a href="/doc/go1.17_spec.html">here</a>.
For more information and other documents, see <a href="/">go.dev</a>.
For more information and other documents, see <a href="/">golang.org</a>.
</p>
<p>
@@ -70,14 +70,6 @@ enumerations or code snippets that are not further specified. The character <cod
language.
</p>
<p>
A link of the form [<a href="#Language_versions">Go 1.xx</a>] indicates that a described
language feature (or some aspect of it) was changed or added with language version 1.xx and
thus requires at minimum that language version to build.
For details, see the <a href="#Language_versions">linked section</a>
in the <a href="#Appendix">appendix</a>.
</p>
<h2 id="Source_code_representation">Source code representation</h2>
<p>
@@ -271,8 +263,7 @@ continue for import return var
<p>
The following character sequences represent <a href="#Operators">operators</a>
(including <a href="#Assignment_statements">assignment operators</a>) and punctuation
[<a href="#Go_1.18">Go 1.18</a>]:
(including <a href="#Assignment_statements">assignment operators</a>) and punctuation:
</p>
<pre class="grammar">
+ &amp; += &amp;= &amp;&amp; == != ( )
@@ -290,8 +281,7 @@ An integer literal is a sequence of digits representing an
<a href="#Constants">integer constant</a>.
An optional prefix sets a non-decimal base: <code>0b</code> or <code>0B</code>
for binary, <code>0</code>, <code>0o</code>, or <code>0O</code> for octal,
and <code>0x</code> or <code>0X</code> for hexadecimal
[<a href="#Go_1.13">Go 1.13</a>].
and <code>0x</code> or <code>0X</code> for hexadecimal.
A single <code>0</code> is considered a decimal zero.
In hexadecimal literals, letters <code>a</code> through <code>f</code>
and <code>A</code> through <code>F</code> represent values 10 through 15.
@@ -357,8 +347,7 @@ prefix, an integer part (hexadecimal digits), a radix point, a fractional part (
and an exponent part (<code>p</code> or <code>P</code> followed by an optional sign and decimal digits).
One of the integer part or the fractional part may be elided; the radix point may be elided as well,
but the exponent part is required. (This syntax matches the one given in IEEE 754-2008 §5.12.3.)
An exponent value exp scales the mantissa (integer and fractional part) by 2<sup>exp</sup>
[<a href="#Go_1.13">Go 1.13</a>].
An exponent value exp scales the mantissa (integer and fractional part) by 2<sup>exp</sup>.
</p>
<p>
@@ -422,8 +411,7 @@ It consists of an <a href="#Integer_literals">integer</a> or
<a href="#Floating-point_literals">floating-point</a> literal
followed by the lowercase letter <code>i</code>.
The value of an imaginary literal is the value of the respective
integer or floating-point literal multiplied by the imaginary unit <i>i</i>
[<a href="#Go_1.13">Go 1.13</a>]
integer or floating-point literal multiplied by the imaginary unit <i>i</i>.
</p>
<pre class="ebnf">
@@ -674,7 +662,7 @@ and are discussed in that section.
<p>
Numeric constants represent exact values of arbitrary precision and do not overflow.
Consequently, there are no constants denoting the IEEE 754 negative zero, infinity,
Consequently, there are no constants denoting the IEEE-754 negative zero, infinity,
and not-a-number values.
</p>
@@ -861,8 +849,8 @@ int16 the set of all signed 16-bit integers (-32768 to 32767)
int32 the set of all signed 32-bit integers (-2147483648 to 2147483647)
int64 the set of all signed 64-bit integers (-9223372036854775808 to 9223372036854775807)
float32 the set of all IEEE 754 32-bit floating-point numbers
float64 the set of all IEEE 754 64-bit floating-point numbers
float32 the set of all IEEE-754 32-bit floating-point numbers
float64 the set of all IEEE-754 64-bit floating-point numbers
complex64 the set of all complex numbers with float32 real and imaginary parts
complex128 the set of all complex numbers with float64 real and imaginary parts
@@ -896,7 +884,7 @@ are required when different numeric types are mixed in an expression
or assignment. For instance, <code>int32</code> and <code>int</code>
are not the same type even though they may have the same size on a
particular architecture.
</p>
<h3 id="String_types">String types</h3>
@@ -1086,7 +1074,7 @@ A field declared with a type but no explicit field name is called an <i>embedded
An embedded field must be specified as
a type name <code>T</code> or as a pointer to a non-interface type name <code>*T</code>,
and <code>T</code> itself may not be
a pointer type or type parameter. The unqualified type name acts as the field name.
a pointer type. The unqualified type name acts as the field name.
</p>
<pre>
@@ -1127,7 +1115,7 @@ of a struct except that they cannot be used as field names in
</p>
<p>
Given a struct type <code>S</code> and a type name
Given a struct type <code>S</code> and a <a href="#Types">named type</a>
<code>T</code>, promoted methods are included in the method set of the struct as follows:
</p>
<ul>
@@ -1352,7 +1340,6 @@ interface{}
<p>
For convenience, the predeclared type <code>any</code> is an alias for the empty interface.
[<a href="#Go_1.18">Go 1.18</a>]
</p>
<p>
@@ -1388,15 +1375,13 @@ as the <code>File</code> interface.
In a slightly more general form
an interface <code>T</code> may use a (possibly qualified) interface type
name <code>E</code> as an interface element. This is called
<i>embedding</i> interface <code>E</code> in <code>T</code>
[<a href="#Go_1.14">Go 1.14</a>].
<i>embedding</i> interface <code>E</code> in <code>T</code>.
The type set of <code>T</code> is the <i>intersection</i> of the type sets
defined by <code>T</code>'s explicitly declared methods and the type sets
of <code>T</code>s embedded interfaces.
In other words, the type set of <code>T</code> is the set of all types that implement all the
explicitly declared methods of <code>T</code> and also all the methods of
<code>E</code>
[<a href="#Go_1.18">Go 1.18</a>].
<code>E</code>.
</p>
<pre>
@@ -1435,8 +1420,7 @@ type ReadCloser interface {
<p>
In their most general form, an interface element may also be an arbitrary type term
<code>T</code>, or a term of the form <code>~T</code> specifying the underlying type <code>T</code>,
or a union of terms <code>t<sub>1</sub>|t<sub>2</sub>|…|t<sub>n</sub></code>
[<a href="#Go_1.18">Go 1.18</a>].
or a union of terms <code>t<sub>1</sub>|t<sub>2</sub>|…|t<sub>n</sub></code>.
Together with method specifications, these elements enable the precise
definition of an interface's type set as follows:
</p>
@@ -1682,7 +1666,6 @@ maps grow to accommodate the number of items
stored in them, with the exception of <code>nil</code> maps.
A <code>nil</code> map is equivalent to an empty map except that no elements
may be added.
</p>
<h3 id="Channel_types">Channel types</h3>
@@ -1927,8 +1910,8 @@ components have identical types. In detail:
<li>Two slice types are identical if they have identical element types.</li>
<li>Two struct types are identical if they have the same sequence of fields,
and if corresponding pairs of fields have the same names, identical types,
and identical tags, and are either both embedded or both not embedded.
and if corresponding fields have the same names, and identical types,
and identical tags.
<a href="#Exported_identifiers">Non-exported</a> field names from different
packages are always different.</li>
@@ -2320,9 +2303,7 @@ as an <a href="#Operands">operand</a>, and in <a href="#Assignment_statements">a
<p>
The following identifiers are implicitly declared in the
<a href="#Blocks">universe block</a>
[<a href="#Go_1.18">Go 1.18</a>]
[<a href="#Go_1.21">Go 1.21</a>]:
<a href="#Blocks">universe block</a>:
</p>
<pre class="grammar">
Types:
@@ -2506,17 +2487,16 @@ TypeSpec = AliasDecl | TypeDef .
<h4 id="Alias_declarations">Alias declarations</h4>
<p>
An alias declaration binds an identifier to the given type
[<a href="#Go_1.9">Go 1.9</a>].
An alias declaration binds an identifier to the given type.
</p>
<pre class="ebnf">
AliasDecl = identifier [ TypeParameters ] "=" Type .
AliasDecl = identifier "=" Type .
</pre>
<p>
Within the <a href="#Declarations_and_scope">scope</a> of
the identifier, it serves as an <i>alias</i> for the given type.
the identifier, it serves as an <i>alias</i> for the type.
</p>
<pre>
@@ -2526,24 +2506,6 @@ type (
)
</pre>
<p>
If the alias declaration specifies <a href="#Type_parameter_declarations">type parameters</a>
[<a href="#Go_1.24">Go 1.24</a>], the type name denotes a <i>generic alias</i>.
Generic aliases must be <a href="#Instantiations">instantiated</a> when they
are used.
</p>
<pre>
type set[P comparable] = map[P]bool
</pre>
<p>
In an alias declaration the given type cannot be a type parameter.
</p>
<pre>
type A[P any] = P // illegal: P is a type parameter
</pre>
<h4 id="Type_definitions">Type definitions</h4>
@@ -2674,8 +2636,7 @@ func (l *List[T]) Len() int { … }
A type parameter list declares the <i>type parameters</i> of a generic function or type declaration.
The type parameter list looks like an ordinary <a href="#Function_types">function parameter list</a>
except that the type parameter names must all be present and the list is enclosed
in square brackets rather than parentheses
[<a href="#Go_1.18">Go 1.18</a>].
in square brackets rather than parentheses.
</p>
<pre class="ebnf">
@@ -2758,8 +2719,7 @@ type T6[P int] struct{ f *T6[P] } // ok: reference to T6 is not in type para
<p>
A <i>type constraint</i> is an <a href="#Interface_types">interface</a> that defines the
set of permissible type arguments for the respective type parameter and controls the
operations supported by values of that type parameter
[<a href="#Go_1.18">Go 1.18</a>].
operations supported by values of that type parameter.
</p>
<pre class="ebnf">
@@ -2789,8 +2749,7 @@ other interfaces based on their type sets. But this should get us going for now.
The <a href="#Predeclared_identifiers">predeclared</a>
<a href="#Interface_types">interface type</a> <code>comparable</code>
denotes the set of all non-interface types that are
<a href="#Comparison_operators">strictly comparable</a>
[<a href="#Go_1.18">Go 1.18</a>].
<a href="#Comparison_operators">strictly comparable</a>.
</p>
<p>
@@ -2823,8 +2782,7 @@ if <code>T</code> is an element of the type set defined by <code>C</code>; i.e.,
if <code>T</code> <a href="#Implementing_an_interface">implements</a> <code>C</code>.
As an exception, a <a href="#Comparison_operators">strictly comparable</a>
type constraint may also be satisfied by a <a href="#Comparison_operators">comparable</a>
(not necessarily strictly comparable) type argument
[<a href="#Go_1.20">Go 1.20</a>].
(not necessarily strictly comparable) type argument.
More precisely:
</p>
@@ -3093,7 +3051,7 @@ to the base type <code>Point</code>.
</p>
<p>
If the receiver base type is a <a href="#Type_definitions">generic type</a>, the
If the receiver base type is a <a href="#Type_declarations">generic type</a>, the
receiver specification must declare corresponding type parameters for the method
to use. This makes the receiver type parameters available to the method.
Syntactically, this type parameter declaration looks like an
@@ -3117,22 +3075,6 @@ func (p Pair[A, B]) Swap() Pair[B, A] { … } // receiver declares A, B
func (p Pair[First, _]) First() First { … } // receiver declares First, corresponds to A in Pair
</pre>
<p>
If the receiver type is denoted by (a pointer to) an <a href="#Alias_declarations">alias</a>,
the alias must not be generic and it must not denote an instantiated generic type, neither
directly nor indirectly via another alias, and irrespective of pointer indirections.
</p>
<pre>
type GPoint[P any] = Point
type HPoint = *GPoint[int]
type IPair = Pair[int, int]
func (*GPoint[P]) Draw(P) { … } // illegal: alias must not be generic
func (HPoint) Draw(P) { … } // illegal: alias must not denote instantiated type GPoint[int]
func (*IPair) Second() int { … } // illegal: alias must not denote instantiated type Pair[int, int]
</pre>
<h2 id="Expressions">Expressions</h2>
<p>
@@ -3968,7 +3910,7 @@ For <code>a</code> of <a href="#Type_parameter_declarations">type parameter type
that <code>P</code> is instantiated with, and the type of <code>a[x]</code> is
the type of the (identical) element types.</li>
<li><code>a[x]</code> may not be assigned to if <code>P</code>'s type set
includes string types.</li>
includes string types.
</ul>
<p>
@@ -4364,7 +4306,7 @@ with the same underlying array.
<p>
A generic function or type is <i>instantiated</i> by substituting <i>type arguments</i>
for the type parameters [<a href="#Go_1.18">Go 1.18</a>].
for the type parameters.
Instantiation proceeds in two steps:
</p>
@@ -4549,7 +4491,7 @@ the type parameters of the functions that need to be instantiated
and for which no explicit type arguments is provided.
These type parameters are called <i>bound</i> type parameters.
For instance, in the <code>dedup</code> example above, the type parameters
<code>S</code> and <code>E</code> are bound to <code>dedup</code>.
<code>P</code> and <code>E</code> are bound to <code>dedup</code>.
An argument to a generic function call may be a generic function itself.
The type parameters of that function are included in the set of bound
type parameters.
@@ -4697,7 +4639,7 @@ Type inference succeeds if no unification step fails and the map has
an entry for each type parameter.
</p>
<p>
</pre>
For example, given the type equation with the bound type parameter
<code>P</code>
</p>
@@ -4817,7 +4759,6 @@ to the type of the other operand.
<p>
The right operand in a shift expression must have <a href="#Numeric_types">integer type</a>
[<a href="#Go_1.13">Go 1.13</a>]
or be an untyped constant <a href="#Representability">representable</a> by a
value of type <code>uint</code>.
If the left operand of a non-constant shift expression is an untyped constant,
@@ -4862,7 +4803,6 @@ As the <code>++</code> and <code>--</code> operators form
statements, not expressions, they fall
outside the operator hierarchy.
As a consequence, statement <code>*p++</code> is the same as <code>(*p)++</code>.
</p>
<p>
There are five precedence levels for binary operators.
Multiplication operators bind strongest, followed by addition
@@ -4885,13 +4825,12 @@ For instance, <code>x / y * z</code> is the same as <code>(x / y) * z</code>.
</p>
<pre>
+x // x
42 + a - b // (42 + a) - b
23 + 3*x[i] // 23 + (3 * x[i])
x &lt;= f() // x &lt;= f()
^a &gt;&gt; b // (^a) >> b
f() || g() // f() || g()
x == y+1 &amp;&amp; &lt;-chanInt &gt; 0 // (x == (y+1)) && ((<-chanInt) > 0)
+x
23 + 3*x[i]
x &lt;= f()
^a &gt;&gt; b
f() || g()
x == y+1 &amp;&amp; &lt;-chanInt &gt; 0
</pre>
@@ -5056,7 +4995,7 @@ For floating-point and complex numbers,
<code>+x</code> is the same as <code>x</code>,
while <code>-x</code> is the negation of <code>x</code>.
The result of a floating-point or complex division by zero is not specified beyond the
IEEE 754 standard; whether a <a href="#Run_time_panics">run-time panic</a>
IEEE-754 standard; whether a <a href="#Run_time_panics">run-time panic</a>
occurs is implementation-specific.
</p>
@@ -5146,7 +5085,7 @@ These terms and the result of the comparisons are defined as follows:
<li>
Floating-point types are comparable and ordered.
Two floating-point values are compared as defined by the IEEE 754 standard.
Two floating-point values are compared as defined by the IEEE-754 standard.
</li>
<li>
@@ -5268,7 +5207,7 @@ Specifically:
<p>
Logical operators apply to <a href="#Boolean_types">boolean</a> values
and yield a result of the same type as the operands.
The left operand is evaluated, and then the right if the condition requires it.
The right operand is evaluated conditionally.
</p>
<pre class="grammar">
@@ -5485,8 +5424,7 @@ in any of these cases:
<code>x</code> is a string and <code>T</code> is a slice of bytes or runes.
</li>
<li>
<code>x</code> is a slice, <code>T</code> is an array [<a href="#Go_1.20">Go 1.20</a>]
or a pointer to an array [<a href="#Go_1.17">Go 1.17</a>],
<code>x</code> is a slice, <code>T</code> is an array or a pointer to an array,
and the slice and array types have <a href="#Type_identity">identical</a> element types.
</li>
</ul>
@@ -5576,7 +5514,7 @@ When converting an integer or floating-point number to a floating-point type,
or a <a href="#Numeric_types">complex number</a> to another complex type, the result value is rounded
to the precision specified by the destination type.
For instance, the value of a variable <code>x</code> of type <code>float32</code>
may be stored using additional precision beyond that of an IEEE 754 32-bit number,
may be stored using additional precision beyond that of an IEEE-754 32-bit number,
but float32(x) represents the result of rounding <code>x</code>'s value to
32-bit precision. Similarly, <code>x + 0.1</code> may use more than 32 bits
of precision, but <code>float32(x + 0.1)</code> does not.
@@ -5631,7 +5569,7 @@ myString([]myRune{0x1f30e}) // "\U0001f30e" == "🌎"
<li>
Converting a value of a string type to a slice of bytes type
yields a non-nil slice whose successive elements are the bytes of the string.
yields a slice whose successive elements are the bytes of the string.
<pre>
[]byte("hellø") // []byte{'h', 'e', 'l', 'l', '\xc3', '\xb8'}
@@ -5847,28 +5785,24 @@ determine the evaluation order of individual initialization expressions in
Otherwise, when evaluating the <a href="#Operands">operands</a> of an
expression, assignment, or
<a href="#Return_statements">return statement</a>,
all function calls, method calls,
<a href="#Receive operator">receive operations</a>,
and <a href="#Logical_operators">binary logical operations</a>
are evaluated in lexical left-to-right order.
all function calls, method calls, and
communication operations are evaluated in lexical left-to-right
order.
</p>
<p>
For example, in the (function-local) assignment
</p>
<pre>
y[f()], ok = g(z || h(), i()+x[j()], &lt;-c), k()
y[f()], ok = g(h(), i()+x[j()], &lt;-c), k()
</pre>
<p>
the function calls and communication happen in the order
<code>f()</code>, <code>h()</code> (if <code>z</code>
evaluates to false), <code>i()</code>, <code>j()</code>,
<code>f()</code>, <code>h()</code>, <code>i()</code>, <code>j()</code>,
<code>&lt;-c</code>, <code>g()</code>, and <code>k()</code>.
However, the order of those events compared to the evaluation
and indexing of <code>x</code> and the evaluation
of <code>y</code> and <code>z</code> is not specified,
except as required lexically. For instance, <code>g</code>
cannot be called before its arguments are evaluated.
of <code>y</code> is not specified.
</p>
<pre>
@@ -6576,6 +6510,7 @@ additionally it may specify an <i>init</i>
and a <i>post</i> statement, such as an assignment,
an increment or decrement statement. The init statement may be a
<a href="#Short_variable_declarations">short variable declaration</a>, but the post statement must not.
Variables declared by the init statement are re-used in each iteration.
</p>
<pre class="ebnf">
@@ -6607,56 +6542,12 @@ for cond { S() } is the same as for ; cond ; { S() }
for { S() } is the same as for true { S() }
</pre>
<p>
Each iteration has its own separate declared variable (or variables)
[<a href="#Go_1.22">Go 1.22</a>].
The variable used by the first iteration is declared by the init statement.
The variable used by each subsequent iteration is declared implicitly before
executing the post statement and initialized to the value of the previous
iteration's variable at that moment.
</p>
<pre>
var prints []func()
for i := 0; i < 5; i++ {
prints = append(prints, func() { println(i) })
i++
}
for _, p := range prints {
p()
}
</pre>
<p>
prints
</p>
<pre>
1
3
5
</pre>
<p>
Prior to [<a href="#Go_1.22">Go 1.22</a>], iterations share one set of variables
instead of having their own separate variables.
In that case, the example above prints
</p>
<pre>
6
6
6
</pre>
<h4 id="For_range">For statements with <code>range</code> clause</h4>
<p>
A "for" statement with a "range" clause
iterates through all entries of an array, slice, string or map, values received on
a channel, integer values from zero to an upper limit [<a href="#Go_1.22">Go 1.22</a>],
or values passed to an iterator function's yield function [<a href="#Go_1.23">Go 1.23</a>].
For each entry it assigns <i>iteration values</i>
iterates through all entries of an array, slice, string or map,
or values received on a channel. For each entry it assigns <i>iteration values</i>
to corresponding <i>iteration variables</i> if present and then executes the block.
</p>
@@ -6667,24 +6558,20 @@ RangeClause = [ ExpressionList "=" | IdentifierList ":=" ] "range" Expression .
<p>
The expression on the right in the "range" clause is called the <i>range expression</i>,
its <a href="#Core_types">core type</a> must be
an array, pointer to an array, slice, string, map, channel permitting
<a href="#Receive_operator">receive operations</a>, an integer, or
a function with specific signature (see below).
an array, pointer to an array, slice, string, map, or channel permitting
<a href="#Receive_operator">receive operations</a>.
As with an assignment, if present the operands on the left must be
<a href="#Address_operators">addressable</a> or map index expressions; they
denote the iteration variables.
If the range expression is a function, the maximum number of iteration variables depends on
the function signature.
If the range expression is a channel or integer, at most one iteration variable is permitted;
otherwise there may be up to two.
denote the iteration variables. If the range expression is a channel, at most
one iteration variable is permitted, otherwise there may be up to two.
If the last iteration variable is the <a href="#Blank_identifier">blank identifier</a>,
the range clause is equivalent to the same clause without that identifier.
</p>
<p>
The range expression <code>x</code> is evaluated before beginning the loop,
with one exception: if at most one iteration variable is present and <code>x</code> or
<a href="#Length_and_capacity"><code>len(x)</code></a> is <a href="#Constants">constant</a>,
The range expression <code>x</code> is evaluated once before beginning the loop,
with one exception: if at most one iteration variable is present and
<code>len(x)</code> is <a href="#Length_and_capacity">constant</a>,
the range expression is not evaluated.
</p>
@@ -6695,16 +6582,12 @@ if the respective iteration variables are present:
</p>
<pre class="grammar">
Range expression 1st value 2nd value
Range expression 1st value 2nd value
array or slice a [n]E, *[n]E, or []E index i int a[i] E
string s string type index i int see below rune
map m map[K]V key k K m[k] V
channel c chan E, &lt;-chan E element e E
integer value n integer type, or untyped int value i see below
function, 0 values f func(func() bool)
function, 1 value f func(func(V) bool) value v V
function, 2 values f func(func(K, V) bool) key k K v V
array or slice a [n]E, *[n]E, or []E index i int a[i] E
string s string type index i int see below rune
map m map[K]V key k K m[k] V
channel c chan E, &lt;-chan E element e E
</pre>
<ol>
@@ -6743,53 +6626,22 @@ For channels, the iteration values produced are the successive values sent on
the channel until the channel is <a href="#Close">closed</a>. If the channel
is <code>nil</code>, the range expression blocks forever.
</li>
<li>
For an integer value <code>n</code>, where <code>n</code> is of <a href="#Numeric_types">integer type</a>
or an untyped <a href="#Constants">integer constant</a>, the iteration values 0 through <code>n-1</code>
are produced in increasing order.
If <code>n</code> is of integer type, the iteration values have that same type.
Otherwise, the type of <code>n</code> is determined as if it were assigned to the
iteration variable.
Specifically:
if the iteration variable is preexisting, the type of the iteration values is the type of the iteration
variable, which must be of integer type.
Otherwise, if the iteration variable is declared by the "range" clause or is absent,
the type of the iteration values is the <a href="#Constants">default type</a> for <code>n</code>.
If <code>n</code> &lt= 0, the loop does not run any iterations.
</li>
<li>
For a function <code>f</code>, the iteration proceeds by calling <code>f</code>
with a new, synthesized <code>yield</code> function as its argument.
If <code>yield</code> is called before <code>f</code> returns,
the arguments to <code>yield</code> become the iteration values
for executing the loop body once.
After each successive loop iteration, <code>yield</code> returns true
and may be called again to continue the loop.
As long as the loop body does not terminate, the "range" clause will continue
to generate iteration values this way for each <code>yield</code> call until
<code>f</code> returns.
If the loop body terminates (such as by a <code>break</code> statement),
<code>yield</code> returns false and must not be called again.
</li>
</ol>
<p>
The iteration values are assigned to the respective
iteration variables as in an <a href="#Assignment_statements">assignment statement</a>.
</p>
<p>
The iteration variables may be declared by the "range" clause using a form of
<a href="#Short_variable_declarations">short variable declaration</a>
(<code>:=</code>).
In this case their <a href="#Declarations_and_scope">scope</a> is the block of the "for" statement
and each iteration has its own new variables [<a href="#Go_1.22">Go 1.22</a>]
(see also <a href="#For_clause">"for" statements with a ForClause</a>).
The variables have the types of their respective iteration values.
</p>
<p>
If the iteration variables are not explicitly declared by the "range" clause,
they must be preexisting.
In this case, the iteration values are assigned to the respective variables
as in an <a href="#Assignment_statements">assignment statement</a>.
In this case their types are set to the types of the respective iteration values
and their <a href="#Declarations_and_scope">scope</a> is the block of the "for"
statement; they are re-used in each iteration.
If the iteration variables are declared outside the "for" statement,
after execution their values will be those of the last iteration.
</p>
<pre>
@@ -6826,59 +6678,6 @@ for w := range ch {
// empty a channel
for range ch {}
// call f(0), f(1), ... f(9)
for i := range 10 {
// type of i is int (default type for untyped constant 10)
f(i)
}
// invalid: 256 cannot be assigned to uint8
var u uint8
for u = range 256 {
}
// invalid: 1e3 is a floating-point constant
for range 1e3 {
}
// fibo generates the Fibonacci sequence
fibo := func(yield func(x int) bool) {
f0, f1 := 0, 1
for yield(f0) {
f0, f1 = f1, f0+f1
}
}
// print the Fibonacci numbers below 1000:
for x := range fibo {
if x >= 1000 {
break
}
fmt.Printf("%d ", x)
}
// output: 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987
// iteration support for a recursive tree data structure
type Tree[K cmp.Ordered, V any] struct {
left, right *Tree[K, V]
key K
value V
}
func (t *Tree[K, V]) walk(yield func(key K, val V) bool) bool {
return t == nil || t.left.walk(yield) && yield(t.key, t.value) && t.right.walk(yield)
}
func (t *Tree[K, V]) Walk(yield func(key K, val V) bool) {
t.walk(yield)
}
// walk tree t in-order
var t Tree[string, int]
for k, v := range t.Walk {
// process k, v
}
</pre>
@@ -7402,7 +7201,7 @@ The number of elements copied is the minimum of
<code>len(src)</code> and <code>len(dst)</code>.
As a special case, if the destination's core type is <code>[]byte</code>,
<code>copy</code> also accepts a source argument with core type
<a href="#Core_types"><code>bytestring</code></a>.
</a> <a href="#Core_types"><code>bytestring</code></a>.
This form copies the bytes from the byte slice or string into the byte slice.
</p>
@@ -7430,8 +7229,7 @@ n3 := copy(b, "Hello, World!") // n3 == 5, b is []byte("Hello")
<p>
The built-in function <code>clear</code> takes an argument of <a href="#Map_types">map</a>,
<a href="#Slice_types">slice</a>, or <a href="#Type_parameter_declarations">type parameter</a> type,
and deletes or zeroes out all elements
[<a href="#Go_1.21">Go 1.21</a>].
and deletes or zeroes out all elements.
</p>
<pre class="grammar">
@@ -7698,8 +7496,7 @@ The precise behavior is implementation-dependent.
The built-in functions <code>min</code> and <code>max</code> compute the
smallest&mdash;or largest, respectively&mdash;value of a fixed number of
arguments of <a href="#Comparison_operators">ordered types</a>.
There must be at least one argument
[<a href="#Go_1.21">Go 1.21</a>].
There must be at least one argument.
</p>
<p>
@@ -8415,8 +8212,8 @@ of if the general conversion rules take care of this.
<p>
A <code>Pointer</code> is a <a href="#Pointer_types">pointer type</a> but a <code>Pointer</code>
value may not be <a href="#Address_operators">dereferenced</a>.
Any pointer or value of <a href="#Core_types">core type</a> <code>uintptr</code> can be
<a href="#Conversions">converted</a> to a type of core type <code>Pointer</code> and vice versa.
Any pointer or value of <a href="#Underlying_types">underlying type</a> <code>uintptr</code> can be
<a href="#Conversions">converted</a> to a type of underlying type <code>Pointer</code> and vice versa.
The effect of converting between <code>Pointer</code> and <code>uintptr</code> is implementation-defined.
</p>
@@ -8427,10 +8224,6 @@ bits = *(*uint64)(unsafe.Pointer(&amp;f))
type ptr unsafe.Pointer
bits = *(*uint64)(ptr(&amp;f))
func f[P ~*B, B any](p P) uintptr {
return uintptr(unsafe.Pointer(p))
}
var p ptr = nil
</pre>
@@ -8479,8 +8272,7 @@ of constant size.
<p>
The function <code>Add</code> adds <code>len</code> to <code>ptr</code>
and returns the updated pointer <code>unsafe.Pointer(uintptr(ptr) + uintptr(len))</code>
[<a href="#Go_1.17">Go 1.17</a>].
and returns the updated pointer <code>unsafe.Pointer(uintptr(ptr) + uintptr(len))</code>.
The <code>len</code> argument must be of <a href="#Numeric_types">integer type</a> or an untyped <a href="#Constants">constant</a>.
A constant <code>len</code> argument must be <a href="#Representability">representable</a> by a value of type <code>int</code>;
if it is an untyped constant it is given type <code>int</code>.
@@ -8500,8 +8292,7 @@ and whose length and capacity are <code>len</code>.
<p>
except that, as a special case, if <code>ptr</code>
is <code>nil</code> and <code>len</code> is zero,
<code>Slice</code> returns <code>nil</code>
[<a href="#Go_1.17">Go 1.17</a>].
<code>Slice</code> returns <code>nil</code>.
</p>
<p>
@@ -8510,16 +8301,14 @@ A constant <code>len</code> argument must be non-negative and <a href="#Represen
if it is an untyped constant it is given type <code>int</code>.
At run time, if <code>len</code> is negative,
or if <code>ptr</code> is <code>nil</code> and <code>len</code> is not zero,
a <a href="#Run_time_panics">run-time panic</a> occurs
[<a href="#Go_1.17">Go 1.17</a>].
a <a href="#Run_time_panics">run-time panic</a> occurs.
</p>
<p>
The function <code>SliceData</code> returns a pointer to the underlying array of the <code>slice</code> argument.
If the slice's capacity <code>cap(slice)</code> is not zero, that pointer is <code>&slice[:1][0]</code>.
If <code>slice</code> is <code>nil</code>, the result is <code>nil</code>.
Otherwise it is a non-<code>nil</code> pointer to an unspecified memory address
[<a href="#Go_1.20">Go 1.20</a>].
Otherwise it is a non-<code>nil</code> pointer to an unspecified memory address.
</p>
<p>
@@ -8528,14 +8317,12 @@ The function <code>String</code> returns a <code>string</code> value whose under
The same requirements apply to the <code>ptr</code> and <code>len</code> argument as in the function
<code>Slice</code>. If <code>len</code> is zero, the result is the empty string <code>""</code>.
Since Go strings are immutable, the bytes passed to <code>String</code> must not be modified afterwards.
[<a href="#Go_1.20">Go 1.20</a>]
</p>
<p>
The function <code>StringData</code> returns a pointer to the underlying bytes of the <code>str</code> argument.
For an empty string the return value is unspecified, and may be <code>nil</code>.
Since Go strings are immutable, the bytes returned by <code>StringData</code> must not be modified
[<a href="#Go_1.20">Go 1.20</a>].
Since Go strings are immutable, the bytes returned by <code>StringData</code> must not be modified.
</p>
<h3 id="Size_and_alignment_guarantees">Size and alignment guarantees</h3>
@@ -8576,160 +8363,6 @@ A struct or array type has size zero if it contains no fields (or elements, resp
<h2 id="Appendix">Appendix</h2>
<h3 id="Language_versions">Language versions</h3>
<p>
The <a href="/doc/go1compat">Go 1 compatibility guarantee</a> ensures that
programs written to the Go 1 specification will continue to compile and run
correctly, unchanged, over the lifetime of that specification.
More generally, as adjustments are made and features added to the language,
the compatibility guarantee ensures that a Go program that works with a
specific Go language version will continue to work with any subsequent version.
</p>
<p>
For instance, the ability to use the prefix <code>0b</code> for binary
integer literals was introduced with Go 1.13, indicated
by [<a href="#Go_1.13">Go 1.13</a>] in the section on
<a href="#Integer_literals">integer literals</a>.
Source code containing an integer literal such as <code>0b1011</code>
will be rejected if the implied or required language version used by
the compiler is older than Go 1.13.
</p>
<p>
The following table describes the minimum language version required for
features introduced after Go 1.
</p>
<h4 id="Go_1.9">Go 1.9</h4>
<ul>
<li>
An <a href="#Alias_declarations">alias declaration</a> may be used to declare an alias name for a type.
</li>
</ul>
<h4 id="Go_1.13">Go 1.13</h4>
<ul>
<li>
<a href="#Integer_literals">Integer literals</a> may use the prefixes <code>0b</code>, <code>0B</code>, <code>0o</code>,
and <code>0O</code> for binary, and octal literals, respectively.
</li>
<li>
Hexadecimal <a href="#Floating-point_literals">floating-point literals</a> may be written using the prefixes
<code>0x</code> and <code>0X</code>.
</li>
<li>
The <a href="#Imaginary_literals">imaginary suffix</a> <code>i</code> may be used with any (binary, decimal, hexadecimal)
integer or floating-point literal, not just decimal literals.
</li>
<li>
The digits of any number literal may be <a href="#Integer_literals">separated</a> (grouped)
using underscores <code>_</code>.
</li>
<li>
The shift count in a <a href="#Operators">shift operation</a> may be a signed integer type.
</li>
</ul>
<h4 id="Go_1.14">Go 1.14</h4>
<ul>
<li>
Emdedding a method more than once through different <a href="#Embedded_interfaces">embedded interfaces</a>
is not an error.
</li>
</ul>
<h4 id="Go_1.17">Go 1.17</h4>
<ul>
<li>
A slice may be <a href="#Conversions">converted</a> to an array pointer if the slice and array element
types match, and the array is not longer than the slice.
</li>
<li>
The built-in <a href="#Package_unsafe">package <code>unsafe</code></a> includes the new functions
<code>Add</code> and <code>Slice</code>.
</li>
</ul>
<h4 id="Go_1.18">Go 1.18</h4>
<p>
The 1.18 release adds polymorphic functions and types ("generics") to the language.
Specifically:
</p>
<ul>
<li>
The set of <a href="#Operators_and_punctuation">operators and punctuation</a> includes the new token <code>~</code>.
</li>
<li>
Function and type declarations may declare <a href="#Type_parameter_declarations">type parameters</a>.
</li>
<li>
Interface types may <a href="#General_interfaces">embed arbitrary types</a> (not just type names of interfaces)
as well as union and <code>~T</code> type elements.
</li>
<li>
The set of <a href="#Predeclared_identifiers">predeclared</a> types includes the new types
<code>any</code> and <code>comparable</code>.
</li>
</ul>
<h4 id="Go_1.20">Go 1.20</h4>
<ul>
<li>
A slice may be <a href="#Conversions">converted</a> to an array if the slice and array element
types match and the array is not longer than the slice.
</li>
<li>
The built-in <a href="#Package_unsafe">package <code>unsafe</code></a> includes the new functions
<code>SliceData</code>, <code>String</code>, and <code>StringData</code>.
</li>
<li>
<a href="#Comparison_operators">Comparable types</a> (such as ordinary interfaces) may satisfy
<code>comparable</code> constraints, even if the type arguments are not strictly comparable.
</li>
</ul>
<h4 id="Go_1.21">Go 1.21</h4>
<ul>
<li>
The set of <a href="#Predeclared_identifiers">predeclared</a> functions includes the new functions
<code>min</code>, <code>max</code>, and <code>clear</code>.
</li>
<li>
<a href="#Type_inference">Type inference</a> uses the types of interface methods for inference.
It also infers type arguments for generic functions assigned to variables or
passed as arguments to other (possibly generic) functions.
</li>
</ul>
<h4 id="Go_1.22">Go 1.22</h4>
<ul>
<li>
In a <a href="#For_statements">"for" statement</a>, each iteration has its own set of iteration
variables rather than sharing the same variables in each iteration.
</li>
<li>
A "for" statement with <a href="#For_range">"range" clause</a> may iterate over
integer values from zero to an upper limit.
</li>
</ul>
<h4 id="Go_1.23">Go 1.23</h4>
<ul>
<li>A "for" statement with <a href="#For_range">"range" clause</a> accepts an iterator
function as range expression.
</li>
</ul>
<h4 id="Go_1.24">Go 1.24</h4>
<ul>
<li>
An <a href="#Alias_declarations">alias declaration</a> may declare
<a href="#Type_parameter_declarations">type parameters</a>.
</li>
</ul>
<h3 id="Type_unification_rules">Type unification rules</h3>
<p>
@@ -8854,7 +8487,7 @@ Finally, two types that are not bound type parameters unify loosely
identical <a href="#Interface_types">type terms</a>,
both or neither embed the predeclared type
<a href="#Predeclared_identifiers">comparable</a>,
corresponding method types unify exactly,
corresponding method types unify per the element matching mode,
and the method set of one of the interfaces is a subset of
the method set of the other interface.
</li>

View File

@@ -34,7 +34,6 @@ For example, if a Go program is running in an environment that contains
then that Go program will disable the use of HTTP/2 by default in both
the HTTP client and the HTTP server.
Unrecognized settings in the `GODEBUG` environment variable are ignored.
It is also possible to set the default `GODEBUG` for a given program
(discussed below).
@@ -89,38 +88,14 @@ Because this method of setting GODEBUG defaults was introduced only in Go 1.21,
programs listing versions of Go earlier than Go 1.20 are configured to match Go 1.20,
not the older version.
To override these defaults, starting in Go 1.23, the work module's `go.mod`
or the workspace's `go.work` can list one or more `godebug` lines:
godebug (
default=go1.21
panicnil=1
asynctimerchan=0
)
The special key `default` indicates a Go version to take unspecified
settings from. This allows setting the GODEBUG defaults separately
from the Go language version in the module.
In this example, the program is asking for Go 1.21 semantics and
then asking for the old pre-Go 1.21 `panic(nil)` behavior and the
new Go 1.23 `asynctimerchan=0` behavior.
Only the work module's `go.mod` is consulted for `godebug` directives.
Any directives in required dependency modules are ignored.
It is an error to list a `godebug` with an unrecognized setting.
(Toolchains older than Go 1.23 reject all `godebug` lines, since they do not
understand `godebug` at all.)
The defaults from the `go` and `godebug` lines apply to all main
packages that are built. For more fine-grained control,
starting in Go 1.21, a main package's source files
To override these defaults, a main package's source files
can include one or more `//go:debug` directives at the top of the file
(preceding the `package` statement).
The `godebug` lines in the previous example would be written:
Continuing the `panicnil` example, if the module or workspace is updated
to say `go` `1.21`, the program can opt back into the old `panic(nil)`
behavior by including this directive:
//go:debug default=go1.21
//go:debug panicnil=1
//go:debug asynctimerchan=0
Starting in Go 1.21, the Go toolchain treats a `//go:debug` directive
with an unrecognized GODEBUG setting as an invalid program.
@@ -151,166 +126,14 @@ for example,
see the [runtime documentation](/pkg/runtime#hdr-Environment_Variables)
and the [go command documentation](/cmd/go#hdr-Build_and_test_caching).
### Go 1.24
Go 1.24 changed the global [`math/rand.Seed`](/pkg/math/rand/#Seed) to be a
no-op. This behavior is controlled by the `randseednop` setting.
For Go 1.24 it defaults to `randseednop=1`.
Using `randseednop=0` reverts to the pre-Go 1.24 behavior.
Go 1.24 added new values for the `multipathtcp` setting.
The possible values for `multipathtcp` are now:
- "0": disable MPTCP on dialers and listeners by default
- "1": enable MPTCP on dialers and listeners by default
- "2": enable MPTCP on listeners only by default
- "3": enable MPTCP on dialers only by default
For Go 1.24, it now defaults to multipathtcp="2", thus
enabled by default on listeners. Using multipathtcp="0" reverts to the
pre-Go 1.24 behavior.
Go 1.24 changed the behavior of `go test -json` to emit build errors as JSON
instead of text.
These new JSON events are distinguished by new `Action` values,
but can still cause problems with CI systems that aren't robust to these events.
This behavior can be controlled with the `gotestjsonbuildtext` setting.
Using `gotestjsonbuildtext=1` restores the 1.23 behavior.
This setting will be removed in a future release, Go 1.28 at the earliest.
Go 1.24 changed [`crypto/rsa`](/pkg/crypto/rsa) to require RSA keys to be at
least 1024 bits. This behavior can be controlled with the `rsa1024min` setting.
Using `rsa1024min=0` restores the Go 1.23 behavior.
Go 1.24 introduced a mechanism for enabling platform specific Data Independent
Timing (DIT) modes in the [`crypto/subtle`](/pkg/crypto/subtle) package. This
mode can be enabled for an entire program with the `dataindependenttiming` setting.
For Go 1.24 it defaults to `dataindependenttiming=0`. There is no change in default
behavior from Go 1.23 when `dataindependenttiming` is unset.
Using `dataindependenttiming=1` enables the DIT mode for the entire Go program.
When enabled, DIT will be enabled when calling into C from Go. When enabled,
calling into Go code from C will enable DIT, and disable it before returning to
C if it was not enabled when Go code was entered.
This currently only affects arm64 programs. For all other platforms it is a no-op.
Go 1.24 removed the `x509sha1` setting. `crypto/x509` no longer supports verifying
signatures on certificates that use SHA-1 based signature algorithms.
Go 1.24 changes the default value of the [`x509usepolicies`
setting.](/pkg/crypto/x509/#CreateCertificate) from `0` to `1`. When marshalling
certificates, policies are now taken from the
[`Certificate.Policies`](/pkg/crypto/x509/#Certificate.Policies) field rather
than the
[`Certificate.PolicyIdentifiers`](/pkg/crypto/x509/#Certificate.PolicyIdentifiers)
field by default.
Go 1.24 enabled the post-quantum key exchange mechanism
X25519MLKEM768 by default. The default can be reverted using the
[`tlsmlkem` setting](/pkg/crypto/tls/#Config.CurvePreferences).
Go 1.24 also removed X25519Kyber768Draft00 and the Go 1.23 `tlskyber` setting.
Go 1.24 made [`ParsePKCS1PrivateKey`](/pkg/crypto/x509/#ParsePKCS1PrivateKey)
use and validate the CRT parameters in the encoded private key. This behavior
can be controlled with the `x509rsacrt` setting. Using `x509rsacrt=0` restores
the Go 1.23 behavior.
### Go 1.23
Go 1.23 changed the channels created by package time to be unbuffered
(synchronous), which makes correct use of the [`Timer.Stop`](/pkg/time/#Timer.Stop)
and [`Timer.Reset`](/pkg/time/#Timer.Reset) method results much easier.
The [`asynctimerchan` setting](/pkg/time/#NewTimer) disables this change.
There are no runtime metrics for this change,
This setting may be removed in a future release, Go 1.27 at the earliest.
Go 1.23 changed the mode bits reported by [`os.Lstat`](/pkg/os#Lstat) and [`os.Stat`](/pkg/os#Stat)
for reparse points, which can be controlled with the `winsymlink` setting.
As of Go 1.23 (`winsymlink=1`), mount points no longer have [`os.ModeSymlink`](/pkg/os#ModeSymlink)
set, and reparse points that are not symlinks, Unix sockets, or dedup files now
always have [`os.ModeIrregular`](/pkg/os#ModeIrregular) set. As a result of these changes,
[`filepath.EvalSymlinks`](/pkg/path/filepath#EvalSymlinks) no longer evaluates
mount points, which was a source of many inconsistencies and bugs.
At previous versions (`winsymlink=0`), mount points are treated as symlinks,
and other reparse points with non-default [`os.ModeType`](/pkg/os#ModeType) bits
(such as [`os.ModeDir`](/pkg/os#ModeDir)) do not have the `ModeIrregular` bit set.
Go 1.23 changed [`os.Readlink`](/pkg/os#Readlink) and [`filepath.EvalSymlinks`](/pkg/path/filepath#EvalSymlinks)
to avoid trying to normalize volumes to drive letters, which was not always even possible.
This behavior is controlled by the `winreadlinkvolume` setting.
For Go 1.23, it defaults to `winreadlinkvolume=1`.
Previous versions default to `winreadlinkvolume=0`.
Go 1.23 enabled the experimental post-quantum key exchange mechanism
X25519Kyber768Draft00 by default. The default can be reverted using the
[`tlskyber` setting](/pkg/crypto/tls/#Config.CurvePreferences).
Go 1.23 changed the behavior of
[crypto/x509.ParseCertificate](/pkg/crypto/x509/#ParseCertificate) to reject
serial numbers that are negative. This change can be reverted with
the [`x509negativeserial` setting](/pkg/crypto/x509/#ParseCertificate).
Go 1.23 re-enabled support in html/template for ECMAScript 6 template literals by default.
The [`jstmpllitinterp` setting](/pkg/html/template#hdr-Security_Model) no longer has
any effect.
Go 1.23 changed the default TLS cipher suites used by clients and servers when
not explicitly configured, removing 3DES cipher suites. The default can be reverted
using the [`tls3des` setting](/pkg/crypto/tls/#Config.CipherSuites).
Go 1.23 changed the behavior of [`tls.X509KeyPair`](/pkg/crypto/tls#X509KeyPair)
and [`tls.LoadX509KeyPair`](/pkg/crypto/tls#LoadX509KeyPair) to populate the
Leaf field of the returned [`tls.Certificate`](/pkg/crypto/tls#Certificate).
This behavior is controlled by the `x509keypairleaf` setting. For Go 1.23, it
defaults to `x509keypairleaf=1`. Previous versions default to
`x509keypairleaf=0`.
Go 1.23 changed
[`net/http.ServeContent`](/pkg/net/http#ServeContent),
[`net/http.ServeFile`](/pkg/net/http#ServeFile), and
[`net/http.ServeFS`](/pkg/net/http#ServeFS) to
remove Cache-Control, Content-Encoding, Etag, and Last-Modified headers
when serving an error. This behavior is controlled by
the [`httpservecontentkeepheaders` setting](/pkg/net/http#ServeContent).
Using `httpservecontentkeepheaders=1` restores the pre-Go 1.23 behavior.
### Go 1.22
Go 1.22 adds a configurable limit to control the maximum acceptable RSA key size
that can be used in TLS handshakes, controlled by the [`tlsmaxrsasize` setting](/pkg/crypto/tls#Conn.Handshake).
that can be used in TLS handshakes, controlled by the [`tlsmaxrsasize`setting](/pkg/crypto/tls#Conn.Handshake).
The default is tlsmaxrsasize=8192, limiting RSA to 8192-bit keys. To avoid
denial of service attacks, this setting and default was backported to Go
1.19.13, Go 1.20.8, and Go 1.21.1.
Go 1.22 made it an error for a request or response read by a net/http
client or server to have an empty Content-Length header.
This behavior is controlled by the `httplaxcontentlength` setting.
Go 1.22 changed the behavior of ServeMux to accept extended
patterns and unescape both patterns and request paths by segment.
This behavior can be controlled by the
[`httpmuxgo121` setting](/pkg/net/http/#ServeMux).
Go 1.22 added the [Alias type](/pkg/go/types#Alias) to [go/types](/pkg/go/types)
for the explicit representation of [type aliases](/ref/spec#Type_declarations).
Whether the type checker produces `Alias` types or not is controlled by the
[`gotypesalias` setting](/pkg/go/types#Alias).
For Go 1.22 it defaults to `gotypesalias=0`.
For Go 1.23, `gotypesalias=1` will become the default.
This setting will be removed in a future release, Go 1.27 at the earliest.
Go 1.22 changed the default minimum TLS version supported by both servers
and clients to TLS 1.2. The default can be reverted to TLS 1.0 using the
[`tls10server` setting](/pkg/crypto/tls/#Config).
Go 1.22 changed the default TLS cipher suites used by clients and servers when
not explicitly configured, removing the cipher suites which used RSA based key
exchange. The default can be reverted using the [`tlsrsakex` setting](/pkg/crypto/tls/#Config).
Go 1.22 disabled
[`ConnectionState.ExportKeyingMaterial`](/pkg/crypto/tls/#ConnectionState.ExportKeyingMaterial)
when the connection supports neither TLS 1.3 nor Extended Master Secret
(implemented in Go 1.21). It can be reenabled with the [`tlsunsafeekm`
setting](/pkg/crypto/tls/#ConnectionState.ExportKeyingMaterial).
Go 1.22 changed how the runtime interacts with transparent huge pages on Linux.
In particular, a common default Linux kernel configuration can result in
significant memory overheads, and Go 1.22 no longer works around this default.
@@ -324,22 +147,6 @@ should adjust their Linux configuration according to the recommendations in the
[GC guide](/doc/gc-guide#Linux_transparent_huge_pages), or switch to a Linux
distribution that disables transparent huge pages altogether.
Go 1.22 added contention on runtime-internal locks to the [`mutex`
profile](/pkg/runtime/pprof#Profile). Contention on these locks is always
reported at `runtime._LostContendedRuntimeLock`. Complete stack traces of
runtime locks can be enabled with the [`runtimecontentionstacks`
setting](/pkg/runtime#hdr-Environment_Variable). These stack traces have
non-standard semantics, see setting documentation for details.
Go 1.22 added a new [`crypto/x509.Certificate`](/pkg/crypto/x509/#Certificate)
field, [`Policies`](/pkg/crypto/x509/#Certificate.Policies), which supports
certificate policy OIDs with components larger than 31 bits. By default this
field is only used during parsing, when it is populated with policy OIDs, but
not used during marshaling. It can be used to marshal these larger OIDs, instead
of the existing PolicyIdentifiers field, by using the
[`x509usepolicies` setting.](/pkg/crypto/x509/#CreateCertificate).
### Go 1.21
Go 1.21 made it a run-time error to call `panic` with a nil interface value,
@@ -394,18 +201,11 @@ Go 1.19 made it an error for path lookups to resolve to binaries in the current
controlled by the [`execerrdot` setting](/pkg/os/exec#hdr-Executables_in_the_current_directory).
There is no plan to remove this setting.
Go 1.19 started sending EDNS0 additional headers on DNS requests.
This can reportedly break the DNS server provided on some routers,
such as CenturyLink Zyxel C3000Z.
This can be changed by the [`netedns0` setting](/pkg/net#hdr-Name_Resolution).
This setting is available in Go 1.21.12, Go 1.22.5, Go 1.23, and later.
There is no plan to remove this setting.
### Go 1.18
Go 1.18 removed support for SHA1 in most X.509 certificates,
controlled by the [`x509sha1` setting](/pkg/crypto/x509#InsecureAlgorithmError).
This setting was removed in Go 1.24.
controlled by the [`x509sha1` setting](/crypto/x509#InsecureAlgorithmError).
This setting will be removed in a future release, Go 1.22 at the earliest.
### Go 1.10

View File

@@ -1,14 +0,0 @@
<!--
NOTE: In this document and others in this directory, the convention is to
set fixed-width phrases with non-fixed-width spaces, as in
`hello` `world`.
-->
<style>
main ul li { margin: 0.5em 0; }
</style>
## DRAFT RELEASE NOTES — Introduction to Go 1.N {#introduction}
**Go 1.N is not yet released. These are work-in-progress release notes.
Go 1.N is expected to be released in {Month} {Year}.**

View File

@@ -1,3 +0,0 @@
## Changes to the language {#language}

View File

@@ -1,6 +0,0 @@
## Tools {#tools}
### Go command {#go-command}
### Cgo {#cgo}

View File

@@ -1 +0,0 @@
## Runtime {#runtime}

View File

@@ -1,7 +0,0 @@
## Compiler {#compiler}
## Assembler {#assembler}
## Linker {#linker}

View File

@@ -1,2 +0,0 @@
## Standard library {#library}

View File

@@ -1,3 +0,0 @@
### Minor changes to the library {#minor_library_changes}

View File

@@ -1 +0,0 @@
API changes and other small changes to the standard library go here.

View File

@@ -1,2 +0,0 @@
## Ports {#ports}

View File

@@ -1,46 +0,0 @@
# Copyright 2024 The Go Authors. All rights reserved.
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file.
# Rules for building and testing new FIPS snapshots.
# For example:
#
# make v1.2.3.zip
# make v1.2.3.test
#
# and then if changes are needed, check them into master
# and run 'make v1.2.3.rm' and repeat.
#
# Note that once published a snapshot zip file should never
# be modified. We record the sha256 hashes of the zip files
# in fips140.sum, and the cmd/go/internal/fips140 test checks
# that the zips match.
#
# When the zip file is finalized, run 'make updatesum' to update
# fips140.sum.
default:
@echo nothing to make
# make v1.2.3.zip builds a v1.2.3.zip file
# from the current origin/master.
# copy and edit the 'go run' command by hand to use a different branch.
v%.zip:
git fetch origin master
go run ../../src/cmd/go/internal/fips140/mkzip.go -b master v$*
# normally mkzip refuses to overwrite an existing zip file.
# make v1.2.3.rm removes the zip file and and unpacked
# copy from the module cache.
v%.rm:
rm -f v$*.zip
chmod -R u+w $$(go env GOMODCACHE)/golang.org/fips140@v$* 2>/dev/null || true
rm -rf $$(go env GOMODCACHE)/golang.org/fips140@v$*
# make v1.2.3.test runs the crypto tests using that snapshot.
v%.test:
GOFIPS140=v$* go test -short crypto...
# make updatesum updates the fips140.sum file.
updatesum:
go test cmd/go/internal/fips140 -update

View File

@@ -1,9 +0,0 @@
This directory holds snapshots of the crypto/internal/fips140 tree
that are being validated and certified for FIPS-140 use.
The file x.txt (for example, inprocess.txt, certified.txt)
defines the meaning of the FIPS version alias x, listing
the exact version to use.
The zip files are created by cmd/go/internal/fips140/mkzip.go.
The fips140.sum file lists checksums for the zip files.
See the Makefile for recipes.

View File

@@ -1,11 +0,0 @@
# SHA256 checksums of snapshot zip files in this directory.
# These checksums are included in the FIPS security policy
# (validation instructions sent to the lab) and MUST NOT CHANGE.
# That is, the zip files themselves must not change.
#
# It is okay to add new zip files to the list, and it is okay to
# remove zip files from the list when they are removed from
# this directory. To update this file:
#
# go test cmd/go/internal/fips140 -update
#

View File

@@ -31,7 +31,7 @@ import (
)
func usage() {
fmt.Fprintf(os.Stderr, "usage: go run mkzip.go zoneinfo.zip\n")
fmt.Fprintf(os.Stderr, "usage: go run mkzip.go ../../zoneinfo.zip\n")
os.Exit(2)
}

View File

@@ -24,8 +24,8 @@
# in the CL match the update.bash in the CL.
# Versions to use.
CODE=2024b
DATA=2024b
CODE=2023c
DATA=2023c
set -e

Binary file not shown.

View File

@@ -3,4 +3,4 @@
// tests and tools.
module misc
go 1.22
go 1.21

View File

@@ -204,7 +204,6 @@ func runMain() (int, error) {
`; export GOPROXY=` + os.Getenv("GOPROXY") +
`; export GOCACHE="` + deviceRoot + `/gocache"` +
`; export PATH="` + deviceGoroot + `/bin":$PATH` +
`; export HOME="` + deviceRoot + `/home"` +
`; cd "` + deviceCwd + `"` +
"; '" + deviceBin + "' " + strings.Join(os.Args[2:], " ")
code, err := adbRun(cmd)

View File

@@ -1,22 +1,19 @@
#!/bin/sh
# This script configures clang to target the iOS simulator. If you'd like to
# build for real iOS devices, change SDK to "iphoneos" and PLATFORM to "ios".
# This uses the latest available iOS SDK, which is recommended. To select a
# specific SDK, run 'xcodebuild -showsdks' to see the available SDKs and replace
# iphonesimulator with one of them.
SDK=iphonesimulator
PLATFORM=ios-simulator
# This uses the latest available iOS SDK, which is recommended.
# To select a specific SDK, run 'xcodebuild -showsdks'
# to see the available SDKs and replace iphoneos with one of them.
if [ "$GOARCH" == "arm64" ]; then
SDK=iphoneos
PLATFORM=ios
CLANGARCH="arm64"
else
SDK=iphonesimulator
PLATFORM=ios-simulator
CLANGARCH="x86_64"
fi
SDK_PATH=`xcrun --sdk $SDK --show-sdk-path`
export IPHONEOS_DEPLOYMENT_TARGET=5.1
# cmd/cgo doesn't support llvm-gcc-4.2, so we have to use clang.
CLANG=`xcrun --sdk $SDK --find clang`

View File

@@ -3,6 +3,7 @@
// license that can be found in the LICENSE file.
//go:build ignore
// +build ignore
// detect attempts to autodetect the correct
// values of the environment variables

View File

@@ -1,21 +1,44 @@
// Copyright 2024 The Go Authors. All rights reserved.
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// This program can be used as go_ios_$GOARCH_exec by the Go tool. It executes
// binaries on the iOS Simulator using the XCode toolchain.
// This program can be used as go_ios_$GOARCH_exec by the Go tool.
// It executes binaries on an iOS device using the XCode toolchain
// and the ios-deploy program: https://github.com/phonegap/ios-deploy
//
// This script supports an extra flag, -lldb, that pauses execution
// just before the main program begins and allows the user to control
// the remote lldb session. This flag is appended to the end of the
// script's arguments and is not passed through to the underlying
// binary.
//
// This script requires that three environment variables be set:
//
// GOIOS_DEV_ID: The codesigning developer id or certificate identifier
// GOIOS_APP_ID: The provisioning app id prefix. Must support wildcard app ids.
// GOIOS_TEAM_ID: The team id that owns the app id prefix.
//
// $GOROOT/misc/ios contains a script, detect.go, that attempts to autodetect these.
package main
import (
"bytes"
"encoding/xml"
"errors"
"fmt"
"go/build"
"io"
"log"
"net"
"os"
"os/exec"
"os/signal"
"path/filepath"
"runtime"
"strconv"
"strings"
"syscall"
"time"
)
const debug = false
@@ -87,8 +110,18 @@ func runMain() (int, error) {
return 1, err
}
err = runOnSimulator(appdir)
if goarch := os.Getenv("GOARCH"); goarch == "arm64" {
err = runOnDevice(appdir)
} else {
err = runOnSimulator(appdir)
}
if err != nil {
// If the lldb driver completed with an exit code, use that.
if err, ok := err.(*exec.ExitError); ok {
if ws, ok := err.Sys().(interface{ ExitStatus() int }); ok {
return ws.ExitStatus(), nil
}
}
return 1, err
}
return 0, nil
@@ -102,6 +135,61 @@ func runOnSimulator(appdir string) error {
return runSimulator(appdir, bundleID, os.Args[2:])
}
func runOnDevice(appdir string) error {
// e.g. B393DDEB490947F5A463FD074299B6C0AXXXXXXX
devID = getenv("GOIOS_DEV_ID")
// e.g. Z8B3JBXXXX.org.golang.sample, Z8B3JBXXXX prefix is available at
// https://developer.apple.com/membercenter/index.action#accountSummary as Team ID.
appID = getenv("GOIOS_APP_ID")
// e.g. Z8B3JBXXXX, available at
// https://developer.apple.com/membercenter/index.action#accountSummary as Team ID.
teamID = getenv("GOIOS_TEAM_ID")
// Device IDs as listed with ios-deploy -c.
deviceID = os.Getenv("GOIOS_DEVICE_ID")
if _, id, ok := strings.Cut(appID, "."); ok {
bundleID = id
}
if err := signApp(appdir); err != nil {
return err
}
if err := uninstallDevice(bundleID); err != nil {
return err
}
if err := installDevice(appdir); err != nil {
return err
}
if err := mountDevImage(); err != nil {
return err
}
// Kill any hanging debug bridges that might take up port 3222.
exec.Command("killall", "idevicedebugserverproxy").Run()
closer, err := startDebugBridge()
if err != nil {
return err
}
defer closer()
return runDevice(appdir, bundleID, os.Args[2:])
}
func getenv(envvar string) string {
s := os.Getenv(envvar)
if s == "" {
log.Fatalf("%s not set\nrun $GOROOT/misc/ios/detect.go to attempt to autodetect", envvar)
}
return s
}
func assembleApp(appdir, bin string) error {
if err := os.MkdirAll(appdir, 0755); err != nil {
return err
@@ -129,6 +217,236 @@ func assembleApp(appdir, bin string) error {
return nil
}
func signApp(appdir string) error {
entitlementsPath := filepath.Join(tmpdir, "Entitlements.plist")
cmd := exec.Command(
"codesign",
"-f",
"-s", devID,
"--entitlements", entitlementsPath,
appdir,
)
if debug {
log.Println(strings.Join(cmd.Args, " "))
}
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
return fmt.Errorf("codesign: %v", err)
}
return nil
}
// mountDevImage ensures a developer image is mounted on the device.
// The image contains the device lldb server for idevicedebugserverproxy
// to connect to.
func mountDevImage() error {
// Check for existing mount.
cmd := idevCmd(exec.Command("ideviceimagemounter", "-l", "-x"))
out, err := cmd.CombinedOutput()
if err != nil {
os.Stderr.Write(out)
return fmt.Errorf("ideviceimagemounter: %v", err)
}
var info struct {
Dict struct {
Data []byte `xml:",innerxml"`
} `xml:"dict"`
}
if err := xml.Unmarshal(out, &info); err != nil {
return fmt.Errorf("mountDevImage: failed to decode mount information: %v", err)
}
dict, err := parsePlistDict(info.Dict.Data)
if err != nil {
return fmt.Errorf("mountDevImage: failed to parse mount information: %v", err)
}
if dict["ImagePresent"] == "true" && dict["Status"] == "Complete" {
return nil
}
// Some devices only give us an ImageSignature key.
if _, exists := dict["ImageSignature"]; exists {
return nil
}
// No image is mounted. Find a suitable image.
imgPath, err := findDevImage()
if err != nil {
return err
}
sigPath := imgPath + ".signature"
cmd = idevCmd(exec.Command("ideviceimagemounter", imgPath, sigPath))
if out, err := cmd.CombinedOutput(); err != nil {
os.Stderr.Write(out)
return fmt.Errorf("ideviceimagemounter: %v", err)
}
return nil
}
// findDevImage use the device iOS version and build to locate a suitable
// developer image.
func findDevImage() (string, error) {
cmd := idevCmd(exec.Command("ideviceinfo"))
out, err := cmd.Output()
if err != nil {
return "", fmt.Errorf("ideviceinfo: %v", err)
}
var iosVer, buildVer string
lines := bytes.Split(out, []byte("\n"))
for _, line := range lines {
key, val, ok := strings.Cut(string(line), ": ")
if !ok {
continue
}
switch key {
case "ProductVersion":
iosVer = val
case "BuildVersion":
buildVer = val
}
}
if iosVer == "" || buildVer == "" {
return "", errors.New("failed to parse ideviceinfo output")
}
verSplit := strings.Split(iosVer, ".")
if len(verSplit) > 2 {
// Developer images are specific to major.minor ios version.
// Cut off the patch version.
iosVer = strings.Join(verSplit[:2], ".")
}
sdkBase := "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport"
patterns := []string{fmt.Sprintf("%s (%s)", iosVer, buildVer), fmt.Sprintf("%s (*)", iosVer), fmt.Sprintf("%s*", iosVer)}
for _, pattern := range patterns {
matches, err := filepath.Glob(filepath.Join(sdkBase, pattern, "DeveloperDiskImage.dmg"))
if err != nil {
return "", fmt.Errorf("findDevImage: %v", err)
}
if len(matches) > 0 {
return matches[0], nil
}
}
return "", fmt.Errorf("failed to find matching developer image for iOS version %s build %s", iosVer, buildVer)
}
// startDebugBridge ensures that the idevicedebugserverproxy runs on
// port 3222.
func startDebugBridge() (func(), error) {
errChan := make(chan error, 1)
cmd := idevCmd(exec.Command("idevicedebugserverproxy", "3222"))
var stderr bytes.Buffer
cmd.Stderr = &stderr
if err := cmd.Start(); err != nil {
return nil, fmt.Errorf("idevicedebugserverproxy: %v", err)
}
go func() {
if err := cmd.Wait(); err != nil {
if _, ok := err.(*exec.ExitError); ok {
errChan <- fmt.Errorf("idevicedebugserverproxy: %s", stderr.Bytes())
} else {
errChan <- fmt.Errorf("idevicedebugserverproxy: %v", err)
}
}
errChan <- nil
}()
closer := func() {
cmd.Process.Kill()
<-errChan
}
// Dial localhost:3222 to ensure the proxy is ready.
delay := time.Second / 4
for attempt := 0; attempt < 5; attempt++ {
conn, err := net.DialTimeout("tcp", "localhost:3222", 5*time.Second)
if err == nil {
conn.Close()
return closer, nil
}
select {
case <-time.After(delay):
delay *= 2
case err := <-errChan:
return nil, err
}
}
closer()
return nil, errors.New("failed to set up idevicedebugserverproxy")
}
// findDeviceAppPath returns the device path to the app with the
// given bundle ID. It parses the output of ideviceinstaller -l -o xml,
// looking for the bundle ID and the corresponding Path value.
func findDeviceAppPath(bundleID string) (string, error) {
cmd := idevCmd(exec.Command("ideviceinstaller", "-l", "-o", "xml"))
out, err := cmd.CombinedOutput()
if err != nil {
os.Stderr.Write(out)
return "", fmt.Errorf("ideviceinstaller: -l -o xml %v", err)
}
var list struct {
Apps []struct {
Data []byte `xml:",innerxml"`
} `xml:"array>dict"`
}
if err := xml.Unmarshal(out, &list); err != nil {
return "", fmt.Errorf("failed to parse ideviceinstaller output: %v", err)
}
for _, app := range list.Apps {
values, err := parsePlistDict(app.Data)
if err != nil {
return "", fmt.Errorf("findDeviceAppPath: failed to parse app dict: %v", err)
}
if values["CFBundleIdentifier"] == bundleID {
if path, ok := values["Path"]; ok {
return path, nil
}
}
}
return "", fmt.Errorf("failed to find device path for bundle: %s", bundleID)
}
// Parse an xml encoded plist. Plist values are mapped to string.
func parsePlistDict(dict []byte) (map[string]string, error) {
d := xml.NewDecoder(bytes.NewReader(dict))
values := make(map[string]string)
var key string
var hasKey bool
for {
tok, err := d.Token()
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
if tok, ok := tok.(xml.StartElement); ok {
if tok.Name.Local == "key" {
if err := d.DecodeElement(&key, &tok); err != nil {
return nil, err
}
hasKey = true
} else if hasKey {
var val string
var err error
switch n := tok.Name.Local; n {
case "true", "false":
// Bools are represented as <true/> and <false/>.
val = n
err = d.Skip()
default:
err = d.DecodeElement(&val, &tok)
}
if err != nil {
return nil, err
}
values[key] = val
hasKey = false
} else {
if err := d.Skip(); err != nil {
return nil, err
}
}
}
}
return values, nil
}
func installSimulator(appdir string) error {
cmd := exec.Command(
"xcrun", "simctl", "install",
@@ -142,20 +460,138 @@ func installSimulator(appdir string) error {
return nil
}
func runSimulator(appdir, bundleID string, args []string) error {
xcrunArgs := []string{"simctl", "spawn",
"booted",
appdir + "/gotest",
func uninstallDevice(bundleID string) error {
cmd := idevCmd(exec.Command(
"ideviceinstaller",
"-U", bundleID,
))
if out, err := cmd.CombinedOutput(); err != nil {
os.Stderr.Write(out)
return fmt.Errorf("ideviceinstaller -U %q: %s", bundleID, err)
}
xcrunArgs = append(xcrunArgs, args...)
cmd := exec.Command("xcrun", xcrunArgs...)
cmd.Stdout, cmd.Stderr = os.Stdout, os.Stderr
err := cmd.Run()
return nil
}
func installDevice(appdir string) error {
attempt := 0
for {
cmd := idevCmd(exec.Command(
"ideviceinstaller",
"-i", appdir,
))
if out, err := cmd.CombinedOutput(); err != nil {
// Sometimes, installing the app fails for some reason.
// Give the device a few seconds and try again.
if attempt < 5 {
time.Sleep(5 * time.Second)
attempt++
continue
}
os.Stderr.Write(out)
return fmt.Errorf("ideviceinstaller -i %q: %v (%d attempts)", appdir, err, attempt)
}
return nil
}
}
func idevCmd(cmd *exec.Cmd) *exec.Cmd {
if deviceID != "" {
// Inject -u device_id after the executable, but before the arguments.
args := []string{cmd.Args[0], "-u", deviceID}
cmd.Args = append(args, cmd.Args[1:]...)
}
return cmd
}
func runSimulator(appdir, bundleID string, args []string) error {
cmd := exec.Command(
"xcrun", "simctl", "launch",
"--wait-for-debugger",
"booted",
bundleID,
)
out, err := cmd.CombinedOutput()
if err != nil {
os.Stderr.Write(out)
return fmt.Errorf("xcrun simctl launch booted %q: %v", bundleID, err)
}
var processID int
var ignore string
if _, err := fmt.Sscanf(string(out), "%s %d", &ignore, &processID); err != nil {
return fmt.Errorf("runSimulator: couldn't find processID from `simctl launch`: %v (%q)", err, out)
}
_, err = runLLDB("ios-simulator", appdir, strconv.Itoa(processID), args)
return err
}
return nil
func runDevice(appdir, bundleID string, args []string) error {
attempt := 0
for {
// The device app path reported by the device might be stale, so retry
// the lookup of the device path along with the lldb launching below.
deviceapp, err := findDeviceAppPath(bundleID)
if err != nil {
// The device app path might not yet exist for a newly installed app.
if attempt == 5 {
return err
}
attempt++
time.Sleep(5 * time.Second)
continue
}
out, err := runLLDB("remote-ios", appdir, deviceapp, args)
// If the program was not started it can be retried without papering over
// real test failures.
started := bytes.HasPrefix(out, []byte("lldb: running program"))
if started || err == nil || attempt == 5 {
return err
}
// Sometimes, the app was not yet ready to launch or the device path was
// stale. Retry.
attempt++
time.Sleep(5 * time.Second)
}
}
func runLLDB(target, appdir, deviceapp string, args []string) ([]byte, error) {
var env []string
for _, e := range os.Environ() {
// Don't override TMPDIR, HOME, GOCACHE on the device.
if strings.HasPrefix(e, "TMPDIR=") || strings.HasPrefix(e, "HOME=") || strings.HasPrefix(e, "GOCACHE=") {
continue
}
env = append(env, e)
}
lldb := exec.Command(
"python",
"-", // Read script from stdin.
target,
appdir,
deviceapp,
)
lldb.Args = append(lldb.Args, args...)
lldb.Env = env
lldb.Stdin = strings.NewReader(lldbDriver)
lldb.Stdout = os.Stdout
var out bytes.Buffer
lldb.Stderr = io.MultiWriter(&out, os.Stderr)
err := lldb.Start()
if err == nil {
// Forward SIGQUIT to the lldb driver which in turn will forward
// to the running program.
sigs := make(chan os.Signal, 1)
signal.Notify(sigs, syscall.SIGQUIT)
proc := lldb.Process
go func() {
for sig := range sigs {
proc.Signal(sig)
}
}()
err = lldb.Wait()
signal.Stop(sigs)
close(sigs)
}
return out.Bytes(), err
}
func copyLocalDir(dst, src string) error {
@@ -364,3 +800,112 @@ const resourceRules = `<?xml version="1.0" encoding="UTF-8"?>
</dict>
</plist>
`
const lldbDriver = `
import sys
import os
import signal
platform, exe, device_exe_or_pid, args = sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4:]
env = []
for k, v in os.environ.items():
env.append(k + "=" + v)
sys.path.append('/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python')
import lldb
debugger = lldb.SBDebugger.Create()
debugger.SetAsync(True)
debugger.SkipLLDBInitFiles(True)
err = lldb.SBError()
target = debugger.CreateTarget(exe, None, platform, True, err)
if not target.IsValid() or not err.Success():
sys.stderr.write("lldb: failed to setup up target: %s\n" % (err))
sys.exit(1)
listener = debugger.GetListener()
if platform == 'remote-ios':
target.modules[0].SetPlatformFileSpec(lldb.SBFileSpec(device_exe_or_pid))
process = target.ConnectRemote(listener, 'connect://localhost:3222', None, err)
else:
process = target.AttachToProcessWithID(listener, int(device_exe_or_pid), err)
if not err.Success():
sys.stderr.write("lldb: failed to connect to remote target %s: %s\n" % (device_exe_or_pid, err))
sys.exit(1)
# Don't stop on signals.
sigs = process.GetUnixSignals()
for i in range(0, sigs.GetNumSignals()):
sig = sigs.GetSignalAtIndex(i)
sigs.SetShouldStop(sig, False)
sigs.SetShouldNotify(sig, False)
event = lldb.SBEvent()
running = False
prev_handler = None
def signal_handler(signal, frame):
process.Signal(signal)
def run_program():
# Forward SIGQUIT to the program.
prev_handler = signal.signal(signal.SIGQUIT, signal_handler)
# Tell the Go driver that the program is running and should not be retried.
sys.stderr.write("lldb: running program\n")
running = True
# Process is stopped at attach/launch. Let it run.
process.Continue()
if platform != 'remote-ios':
# For the local emulator the program is ready to run.
# For remote device runs, we need to wait for eStateConnected,
# below.
run_program()
while True:
if not listener.WaitForEvent(1, event):
continue
if not lldb.SBProcess.EventIsProcessEvent(event):
continue
if running:
# Pass through stdout and stderr.
while True:
out = process.GetSTDOUT(8192)
if not out:
break
sys.stdout.write(out)
while True:
out = process.GetSTDERR(8192)
if not out:
break
sys.stderr.write(out)
state = process.GetStateFromEvent(event)
if state in [lldb.eStateCrashed, lldb.eStateDetached, lldb.eStateUnloaded, lldb.eStateExited]:
if running:
signal.signal(signal.SIGQUIT, prev_handler)
break
elif state == lldb.eStateConnected:
if platform == 'remote-ios':
process.RemoteLaunch(args, env, None, None, None, None, 0, False, err)
if not err.Success():
sys.stderr.write("lldb: failed to launch remote process: %s\n" % (err))
process.Kill()
debugger.Terminate()
sys.exit(1)
run_program()
exitStatus = process.GetExitStatus()
exitDesc = process.GetExitDescription()
process.Kill()
debugger.Terminate()
if exitStatus == 0 and exitDesc is not None:
# Ensure tests fail when killed by a signal.
exitStatus = 123
sys.exit(exitStatus)
`

View File

@@ -14,7 +14,7 @@ case "$GOWASIRUNTIME" in
exec wazero run -mount /:/ -env-inherit -cachedir "${TMPDIR:-/tmp}"/wazero ${GOWASIRUNTIMEARGS:-} "$1" "${@:2}"
;;
"wasmtime" | "")
exec wasmtime run --dir=/ --env PWD="$PWD" --env PATH="$PATH" -W max-wasm-stack=1048576 ${GOWASIRUNTIMEARGS:-} "$1" "${@:2}"
exec wasmtime run --dir=/ --env PWD="$PWD" --env PATH="$PATH" --max-wasm-stack 1048576 ${GOWASIRUNTIMEARGS:-} "$1" -- "${@:2}"
;;
*)
echo "Unknown Go WASI runtime specified: $GOWASIRUNTIME"

View File

@@ -17,7 +17,7 @@ license that can be found in the LICENSE file.
<script src="https://cdn.jsdelivr.net/npm/text-encoding@0.7.0/lib/encoding.min.js"></script>
(see https://caniuse.com/#feat=textencoder)
-->
<script src="../../lib/wasm/wasm_exec.js"></script>
<script src="wasm_exec.js"></script>
<script>
if (!WebAssembly.instantiateStreaming) { // polyfill
WebAssembly.instantiateStreaming = async (resp, importObject) => {

View File

@@ -14,7 +14,7 @@
if (!globalThis.fs) {
let outputBuf = "";
globalThis.fs = {
constants: { O_WRONLY: -1, O_RDWR: -1, O_CREAT: -1, O_TRUNC: -1, O_APPEND: -1, O_EXCL: -1, O_DIRECTORY: -1 }, // unused
constants: { O_WRONLY: -1, O_RDWR: -1, O_CREAT: -1, O_TRUNC: -1, O_APPEND: -1, O_EXCL: -1 }, // unused
writeSync(fd, buf) {
outputBuf += decoder.decode(buf);
const nl = outputBuf.lastIndexOf("\n");
@@ -73,14 +73,6 @@
}
}
if (!globalThis.path) {
globalThis.path = {
resolve(...pathSegments) {
return pathSegments.join("/");
}
}
}
if (!globalThis.crypto) {
throw new Error("globalThis.crypto is not available, polyfill required (crypto.getRandomValues only)");
}
@@ -216,16 +208,10 @@
return decoder.decode(new DataView(this._inst.exports.mem.buffer, saddr, len));
}
const testCallExport = (a, b) => {
this._inst.exports.testExport0();
return this._inst.exports.testExport(a, b);
}
const timeOrigin = Date.now() - performance.now();
this.importObject = {
_gotest: {
add: (a, b) => a + b,
callExport: testCallExport,
},
gojs: {
// Go's SP does not change as long as no Go code is running. Some operations (e.g. calls, getters and setters)

View File

@@ -11,7 +11,6 @@ if (process.argv.length < 3) {
globalThis.require = require;
globalThis.fs = require("fs");
globalThis.path = require("path");
globalThis.TextEncoder = require("util").TextEncoder;
globalThis.TextDecoder = require("util").TextDecoder;

View File

@@ -31,17 +31,13 @@ Maintaining vendor directories
Before updating vendor directories, ensure that module mode is enabled.
Make sure that GO111MODULE is not set in the environment, or that it is
set to 'on' or 'auto', and if you use a go.work file, set GOWORK=off.
Also, ensure that 'go env GOROOT' shows the root of this Go source
tree. Otherwise, the results are undefined. It's recommended to build
Go from source and use that 'go' binary to update its source tree.
set to 'on' or 'auto'.
Requirements may be added, updated, and removed with 'go get'.
The vendor directory may be updated with 'go mod vendor'.
A typical sequence might be:
cd src # or src/cmd
cd src
go get golang.org/x/net@master
go mod tidy
go mod vendor

View File

@@ -10,4 +10,4 @@ if [ ! -f make.bash ]; then
fi
. ./make.bash "$@" --no-banner
bash run.bash --no-rebuild
"$GOTOOLDIR/dist" banner # print build info
$GOTOOLDIR/dist banner # print build info

View File

@@ -15,7 +15,6 @@ import (
"fmt"
"internal/godebug"
"io/fs"
"maps"
"math"
"path"
"reflect"
@@ -613,7 +612,7 @@ func (fi headerFileInfo) String() string {
}
// sysStat, if non-nil, populates h from system-dependent fields of fi.
var sysStat func(fi fs.FileInfo, h *Header, doNameLookups bool) error
var sysStat func(fi fs.FileInfo, h *Header) error
const (
// Mode constants from the USTAR spec:
@@ -633,17 +632,13 @@ const (
c_ISSOCK = 0140000 // Socket
)
// FileInfoHeader creates a partially-populated [Header] from fi.
// FileInfoHeader creates a partially-populated Header from fi.
// If fi describes a symlink, FileInfoHeader records link as the link target.
// If fi describes a directory, a slash is appended to the name.
//
// Since fs.FileInfo's Name method only returns the base name of
// the file it describes, it may be necessary to modify Header.Name
// to provide the full path name of the file.
//
// If fi implements [FileInfoNames]
// Header.Gname and Header.Uname
// are provided by the methods of the interface.
func FileInfoHeader(fi fs.FileInfo, link string) (*Header, error) {
if fi == nil {
return nil, errors.New("archive/tar: FileInfo is nil")
@@ -697,45 +692,31 @@ func FileInfoHeader(fi fs.FileInfo, link string) (*Header, error) {
h.Gname = sys.Gname
h.AccessTime = sys.AccessTime
h.ChangeTime = sys.ChangeTime
h.Xattrs = maps.Clone(sys.Xattrs)
if sys.Xattrs != nil {
h.Xattrs = make(map[string]string)
for k, v := range sys.Xattrs {
h.Xattrs[k] = v
}
}
if sys.Typeflag == TypeLink {
// hard link
h.Typeflag = TypeLink
h.Size = 0
h.Linkname = sys.Linkname
}
h.PAXRecords = maps.Clone(sys.PAXRecords)
}
var doNameLookups = true
if iface, ok := fi.(FileInfoNames); ok {
doNameLookups = false
var err error
h.Gname, err = iface.Gname()
if err != nil {
return nil, err
}
h.Uname, err = iface.Uname()
if err != nil {
return nil, err
if sys.PAXRecords != nil {
h.PAXRecords = make(map[string]string)
for k, v := range sys.PAXRecords {
h.PAXRecords[k] = v
}
}
}
if sysStat != nil {
return h, sysStat(fi, h, doNameLookups)
return h, sysStat(fi, h)
}
return h, nil
}
// FileInfoNames extends [fs.FileInfo].
// Passing an instance of this to [FileInfoHeader] permits the caller
// to avoid a system-dependent name lookup by specifying the Uname and Gname directly.
type FileInfoNames interface {
fs.FileInfo
// Uname should give a user name.
Uname() (string, error)
// Gname should give a group name.
Gname() (string, error)
}
// isHeaderOnlyType checks if the given type flag is of the type that has no
// data section even if a size is specified.
func isHeaderOnlyType(flag byte) bool {
@@ -746,3 +727,10 @@ func isHeaderOnlyType(flag byte) bool {
return false
}
}
func min(a, b int64) int64 {
if a < b {
return a
}
return b
}

View File

@@ -33,7 +33,7 @@ import "strings"
// sub-second times | no | yes | no
// sparse files | no | yes | yes
//
// The table's upper portion shows the [Header] fields, where each format reports
// The table's upper portion shows the Header fields, where each format reports
// the maximum number of bytes allowed for each string field and
// the integer type used to store each numeric field
// (where timestamps are stored as the number of seconds since the Unix epoch).

View File

@@ -35,7 +35,7 @@ type fileReader interface {
WriteTo(io.Writer) (int64, error)
}
// NewReader creates a new [Reader] reading from r.
// NewReader creates a new Reader reading from r.
func NewReader(r io.Reader) *Reader {
return &Reader{r: r, curr: &regFileReader{r, 0}}
}
@@ -47,10 +47,10 @@ func NewReader(r io.Reader) *Reader {
//
// If Next encounters a non-local name (as defined by [filepath.IsLocal])
// and the GODEBUG environment variable contains `tarinsecurepath=0`,
// Next returns the header with an [ErrInsecurePath] error.
// Next returns the header with an ErrInsecurePath error.
// A future version of Go may introduce this behavior by default.
// Programs that want to accept non-local names can ignore
// the [ErrInsecurePath] error and use the returned header.
// the ErrInsecurePath error and use the returned header.
func (tr *Reader) Next() (*Header, error) {
if tr.err != nil {
return nil, tr.err
@@ -623,14 +623,14 @@ func readGNUSparseMap0x1(paxHdrs map[string]string) (sparseDatas, error) {
// Read reads from the current file in the tar archive.
// It returns (0, io.EOF) when it reaches the end of that file,
// until [Next] is called to advance to the next file.
// until Next is called to advance to the next file.
//
// If the current file is sparse, then the regions marked as a hole
// are read back as NUL-bytes.
//
// Calling Read on special types like [TypeLink], [TypeSymlink], [TypeChar],
// [TypeBlock], [TypeDir], and [TypeFifo] returns (0, [io.EOF]) regardless of what
// the [Header.Size] claims.
// Calling Read on special types like TypeLink, TypeSymlink, TypeChar,
// TypeBlock, TypeDir, and TypeFifo returns (0, io.EOF) regardless of what
// the Header.Size claims.
func (tr *Reader) Read(b []byte) (int, error) {
if tr.err != nil {
return 0, tr.err
@@ -811,7 +811,9 @@ func (sr sparseFileReader) physicalRemaining() int64 {
type zeroReader struct{}
func (zeroReader) Read(b []byte) (int, error) {
clear(b)
for i := range b {
b[i] = 0
}
return len(b), nil
}

View File

@@ -7,16 +7,14 @@ package tar
import (
"bytes"
"compress/bzip2"
"crypto/md5"
"errors"
"fmt"
"hash/crc32"
"io"
"maps"
"math"
"os"
"path"
"reflect"
"slices"
"strconv"
"strings"
"testing"
@@ -27,7 +25,7 @@ func TestReader(t *testing.T) {
vectors := []struct {
file string // Test input file
headers []*Header // Expected output headers
chksums []string // CRC32 checksum of files, leave as nil if not checked
chksums []string // MD5 checksum of files, leave as nil if not checked
err error // Expected error to occur
}{{
file: "testdata/gnu.tar",
@@ -55,8 +53,8 @@ func TestReader(t *testing.T) {
Format: FormatGNU,
}},
chksums: []string{
"6cbd88fc",
"ddac04b3",
"e38b27eaccb4391bdec553a7f3ae6b2f",
"c65bd2e50a56a2138bf1716f2fd56fe9",
},
}, {
file: "testdata/sparse-formats.tar",
@@ -149,11 +147,11 @@ func TestReader(t *testing.T) {
Format: FormatGNU,
}},
chksums: []string{
"5375e1d2",
"5375e1d2",
"5375e1d2",
"5375e1d2",
"8eb179ba",
"6f53234398c2449fe67c1812d993012f",
"6f53234398c2449fe67c1812d993012f",
"6f53234398c2449fe67c1812d993012f",
"6f53234398c2449fe67c1812d993012f",
"b0061974914468de549a2af8ced10316",
},
}, {
file: "testdata/star.tar",
@@ -270,7 +268,7 @@ func TestReader(t *testing.T) {
Format: FormatPAX,
}},
chksums: []string{
"5fd7e86a",
"0afb597b283fe61b5d4879669a350556",
},
}, {
file: "testdata/pax-records.tar",
@@ -657,7 +655,7 @@ func TestReader(t *testing.T) {
if v.chksums == nil {
continue
}
h := crc32.NewIEEE()
h := md5.New()
_, err = io.CopyBuffer(h, tr, rdbuf) // Effectively an incremental read
if err != nil {
break
@@ -1019,7 +1017,7 @@ func TestParsePAX(t *testing.T) {
for i, v := range vectors {
r := strings.NewReader(v.in)
got, err := parsePAX(r)
if !maps.Equal(got, v.want) && !(len(got) == 0 && len(v.want) == 0) {
if !reflect.DeepEqual(got, v.want) && !(len(got) == 0 && len(v.want) == 0) {
t.Errorf("test %d, parsePAX():\ngot %v\nwant %v", i, got, v.want)
}
if ok := err == nil; ok != v.ok {
@@ -1136,7 +1134,7 @@ func TestReadOldGNUSparseMap(t *testing.T) {
v.input = v.input[copy(blk[:], v.input):]
tr := Reader{r: bytes.NewReader(v.input)}
got, err := tr.readOldGNUSparseMap(&hdr, &blk)
if !slices.Equal(got, v.wantMap) {
if !equalSparseEntries(got, v.wantMap) {
t.Errorf("test %d, readOldGNUSparseMap(): got %v, want %v", i, got, v.wantMap)
}
if err != v.wantErr {
@@ -1327,7 +1325,7 @@ func TestReadGNUSparsePAXHeaders(t *testing.T) {
r := strings.NewReader(v.inputData + "#") // Add canary byte
tr := Reader{curr: &regFileReader{r, int64(r.Len())}}
got, err := tr.readGNUSparsePAXHeaders(&hdr)
if !slices.Equal(got, v.wantMap) {
if !equalSparseEntries(got, v.wantMap) {
t.Errorf("test %d, readGNUSparsePAXHeaders(): got %v, want %v", i, got, v.wantMap)
}
if err != v.wantErr {

View File

@@ -23,30 +23,30 @@ func init() {
// The downside is that renaming uname or gname by the OS never takes effect.
var userMap, groupMap sync.Map // map[int]string
func statUnix(fi fs.FileInfo, h *Header, doNameLookups bool) error {
func statUnix(fi fs.FileInfo, h *Header) error {
sys, ok := fi.Sys().(*syscall.Stat_t)
if !ok {
return nil
}
h.Uid = int(sys.Uid)
h.Gid = int(sys.Gid)
if doNameLookups {
// Best effort at populating Uname and Gname.
// The os/user functions may fail for any number of reasons
// (not implemented on that platform, cgo not enabled, etc).
if u, ok := userMap.Load(h.Uid); ok {
h.Uname = u.(string)
} else if u, err := user.LookupId(strconv.Itoa(h.Uid)); err == nil {
h.Uname = u.Username
userMap.Store(h.Uid, h.Uname)
}
if g, ok := groupMap.Load(h.Gid); ok {
h.Gname = g.(string)
} else if g, err := user.LookupGroupId(strconv.Itoa(h.Gid)); err == nil {
h.Gname = g.Name
groupMap.Store(h.Gid, h.Gname)
}
// Best effort at populating Uname and Gname.
// The os/user functions may fail for any number of reasons
// (not implemented on that platform, cgo not enabled, etc).
if u, ok := userMap.Load(h.Uid); ok {
h.Uname = u.(string)
} else if u, err := user.LookupId(strconv.Itoa(h.Uid)); err == nil {
h.Uname = u.Username
userMap.Store(h.Uid, h.Uname)
}
if g, ok := groupMap.Load(h.Gid); ok {
h.Gname = g.(string)
} else if g, err := user.LookupGroupId(strconv.Itoa(h.Gid)); err == nil {
h.Gname = g.Name
groupMap.Store(h.Gid, h.Gname)
}
h.AccessTime = statAtime(sys)
h.ChangeTime = statCtime(sys)

View File

@@ -73,7 +73,7 @@ func (f *formatter) formatString(b []byte, s string) {
// in the V7 path field as a directory even though the full path
// recorded elsewhere (e.g., via PAX record) contains no trailing slash.
if len(s) > len(b) && b[len(b)-1] == '/' {
n := len(strings.TrimRight(s[:len(b)-1], "/"))
n := len(strings.TrimRight(s[:len(b)], "/"))
b[n] = 0 // Replace trailing slash with NUL terminator
}
}

View File

@@ -11,13 +11,11 @@ import (
"internal/testenv"
"io"
"io/fs"
"maps"
"math"
"os"
"path"
"path/filepath"
"reflect"
"slices"
"strings"
"testing"
"time"
@@ -100,6 +98,10 @@ func (f *testFile) Seek(pos int64, whence int) (int64, error) {
return f.pos, nil
}
func equalSparseEntries(x, y []sparseEntry) bool {
return (len(x) == 0 && len(y) == 0) || reflect.DeepEqual(x, y)
}
func TestSparseEntries(t *testing.T) {
vectors := []struct {
in []sparseEntry
@@ -196,11 +198,11 @@ func TestSparseEntries(t *testing.T) {
continue
}
gotAligned := alignSparseEntries(append([]sparseEntry{}, v.in...), v.size)
if !slices.Equal(gotAligned, v.wantAligned) {
if !equalSparseEntries(gotAligned, v.wantAligned) {
t.Errorf("test %d, alignSparseEntries():\ngot %v\nwant %v", i, gotAligned, v.wantAligned)
}
gotInverted := invertSparseEntries(append([]sparseEntry{}, v.in...), v.size)
if !slices.Equal(gotInverted, v.wantInverted) {
if !equalSparseEntries(gotInverted, v.wantInverted) {
t.Errorf("test %d, inverseSparseEntries():\ngot %v\nwant %v", i, gotInverted, v.wantInverted)
}
}
@@ -742,7 +744,7 @@ func TestHeaderAllowedFormats(t *testing.T) {
if formats != v.formats {
t.Errorf("test %d, allowedFormats(): got %v, want %v", i, formats, v.formats)
}
if formats&FormatPAX > 0 && !maps.Equal(paxHdrs, v.paxHdrs) && !(len(paxHdrs) == 0 && len(v.paxHdrs) == 0) {
if formats&FormatPAX > 0 && !reflect.DeepEqual(paxHdrs, v.paxHdrs) && !(len(paxHdrs) == 0 && len(v.paxHdrs) == 0) {
t.Errorf("test %d, allowedFormats():\ngot %v\nwant %s", i, paxHdrs, v.paxHdrs)
}
if (formats != FormatUnknown) && (err != nil) {
@@ -846,53 +848,3 @@ func Benchmark(b *testing.B) {
})
}
var _ fileInfoNames = fileInfoNames{}
type fileInfoNames struct{}
func (f *fileInfoNames) Name() string {
return "tmp"
}
func (f *fileInfoNames) Size() int64 {
return 0
}
func (f *fileInfoNames) Mode() fs.FileMode {
return 0777
}
func (f *fileInfoNames) ModTime() time.Time {
return time.Time{}
}
func (f *fileInfoNames) IsDir() bool {
return false
}
func (f *fileInfoNames) Sys() any {
return nil
}
func (f *fileInfoNames) Uname() (string, error) {
return "Uname", nil
}
func (f *fileInfoNames) Gname() (string, error) {
return "Gname", nil
}
func TestFileInfoHeaderUseFileInfoNames(t *testing.T) {
info := &fileInfoNames{}
header, err := FileInfoHeader(info, "")
if err != nil {
t.Fatal(err)
}
if header.Uname != "Uname" {
t.Fatalf("header.Uname: got %s, want %s", header.Uname, "Uname")
}
if header.Gname != "Gname" {
t.Fatalf("header.Gname: got %s, want %s", header.Gname, "Gname")
}
}

View File

@@ -5,19 +5,16 @@
package tar
import (
"errors"
"fmt"
"io"
"io/fs"
"maps"
"path"
"slices"
"sort"
"strings"
"time"
)
// Writer provides sequential writing of a tar archive.
// [Writer.WriteHeader] begins a new file with the provided [Header],
// Write.WriteHeader begins a new file with the provided Header,
// and then Writer can be treated as an io.Writer to supply that file's data.
type Writer struct {
w io.Writer
@@ -47,7 +44,7 @@ type fileWriter interface {
// Flush finishes writing the current file's block padding.
// The current file must be fully written before Flush can be called.
//
// This is unnecessary as the next call to [Writer.WriteHeader] or [Writer.Close]
// This is unnecessary as the next call to WriteHeader or Close
// will implicitly flush out the file's padding.
func (tw *Writer) Flush() error {
if tw.err != nil {
@@ -170,10 +167,16 @@ func (tw *Writer) writePAXHeader(hdr *Header, paxHdrs map[string]string) error {
// Write PAX records to the output.
isGlobal := hdr.Typeflag == TypeXGlobalHeader
if len(paxHdrs) > 0 || isGlobal {
// Sort keys for deterministic ordering.
var keys []string
for k := range paxHdrs {
keys = append(keys, k)
}
sort.Strings(keys)
// Write each record to a buffer.
var buf strings.Builder
// Sort keys for deterministic ordering.
for _, k := range slices.Sorted(maps.Keys(paxHdrs)) {
for _, k := range keys {
rec, err := formatPAXRecord(k, paxHdrs[k])
if err != nil {
return err
@@ -400,46 +403,6 @@ func (tw *Writer) writeRawHeader(blk *block, size int64, flag byte) error {
return nil
}
// AddFS adds the files from fs.FS to the archive.
// It walks the directory tree starting at the root of the filesystem
// adding each file to the tar archive while maintaining the directory structure.
func (tw *Writer) AddFS(fsys fs.FS) error {
return fs.WalkDir(fsys, ".", func(name string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if name == "." {
return nil
}
info, err := d.Info()
if err != nil {
return err
}
// TODO(#49580): Handle symlinks when fs.ReadLinkFS is available.
if !d.IsDir() && !info.Mode().IsRegular() {
return errors.New("tar: cannot add non-regular file")
}
h, err := FileInfoHeader(info, "")
if err != nil {
return err
}
h.Name = name
if err := tw.WriteHeader(h); err != nil {
return err
}
if d.IsDir() {
return nil
}
f, err := fsys.Open(name)
if err != nil {
return err
}
defer f.Close()
_, err = io.Copy(tw, f)
return err
})
}
// splitUSTARPath splits a path according to USTAR prefix and suffix rules.
// If the path is not splittable, then it will return ("", "", false).
func splitUSTARPath(name string) (prefix, suffix string, ok bool) {
@@ -462,12 +425,12 @@ func splitUSTARPath(name string) (prefix, suffix string, ok bool) {
}
// Write writes to the current file in the tar archive.
// Write returns the error [ErrWriteTooLong] if more than
// Header.Size bytes are written after [Writer.WriteHeader].
// Write returns the error ErrWriteTooLong if more than
// Header.Size bytes are written after WriteHeader.
//
// Calling Write on special types like [TypeLink], [TypeSymlink], [TypeChar],
// [TypeBlock], [TypeDir], and [TypeFifo] returns (0, [ErrWriteTooLong]) regardless
// of what the [Header.Size] claims.
// Calling Write on special types like TypeLink, TypeSymlink, TypeChar,
// TypeBlock, TypeDir, and TypeFifo returns (0, ErrWriteTooLong) regardless
// of what the Header.Size claims.
func (tw *Writer) Write(b []byte) (int, error) {
if tw.err != nil {
return 0, tw.err
@@ -501,7 +464,7 @@ func (tw *Writer) readFrom(r io.Reader) (int64, error) {
}
// Close closes the tar archive by flushing the padding, and writing the footer.
// If the current file (from a prior call to [Writer.WriteHeader]) is not fully written,
// If the current file (from a prior call to WriteHeader) is not fully written,
// then this returns an error.
func (tw *Writer) Close() error {
if tw.err == ErrWriteAfterClose {
@@ -666,7 +629,6 @@ func (sw *sparseFileWriter) ReadFrom(r io.Reader) (n int64, err error) {
func (sw sparseFileWriter) logicalRemaining() int64 {
return sw.sp[len(sw.sp)-1].endOffset() - sw.pos
}
func (sw sparseFileWriter) physicalRemaining() int64 {
return sw.fw.physicalRemaining()
}

View File

@@ -9,15 +9,12 @@ import (
"encoding/hex"
"errors"
"io"
"io/fs"
"maps"
"os"
"path"
"slices"
"reflect"
"sort"
"strings"
"testing"
"testing/fstest"
"testing/iotest"
"time"
)
@@ -582,10 +579,10 @@ func TestPaxSymlink(t *testing.T) {
t.Fatal(err)
}
hdr, err := FileInfoHeader(fileinfo, "")
hdr.Typeflag = TypeSymlink
if err != nil {
t.Fatalf("os.Stat:1 %v", err)
}
hdr.Typeflag = TypeSymlink
// Force a PAX long linkname to be written
longLinkname := strings.Repeat("1234567890/1234567890", 10)
hdr.Linkname = longLinkname
@@ -703,7 +700,7 @@ func TestPaxXattrs(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if !maps.Equal(hdr.Xattrs, xattrs) {
if !reflect.DeepEqual(hdr.Xattrs, xattrs) {
t.Fatalf("xattrs did not survive round trip: got %+v, want %+v",
hdr.Xattrs, xattrs)
}
@@ -750,7 +747,7 @@ func TestPaxHeadersSorted(t *testing.T) {
bytes.Index(buf.Bytes(), []byte("foo=foo")),
bytes.Index(buf.Bytes(), []byte("qux=qux")),
}
if !slices.IsSorted(indices) {
if !sort.IntsAreSorted(indices) {
t.Fatal("PAX headers are not sorted")
}
}
@@ -762,10 +759,10 @@ func TestUSTARLongName(t *testing.T) {
t.Fatal(err)
}
hdr, err := FileInfoHeader(fileinfo, "")
hdr.Typeflag = TypeDir
if err != nil {
t.Fatalf("os.Stat:1 %v", err)
}
hdr.Typeflag = TypeDir
// Force a PAX long name to be written. The name was taken from a practical example
// that fails and replaced ever char through numbers to anonymize the sample.
longName := "/0000_0000000/00000-000000000/0000_0000000/00000-0000000000000/0000_0000000/00000-0000000-00000000/0000_0000000/00000000/0000_0000000/000/0000_0000000/00000000v00/0000_0000000/000000/0000_0000000/0000000/0000_0000000/00000y-00/0000/0000/00000000/0x000000/"
@@ -1336,89 +1333,3 @@ func TestFileWriter(t *testing.T) {
}
}
}
func TestWriterAddFS(t *testing.T) {
fsys := fstest.MapFS{
"emptyfolder": {Mode: 0o755 | os.ModeDir},
"file.go": {Data: []byte("hello")},
"subfolder/another.go": {Data: []byte("world")},
// Notably missing here is the "subfolder" directory. This makes sure even
// if we don't have a subfolder directory listed.
}
var buf bytes.Buffer
tw := NewWriter(&buf)
if err := tw.AddFS(fsys); err != nil {
t.Fatal(err)
}
if err := tw.Close(); err != nil {
t.Fatal(err)
}
// Add subfolder into fsys to match what we'll read from the tar.
fsys["subfolder"] = &fstest.MapFile{Mode: 0o555 | os.ModeDir}
// Test that we can get the files back from the archive
tr := NewReader(&buf)
names := make([]string, 0, len(fsys))
for name := range fsys {
names = append(names, name)
}
sort.Strings(names)
entriesLeft := len(fsys)
for _, name := range names {
entriesLeft--
entryInfo, err := fsys.Stat(name)
if err != nil {
t.Fatalf("getting entry info error: %v", err)
}
hdr, err := tr.Next()
if err == io.EOF {
break // End of archive
}
if err != nil {
t.Fatal(err)
}
if hdr.Name != name {
t.Errorf("test fs has filename %v; archive header has %v",
name, hdr.Name)
}
if entryInfo.Mode() != hdr.FileInfo().Mode() {
t.Errorf("%s: test fs has mode %v; archive header has %v",
name, entryInfo.Mode(), hdr.FileInfo().Mode())
}
if entryInfo.IsDir() {
continue
}
data, err := io.ReadAll(tr)
if err != nil {
t.Fatal(err)
}
origdata := fsys[name].Data
if string(data) != string(origdata) {
t.Fatalf("test fs has file content %v; archive header has %v",
data, origdata)
}
}
if entriesLeft > 0 {
t.Fatalf("not all entries are in the archive")
}
}
func TestWriterAddFSNonRegularFiles(t *testing.T) {
fsys := fstest.MapFS{
"device": {Data: []byte("hello"), Mode: 0755 | fs.ModeDevice},
"symlink": {Data: []byte("world"), Mode: 0755 | fs.ModeSymlink},
}
var buf bytes.Buffer
tw := NewWriter(&buf)
if err := tw.AddFS(fsys); err == nil {
t.Fatal("expected error, got nil")
}
}

View File

@@ -16,7 +16,7 @@ import (
"os"
"path"
"path/filepath"
"slices"
"sort"
"strings"
"sync"
"time"
@@ -48,15 +48,15 @@ type Reader struct {
fileList []fileListEntry
}
// A ReadCloser is a [Reader] that must be closed when no longer needed.
// A ReadCloser is a Reader that must be closed when no longer needed.
type ReadCloser struct {
f *os.File
Reader
}
// A File is a single file in a ZIP archive.
// The file information is in the embedded [FileHeader].
// The file content can be accessed by calling [File.Open].
// The file information is in the embedded FileHeader.
// The file content can be accessed by calling Open.
type File struct {
FileHeader
zip *Reader
@@ -93,16 +93,16 @@ func OpenReader(name string) (*ReadCloser, error) {
return r, err
}
// NewReader returns a new [Reader] reading from r, which is assumed to
// NewReader returns a new Reader reading from r, which is assumed to
// have the given size in bytes.
//
// If any file inside the archive uses a non-local name
// (as defined by [filepath.IsLocal]) or a name containing backslashes
// and the GODEBUG environment variable contains `zipinsecurepath=0`,
// NewReader returns the reader with an [ErrInsecurePath] error.
// NewReader returns the reader with an ErrInsecurePath error.
// A future version of Go may introduce this behavior by default.
// Programs that want to accept non-local names can ignore
// the [ErrInsecurePath] error and use the returned reader.
// the ErrInsecurePath error and use the returned reader.
func NewReader(r io.ReaderAt, size int64) (*Reader, error) {
if size < 0 {
return nil, errors.New("zip: size cannot be negative")
@@ -178,7 +178,7 @@ func (r *Reader) init(rdr io.ReaderAt, size int64) error {
// RegisterDecompressor registers or overrides a custom decompressor for a
// specific method ID. If a decompressor for a given method is not found,
// [Reader] will default to looking up the decompressor at the package level.
// Reader will default to looking up the decompressor at the package level.
func (r *Reader) RegisterDecompressor(method uint16, dcomp Decompressor) {
if r.decompressors == nil {
r.decompressors = make(map[uint16]Decompressor)
@@ -202,7 +202,7 @@ func (rc *ReadCloser) Close() error {
// DataOffset returns the offset of the file's possibly-compressed
// data, relative to the beginning of the zip file.
//
// Most callers should instead use [File.Open], which transparently
// Most callers should instead use Open, which transparently
// decompresses data and verifies checksums.
func (f *File) DataOffset() (offset int64, err error) {
bodyOffset, err := f.findBodyOffset()
@@ -212,7 +212,7 @@ func (f *File) DataOffset() (offset int64, err error) {
return f.headerOffset + bodyOffset, nil
}
// Open returns a [ReadCloser] that provides access to the [File]'s contents.
// Open returns a ReadCloser that provides access to the File's contents.
// Multiple files may be read concurrently.
func (f *File) Open() (io.ReadCloser, error) {
bodyOffset, err := f.findBodyOffset()
@@ -255,7 +255,7 @@ func (f *File) Open() (io.ReadCloser, error) {
return rc, nil
}
// OpenRaw returns a [Reader] that provides access to the [File]'s contents without
// OpenRaw returns a Reader that provides access to the File's contents without
// decompression.
func (f *File) OpenRaw() (io.Reader, error) {
bodyOffset, err := f.findBodyOffset()
@@ -469,8 +469,8 @@ parseExtras:
const ticksPerSecond = 1e7 // Windows timestamp resolution
ts := int64(attrBuf.uint64()) // ModTime since Windows epoch
secs := ts / ticksPerSecond
nsecs := (1e9 / ticksPerSecond) * (ts % ticksPerSecond)
secs := int64(ts / ticksPerSecond)
nsecs := (1e9 / ticksPerSecond) * int64(ts%ticksPerSecond)
epoch := time.Date(1601, time.January, 1, 0, 0, 0, 0, time.UTC)
modified = time.Unix(epoch.Unix()+secs, nsecs)
}
@@ -699,13 +699,9 @@ func findSignatureInBlock(b []byte) int {
if b[i] == 'P' && b[i+1] == 'K' && b[i+2] == 0x05 && b[i+3] == 0x06 {
// n is length of comment
n := int(b[i+directoryEndLen-2]) | int(b[i+directoryEndLen-1])<<8
if n+directoryEndLen+i > len(b) {
// Truncated comment.
// Some parsers (such as Info-ZIP) ignore the truncated comment
// rather than treating it as a hard error.
return -1
if n+directoryEndLen+i <= len(b) {
return i
}
return i
}
}
return -1
@@ -862,19 +858,14 @@ func (r *Reader) initFileList() {
}
}
slices.SortFunc(r.fileList, func(a, b fileListEntry) int {
return fileEntryCompare(a.name, b.name)
})
sort.Slice(r.fileList, func(i, j int) bool { return fileEntryLess(r.fileList[i].name, r.fileList[j].name) })
})
}
func fileEntryCompare(x, y string) int {
func fileEntryLess(x, y string) bool {
xdir, xelem, _ := split(x)
ydir, yelem, _ := split(y)
if xdir != ydir {
return strings.Compare(xdir, ydir)
}
return strings.Compare(xelem, yelem)
return xdir < ydir || xdir == ydir && xelem < yelem
}
// Open opens the named file in the ZIP archive,
@@ -902,8 +893,14 @@ func (r *Reader) Open(name string) (fs.File, error) {
}
func split(name string) (dir, elem string, isDir bool) {
name, isDir = strings.CutSuffix(name, "/")
i := strings.LastIndexByte(name, '/')
if len(name) > 0 && name[len(name)-1] == '/' {
isDir = true
name = name[:len(name)-1]
}
i := len(name) - 1
for i >= 0 && name[i] != '/' {
i--
}
if i < 0 {
return ".", name, isDir
}
@@ -919,12 +916,9 @@ func (r *Reader) openLookup(name string) *fileListEntry {
dir, elem, _ := split(name)
files := r.fileList
i, _ := slices.BinarySearchFunc(files, dir, func(a fileListEntry, dir string) (ret int) {
idir, ielem, _ := split(a.name)
if dir != idir {
return strings.Compare(idir, dir)
}
return strings.Compare(ielem, elem)
i := sort.Search(len(files), func(i int) bool {
idir, ielem, _ := split(files[i].name)
return idir > dir || idir == dir && ielem >= elem
})
if i < len(files) {
fname := files[i].name
@@ -937,21 +931,13 @@ func (r *Reader) openLookup(name string) *fileListEntry {
func (r *Reader) openReadDir(dir string) []fileListEntry {
files := r.fileList
i, _ := slices.BinarySearchFunc(files, dir, func(a fileListEntry, dir string) int {
idir, _, _ := split(a.name)
if dir != idir {
return strings.Compare(idir, dir)
}
// find the first entry with dir
return +1
i := sort.Search(len(files), func(i int) bool {
idir, _, _ := split(files[i].name)
return idir >= dir
})
j, _ := slices.BinarySearchFunc(files, dir, func(a fileListEntry, dir string) int {
jdir, _, _ := split(a.name)
if dir != jdir {
return strings.Compare(jdir, dir)
}
// find the last entry with dir
return -1
j := sort.Search(len(files), func(j int) bool {
jdir, _, _ := split(files[j].name)
return jdir > dir
})
return files[i:j]
}

View File

@@ -13,8 +13,8 @@ import (
"io/fs"
"os"
"path/filepath"
"reflect"
"regexp"
"slices"
"strings"
"testing"
"testing/fstest"
@@ -570,14 +570,6 @@ var tests = []ZipTest{
},
},
},
// Issue 66869: Don't skip over an EOCDR with a truncated comment.
// The test file sneakily hides a second EOCDR before the first one;
// previously we would extract one file ("file") from this archive,
// while most other tools would reject the file or extract a different one ("FILE").
{
Name: "comment-truncated.zip",
Error: ErrFormat,
},
}
func TestReader(t *testing.T) {
@@ -912,7 +904,9 @@ func returnRecursiveZip() (r io.ReaderAt, size int64) {
// type zeros struct{}
//
// func (zeros) Read(b []byte) (int, error) {
// clear(b)
// for i := range b {
// b[i] = 0
// }
// return len(b), nil
// }
//
@@ -1192,7 +1186,7 @@ func TestIssue12449(t *testing.T) {
0x00, 0x50, 0x00, 0x00, 0x00, 0x00, 0x00,
}
// Read in the archive.
_, err := NewReader(bytes.NewReader(data), int64(len(data)))
_, err := NewReader(bytes.NewReader([]byte(data)), int64(len(data)))
if err != nil {
t.Errorf("Error reading the archive: %v", err)
}
@@ -1274,7 +1268,7 @@ func TestFSWalk(t *testing.T) {
} else if !test.wantErr && sawErr {
t.Error("unexpected error")
}
if test.want != nil && !slices.Equal(files, test.want) {
if test.want != nil && !reflect.DeepEqual(files, test.want) {
t.Errorf("got %v want %v", files, test.want)
}
})
@@ -1339,7 +1333,7 @@ func TestCVE202127919(t *testing.T) {
0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x39, 0x00,
0x00, 0x00, 0x59, 0x00, 0x00, 0x00, 0x00, 0x00,
}
r, err := NewReader(bytes.NewReader(data), int64(len(data)))
r, err := NewReader(bytes.NewReader([]byte(data)), int64(len(data)))
if err != ErrInsecurePath {
t.Fatalf("Error reading the archive: %v", err)
}
@@ -1565,7 +1559,7 @@ func TestCVE202141772(t *testing.T) {
0x00, 0x04, 0x00, 0x04, 0x00, 0x31, 0x01, 0x00,
0x00, 0x90, 0x00, 0x00, 0x00, 0x00, 0x00,
}
r, err := NewReader(bytes.NewReader(data), int64(len(data)))
r, err := NewReader(bytes.NewReader([]byte(data)), int64(len(data)))
if err != ErrInsecurePath {
t.Fatalf("Error reading the archive: %v", err)
}
@@ -1580,7 +1574,7 @@ func TestCVE202141772(t *testing.T) {
t.Errorf("Opening %q with fs.FS API succeeded", f.Name)
}
}
if !slices.Equal(names, entryNames) {
if !reflect.DeepEqual(names, entryNames) {
t.Errorf("Unexpected file entries: %q", names)
}
if _, err := r.Open(""); err == nil {
@@ -1693,7 +1687,7 @@ func TestInsecurePaths(t *testing.T) {
for _, f := range zr.File {
gotPaths = append(gotPaths, f.Name)
}
if !slices.Equal(gotPaths, []string{path}) {
if !reflect.DeepEqual(gotPaths, []string{path}) {
t.Errorf("NewReader for archive with file %q: got files %q", path, gotPaths)
continue
}
@@ -1718,7 +1712,7 @@ func TestDisableInsecurePathCheck(t *testing.T) {
for _, f := range zr.File {
gotPaths = append(gotPaths, f.Name)
}
if want := []string{name}; !slices.Equal(gotPaths, want) {
if want := []string{name}; !reflect.DeepEqual(gotPaths, want) {
t.Errorf("NewReader with zipinsecurepath=1: got files %q, want %q", gotPaths, want)
}
}
@@ -1828,7 +1822,7 @@ func TestBaseOffsetPlusOverflow(t *testing.T) {
}
}()
// Previously, this would trigger a panic as we attempt to read from
// an io.SectionReader which would access a slice at a negative offset
// a io.SectionReader which would access a slice at a negative offset
// as the section reader offset & size were < 0.
NewReader(bytes.NewReader(data), int64(len(data))+1875)
}

View File

@@ -19,7 +19,7 @@ import (
type Compressor func(w io.Writer) (io.WriteCloser, error)
// A Decompressor returns a new decompressing reader, reading from r.
// The [io.ReadCloser]'s Close method must be used to release associated resources.
// The ReadCloser's Close method must be used to release associated resources.
// The Decompressor itself must be safe to invoke from multiple goroutines
// simultaneously, but each returned reader will be used only by
// one goroutine at a time.
@@ -115,7 +115,7 @@ func init() {
}
// RegisterDecompressor allows custom decompressors for a specified method ID.
// The common methods [Store] and [Deflate] are built in.
// The common methods Store and Deflate are built in.
func RegisterDecompressor(method uint16, dcomp Decompressor) {
if _, dup := decompressors.LoadOrStore(method, dcomp); dup {
panic("decompressor already registered")
@@ -123,7 +123,7 @@ func RegisterDecompressor(method uint16, dcomp Decompressor) {
}
// RegisterCompressor registers custom compressors for a specified method ID.
// The common methods [Store] and [Deflate] are built in.
// The common methods Store and Deflate are built in.
func RegisterCompressor(method uint16, comp Compressor) {
if _, dup := compressors.LoadOrStore(method, comp); dup {
panic("compressor already registered")

View File

@@ -17,7 +17,7 @@ for normal archives both fields will be the same. For files requiring
the ZIP64 format the 32 bit fields will be 0xffffffff and the 64 bit
fields must be used instead.
[ZIP specification]: https://support.pkware.com/pkzip/appnote
[ZIP specification]: https://www.pkware.com/appnote
*/
package zip
@@ -82,7 +82,7 @@ const (
// FileHeader describes a file within a ZIP file.
// See the [ZIP specification] for details.
//
// [ZIP specification]: https://support.pkware.com/pkzip/appnote
// [ZIP specification]: https://www.pkware.com/appnote
type FileHeader struct {
// Name is the name of the file.
//
@@ -143,9 +143,9 @@ type FileHeader struct {
// Deprecated: Use CompressedSize64 instead.
CompressedSize uint32
// UncompressedSize is the uncompressed size of the file in bytes.
// UncompressedSize is the compressed size of the file in bytes.
// If either the uncompressed or compressed size of the file
// does not fit in 32 bits, UncompressedSize is set to ^uint32(0).
// does not fit in 32 bits, CompressedSize is set to ^uint32(0).
//
// Deprecated: Use UncompressedSize64 instead.
UncompressedSize uint32
@@ -160,12 +160,12 @@ type FileHeader struct {
ExternalAttrs uint32 // Meaning depends on CreatorVersion
}
// FileInfo returns an fs.FileInfo for the [FileHeader].
// FileInfo returns an fs.FileInfo for the FileHeader.
func (h *FileHeader) FileInfo() fs.FileInfo {
return headerFileInfo{h}
}
// headerFileInfo implements [fs.FileInfo].
// headerFileInfo implements fs.FileInfo.
type headerFileInfo struct {
fh *FileHeader
}
@@ -194,7 +194,7 @@ func (fi headerFileInfo) String() string {
return fs.FormatFileInfo(fi)
}
// FileInfoHeader creates a partially-populated [FileHeader] from an
// FileInfoHeader creates a partially-populated FileHeader from an
// fs.FileInfo.
// Because fs.FileInfo's Name method returns only the base name of
// the file it describes, it may be necessary to modify the Name field
@@ -245,7 +245,7 @@ func timeZone(offset time.Duration) *time.Location {
// msDosTimeToTime converts an MS-DOS date and time into a time.Time.
// The resolution is 2s.
// See: https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-dosdatetimetofiletime
// See: https://msdn.microsoft.com/en-us/library/ms724247(v=VS.85).aspx
func msDosTimeToTime(dosDate, dosTime uint16) time.Time {
return time.Date(
// date bits 0-4: day of month; 5-8: month; 9-15: years since 1980
@@ -265,7 +265,7 @@ func msDosTimeToTime(dosDate, dosTime uint16) time.Time {
// timeToMsDosTime converts a time.Time to an MS-DOS date and time.
// The resolution is 2s.
// See: https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-filetimetodosdatetime
// See: https://msdn.microsoft.com/en-us/library/ms724274(v=VS.85).aspx
func timeToMsDosTime(t time.Time) (fDate uint16, fTime uint16) {
fDate = uint16(t.Day() + int(t.Month())<<5 + (t.Year()-1980)<<9)
fTime = uint16(t.Second()/2 + t.Minute()<<5 + t.Hour()<<11)
@@ -273,17 +273,17 @@ func timeToMsDosTime(t time.Time) (fDate uint16, fTime uint16) {
}
// ModTime returns the modification time in UTC using the legacy
// [ModifiedDate] and [ModifiedTime] fields.
// ModifiedDate and ModifiedTime fields.
//
// Deprecated: Use [Modified] instead.
// Deprecated: Use Modified instead.
func (h *FileHeader) ModTime() time.Time {
return msDosTimeToTime(h.ModifiedDate, h.ModifiedTime)
}
// SetModTime sets the [Modified], [ModifiedTime], and [ModifiedDate] fields
// SetModTime sets the Modified, ModifiedTime, and ModifiedDate fields
// to the given time in UTC.
//
// Deprecated: Use [Modified] instead.
// Deprecated: Use Modified instead.
func (h *FileHeader) SetModTime(t time.Time) {
t = t.UTC() // Convert to UTC for compatibility
h.Modified = t
@@ -309,7 +309,7 @@ const (
msdosReadOnly = 0x01
)
// Mode returns the permission and mode bits for the [FileHeader].
// Mode returns the permission and mode bits for the FileHeader.
func (h *FileHeader) Mode() (mode fs.FileMode) {
switch h.CreatorVersion >> 8 {
case creatorUnix, creatorMacOSX:
@@ -323,7 +323,7 @@ func (h *FileHeader) Mode() (mode fs.FileMode) {
return mode
}
// SetMode changes the permission and mode bits for the [FileHeader].
// SetMode changes the permission and mode bits for the FileHeader.
func (h *FileHeader) SetMode(mode fs.FileMode) {
h.CreatorVersion = h.CreatorVersion&0xff | creatorUnix<<8
h.ExternalAttrs = fileModeToUnixMode(mode) << 16

Binary file not shown.

View File

@@ -11,7 +11,6 @@ import (
"hash"
"hash/crc32"
"io"
"io/fs"
"strings"
"unicode/utf8"
)
@@ -41,7 +40,7 @@ type header struct {
raw bool
}
// NewWriter returns a new [Writer] writing a zip file to w.
// NewWriter returns a new Writer writing a zip file to w.
func NewWriter(w io.Writer) *Writer {
return &Writer{cw: &countWriter{w: bufio.NewWriter(w)}}
}
@@ -64,7 +63,7 @@ func (w *Writer) Flush() error {
}
// SetComment sets the end-of-central-directory comment field.
// It can only be called before [Writer.Close].
// It can only be called before Close.
func (w *Writer) SetComment(comment string) error {
if len(comment) > uint16max {
return errors.New("zip: Writer.Comment too long")
@@ -208,15 +207,14 @@ func (w *Writer) Close() error {
}
// Create adds a file to the zip file using the provided name.
// It returns a [Writer] to which the file contents should be written.
// The file contents will be compressed using the [Deflate] method.
// It returns a Writer to which the file contents should be written.
// The file contents will be compressed using the Deflate method.
// The name must be a relative path: it must not start with a drive
// letter (e.g. C:) or leading slash, and only forward slashes are
// allowed. To create a directory instead of a file, add a trailing
// slash to the name. Duplicate names will not overwrite previous entries
// and are appended to the zip file.
// The file's contents must be written to the [io.Writer] before the next
// call to [Writer.Create], [Writer.CreateHeader], or [Writer.Close].
// slash to the name.
// The file's contents must be written to the io.Writer before the next
// call to Create, CreateHeader, or Close.
func (w *Writer) Create(name string) (io.Writer, error) {
header := &FileHeader{
Name: name,
@@ -263,13 +261,13 @@ func (w *Writer) prepare(fh *FileHeader) error {
return nil
}
// CreateHeader adds a file to the zip archive using the provided [FileHeader]
// for the file metadata. [Writer] takes ownership of fh and may mutate
// its fields. The caller must not modify fh after calling [Writer.CreateHeader].
// CreateHeader adds a file to the zip archive using the provided FileHeader
// for the file metadata. Writer takes ownership of fh and may mutate
// its fields. The caller must not modify fh after calling CreateHeader.
//
// This returns a [Writer] to which the file contents should be written.
// This returns a Writer to which the file contents should be written.
// The file's contents must be written to the io.Writer before the next
// call to [Writer.Create], [Writer.CreateHeader], [Writer.CreateRaw], or [Writer.Close].
// call to Create, CreateHeader, CreateRaw, or Close.
func (w *Writer) CreateHeader(fh *FileHeader) (io.Writer, error) {
if err := w.prepare(fh); err != nil {
return nil, err
@@ -407,8 +405,8 @@ func writeHeader(w io.Writer, h *header) error {
// flags.
if h.raw && !h.hasDataDescriptor() {
b.uint32(h.CRC32)
b.uint32(uint32(min(h.CompressedSize64, uint32max)))
b.uint32(uint32(min(h.UncompressedSize64, uint32max)))
b.uint32(uint32(min64(h.CompressedSize64, uint32max)))
b.uint32(uint32(min64(h.UncompressedSize64, uint32max)))
} else {
// When this package handle the compression, these values are
// always written to the trailing data descriptor.
@@ -428,23 +426,26 @@ func writeHeader(w io.Writer, h *header) error {
return err
}
// CreateRaw adds a file to the zip archive using the provided [FileHeader] and
// returns a [Writer] to which the file contents should be written. The file's
// contents must be written to the io.Writer before the next call to [Writer.Create],
// [Writer.CreateHeader], [Writer.CreateRaw], or [Writer.Close].
func min64(x, y uint64) uint64 {
if x < y {
return x
}
return y
}
// CreateRaw adds a file to the zip archive using the provided FileHeader and
// returns a Writer to which the file contents should be written. The file's
// contents must be written to the io.Writer before the next call to Create,
// CreateHeader, CreateRaw, or Close.
//
// In contrast to [Writer.CreateHeader], the bytes passed to Writer are not compressed.
//
// CreateRaw's argument is stored in w. If the argument is a pointer to the embedded
// [FileHeader] in a [File] obtained from a [Reader] created from in-memory data,
// then w will refer to all of that memory.
// In contrast to CreateHeader, the bytes passed to Writer are not compressed.
func (w *Writer) CreateRaw(fh *FileHeader) (io.Writer, error) {
if err := w.prepare(fh); err != nil {
return nil, err
}
fh.CompressedSize = uint32(min(fh.CompressedSize64, uint32max))
fh.UncompressedSize = uint32(min(fh.UncompressedSize64, uint32max))
fh.CompressedSize = uint32(min64(fh.CompressedSize64, uint32max))
fh.UncompressedSize = uint32(min64(fh.UncompressedSize64, uint32max))
h := &header{
FileHeader: fh,
@@ -469,17 +470,14 @@ func (w *Writer) CreateRaw(fh *FileHeader) (io.Writer, error) {
return fw, nil
}
// Copy copies the file f (obtained from a [Reader]) into w. It copies the raw
// Copy copies the file f (obtained from a Reader) into w. It copies the raw
// form directly bypassing decompression, compression, and validation.
func (w *Writer) Copy(f *File) error {
r, err := f.OpenRaw()
if err != nil {
return err
}
// Copy the FileHeader so w doesn't store a pointer to the data
// of f's entire archive. See #65499.
fh := f.FileHeader
fw, err := w.CreateRaw(&fh)
fw, err := w.CreateRaw(&f.FileHeader)
if err != nil {
return err
}
@@ -488,7 +486,7 @@ func (w *Writer) Copy(f *File) error {
}
// RegisterCompressor registers or overrides a custom compressor for a specific
// method ID. If a compressor for a given method is not found, [Writer] will
// method ID. If a compressor for a given method is not found, Writer will
// default to looking up the compressor at the package level.
func (w *Writer) RegisterCompressor(method uint16, comp Compressor) {
if w.compressors == nil {
@@ -497,47 +495,6 @@ func (w *Writer) RegisterCompressor(method uint16, comp Compressor) {
w.compressors[method] = comp
}
// AddFS adds the files from fs.FS to the archive.
// It walks the directory tree starting at the root of the filesystem
// adding each file to the zip using deflate while maintaining the directory structure.
func (w *Writer) AddFS(fsys fs.FS) error {
return fs.WalkDir(fsys, ".", func(name string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if name == "." {
return nil
}
info, err := d.Info()
if err != nil {
return err
}
if !d.IsDir() && !info.Mode().IsRegular() {
return errors.New("zip: cannot add non-regular file")
}
h, err := FileInfoHeader(info)
if err != nil {
return err
}
h.Name = name
h.Method = Deflate
fw, err := w.CreateHeader(h)
if err != nil {
return err
}
if d.IsDir() {
return nil
}
f, err := fsys.Open(name)
if err != nil {
return err
}
defer f.Close()
_, err = io.Copy(fw, f)
return err
})
}
func (w *Writer) compressor(method uint16) Compressor {
comp := w.compressors[method]
if comp == nil {
@@ -612,7 +569,7 @@ func (w *fileWriter) writeDataDescriptor() error {
}
// Write data descriptor. This is more complicated than one would
// think, see e.g. comments in zipfile.c:putextended() and
// https://bugs.openjdk.org/browse/JDK-7073588.
// http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7073588.
// The approach here is to write 8 byte sizes if needed without
// adding a zip64 extra in the local header (too late anyway).
var buf []byte

View File

@@ -16,7 +16,6 @@ import (
"os"
"strings"
"testing"
"testing/fstest"
"time"
)
@@ -108,7 +107,7 @@ func TestWriter(t *testing.T) {
// TestWriterComment is test for EOCD comment read/write.
func TestWriterComment(t *testing.T) {
tests := []struct {
var tests = []struct {
comment string
ok bool
}{
@@ -158,7 +157,7 @@ func TestWriterComment(t *testing.T) {
}
func TestWriterUTF8(t *testing.T) {
utf8Tests := []struct {
var utf8Tests = []struct {
name string
comment string
nonUTF8 bool
@@ -603,68 +602,3 @@ func BenchmarkCompressedZipGarbage(b *testing.B) {
}
})
}
func writeTestsToFS(tests []WriteTest) fs.FS {
fsys := fstest.MapFS{}
for _, wt := range tests {
fsys[wt.Name] = &fstest.MapFile{
Data: wt.Data,
Mode: wt.Mode,
}
}
return fsys
}
func TestWriterAddFS(t *testing.T) {
buf := new(bytes.Buffer)
w := NewWriter(buf)
tests := []WriteTest{
{Name: "emptyfolder", Mode: 0o755 | os.ModeDir},
{Name: "file.go", Data: []byte("hello"), Mode: 0644},
{Name: "subfolder/another.go", Data: []byte("world"), Mode: 0644},
// Notably missing here is the "subfolder" directory. This makes sure even
// if we don't have a subfolder directory listed.
}
err := w.AddFS(writeTestsToFS(tests))
if err != nil {
t.Fatal(err)
}
if err := w.Close(); err != nil {
t.Fatal(err)
}
// Add subfolder into fsys to match what we'll read from the tar.
tests = append(tests[:2:2], WriteTest{Name: "subfolder", Mode: 0o555 | os.ModeDir}, tests[2])
// read it back
r, err := NewReader(bytes.NewReader(buf.Bytes()), int64(buf.Len()))
if err != nil {
t.Fatal(err)
}
for i, wt := range tests {
testReadFile(t, r.File[i], &wt)
}
}
func TestIssue61875(t *testing.T) {
buf := new(bytes.Buffer)
w := NewWriter(buf)
tests := []WriteTest{
{
Name: "symlink",
Data: []byte("../link/target"),
Method: Deflate,
Mode: 0755 | fs.ModeSymlink,
},
{
Name: "device",
Data: []byte(""),
Method: Deflate,
Mode: 0755 | fs.ModeDevice,
},
}
err := w.AddFS(writeTestsToFS(tests))
if err == nil {
t.Errorf("expected error, got nil")
}
}

View File

@@ -8,14 +8,13 @@ package zip
import (
"bytes"
"cmp"
"errors"
"fmt"
"hash"
"internal/testenv"
"io"
"runtime"
"slices"
"sort"
"strings"
"testing"
"time"
@@ -199,6 +198,13 @@ func (r *rleBuffer) Write(p []byte) (n int, err error) {
return len(p), nil
}
func min(x, y int64) int64 {
if x < y {
return x
}
return y
}
func memset(a []byte, b byte) {
if len(a) == 0 {
return
@@ -215,8 +221,9 @@ func (r *rleBuffer) ReadAt(p []byte, off int64) (n int, err error) {
if len(p) == 0 {
return
}
skipParts, _ := slices.BinarySearchFunc(r.buf, off, func(rb repeatedByte, off int64) int {
return cmp.Compare(rb.off+rb.n, off)
skipParts := sort.Search(len(r.buf), func(i int) bool {
part := &r.buf[i]
return part.off+part.n > off
})
parts := r.buf[skipParts:]
if len(parts) > 0 {
@@ -590,7 +597,7 @@ func testZip64(t testing.TB, size int64) *rleBuffer {
}
// read back zip file and check that we get to the end of it
r, err := NewReader(buf, buf.Size())
r, err := NewReader(buf, int64(buf.Size()))
if err != nil {
t.Fatal("reader:", err)
}
@@ -814,6 +821,8 @@ func TestSuffixSaver(t *testing.T) {
type zeros struct{}
func (zeros) Read(p []byte) (int, error) {
clear(p)
for i := range p {
p[i] = 0
}
return len(p), nil
}

View File

@@ -29,9 +29,6 @@ var (
// Buffered input.
// Reader implements buffering for an io.Reader object.
// A new Reader is created by calling [NewReader] or [NewReaderSize];
// alternatively the zero value of a Reader may be used after calling [Reset]
// on it.
type Reader struct {
buf []byte
rd io.Reader // reader provided by the client
@@ -44,21 +41,24 @@ type Reader struct {
const minReadBufferSize = 16
const maxConsecutiveEmptyReads = 100
// NewReaderSize returns a new [Reader] whose buffer has at least the specified
// size. If the argument io.Reader is already a [Reader] with large enough
// size, it returns the underlying [Reader].
// NewReaderSize returns a new Reader whose buffer has at least the specified
// size. If the argument io.Reader is already a Reader with large enough
// size, it returns the underlying Reader.
func NewReaderSize(rd io.Reader, size int) *Reader {
// Is it already a Reader?
b, ok := rd.(*Reader)
if ok && len(b.buf) >= size {
return b
}
if size < minReadBufferSize {
size = minReadBufferSize
}
r := new(Reader)
r.reset(make([]byte, max(size, minReadBufferSize)), rd)
r.reset(make([]byte, size), rd)
return r
}
// NewReader returns a new [Reader] whose buffer has the default size.
// NewReader returns a new Reader whose buffer has the default size.
func NewReader(rd io.Reader) *Reader {
return NewReaderSize(rd, defaultBufSize)
}
@@ -68,9 +68,9 @@ func (b *Reader) Size() int { return len(b.buf) }
// Reset discards any buffered data, resets all state, and switches
// the buffered reader to read from r.
// Calling Reset on the zero value of [Reader] initializes the internal buffer
// Calling Reset on the zero value of Reader initializes the internal buffer
// to the default size.
// Calling b.Reset(b) (that is, resetting a [Reader] to itself) does nothing.
// Calling b.Reset(b) (that is, resetting a Reader to itself) does nothing.
func (b *Reader) Reset(r io.Reader) {
// If a Reader r is passed to NewReader, NewReader will return r.
// Different layers of code may do that, and then later pass r
@@ -133,12 +133,11 @@ func (b *Reader) readErr() error {
}
// Peek returns the next n bytes without advancing the reader. The bytes stop
// being valid at the next read call. If necessary, Peek will read more bytes
// into the buffer in order to make n bytes available. If Peek returns fewer
// than n bytes, it also returns an error explaining why the read is short.
// The error is [ErrBufferFull] if n is larger than b's buffer size.
// being valid at the next read call. If Peek returns fewer than n bytes, it
// also returns an error explaining why the read is short. The error is
// ErrBufferFull if n is larger than b's buffer size.
//
// Calling Peek prevents a [Reader.UnreadByte] or [Reader.UnreadRune] call from succeeding
// Calling Peek prevents a UnreadByte or UnreadRune call from succeeding
// until the next read operation.
func (b *Reader) Peek(n int) ([]byte, error) {
if n < 0 {
@@ -208,10 +207,10 @@ func (b *Reader) Discard(n int) (discarded int, err error) {
// Read reads data into p.
// It returns the number of bytes read into p.
// The bytes are taken from at most one Read on the underlying [Reader],
// The bytes are taken from at most one Read on the underlying Reader,
// hence n may be less than len(p).
// To read exactly len(p) bytes, use io.ReadFull(b, p).
// If the underlying [Reader] can return a non-zero count with io.EOF,
// If the underlying Reader can return a non-zero count with io.EOF,
// then this Read method can do so as well; see the [io.Reader] docs.
func (b *Reader) Read(p []byte) (n int, err error) {
n = len(p)
@@ -281,7 +280,7 @@ func (b *Reader) ReadByte() (byte, error) {
// UnreadByte unreads the last byte. Only the most recently read byte can be unread.
//
// UnreadByte returns an error if the most recent method called on the
// [Reader] was not a read operation. Notably, [Reader.Peek], [Reader.Discard], and [Reader.WriteTo] are not
// Reader was not a read operation. Notably, Peek, Discard, and WriteTo are not
// considered read operations.
func (b *Reader) UnreadByte() error {
if b.lastByte < 0 || b.r == 0 && b.w > 0 {
@@ -322,8 +321,8 @@ func (b *Reader) ReadRune() (r rune, size int, err error) {
}
// UnreadRune unreads the last rune. If the most recent method called on
// the [Reader] was not a [Reader.ReadRune], [Reader.UnreadRune] returns an error. (In this
// regard it is stricter than [Reader.UnreadByte], which will unread the last byte
// the Reader was not a ReadRune, UnreadRune returns an error. (In this
// regard it is stricter than UnreadByte, which will unread the last byte
// from any read operation.)
func (b *Reader) UnreadRune() error {
if b.lastRuneSize < 0 || b.r < b.lastRuneSize {
@@ -343,10 +342,10 @@ func (b *Reader) Buffered() int { return b.w - b.r }
// The bytes stop being valid at the next read.
// If ReadSlice encounters an error before finding a delimiter,
// it returns all the data in the buffer and the error itself (often io.EOF).
// ReadSlice fails with error [ErrBufferFull] if the buffer fills without a delim.
// ReadSlice fails with error ErrBufferFull if the buffer fills without a delim.
// Because the data returned from ReadSlice will be overwritten
// by the next I/O operation, most clients should use
// [Reader.ReadBytes] or ReadString instead.
// ReadBytes or ReadString instead.
// ReadSlice returns err != nil if and only if line does not end in delim.
func (b *Reader) ReadSlice(delim byte) (line []byte, err error) {
s := 0 // search start index
@@ -390,7 +389,7 @@ func (b *Reader) ReadSlice(delim byte) (line []byte, err error) {
}
// ReadLine is a low-level line-reading primitive. Most callers should use
// [Reader.ReadBytes]('\n') or [Reader.ReadString]('\n') instead or use a [Scanner].
// ReadBytes('\n') or ReadString('\n') instead or use a Scanner.
//
// ReadLine tries to return a single line, not including the end-of-line bytes.
// If the line was too long for the buffer then isPrefix is set and the
@@ -402,7 +401,7 @@ func (b *Reader) ReadSlice(delim byte) (line []byte, err error) {
//
// The text returned from ReadLine does not include the line end ("\r\n" or "\n").
// No indication or error is given if the input ends without a final line end.
// Calling [Reader.UnreadByte] after ReadLine will always unread the last byte read
// Calling UnreadByte after ReadLine will always unread the last byte read
// (possibly a character belonging to the line end) even if that byte is not
// part of the line returned by ReadLine.
func (b *Reader) ReadLine() (line []byte, isPrefix bool, err error) {
@@ -512,9 +511,9 @@ func (b *Reader) ReadString(delim byte) (string, error) {
}
// WriteTo implements io.WriterTo.
// This may make multiple calls to the [Reader.Read] method of the underlying [Reader].
// If the underlying reader supports the [Reader.WriteTo] method,
// this calls the underlying [Reader.WriteTo] without buffering.
// This may make multiple calls to the Read method of the underlying Reader.
// If the underlying reader supports the WriteTo method,
// this calls the underlying WriteTo without buffering.
func (b *Reader) WriteTo(w io.Writer) (n int64, err error) {
b.lastByte = -1
b.lastRuneSize = -1
@@ -559,7 +558,7 @@ func (b *Reader) WriteTo(w io.Writer) (n int64, err error) {
var errNegativeWrite = errors.New("bufio: writer returned negative count from Write")
// writeBuf writes the [Reader]'s buffer to the writer.
// writeBuf writes the Reader's buffer to the writer.
func (b *Reader) writeBuf(w io.Writer) (int64, error) {
n, err := w.Write(b.buf[b.r:b.w])
if n < 0 {
@@ -571,12 +570,12 @@ func (b *Reader) writeBuf(w io.Writer) (int64, error) {
// buffered output
// Writer implements buffering for an [io.Writer] object.
// If an error occurs writing to a [Writer], no more data will be
// accepted and all subsequent writes, and [Writer.Flush], will return the error.
// Writer implements buffering for an io.Writer object.
// If an error occurs writing to a Writer, no more data will be
// accepted and all subsequent writes, and Flush, will return the error.
// After all data has been written, the client should call the
// [Writer.Flush] method to guarantee all data has been forwarded to
// the underlying [io.Writer].
// Flush method to guarantee all data has been forwarded to
// the underlying io.Writer.
type Writer struct {
err error
buf []byte
@@ -584,9 +583,9 @@ type Writer struct {
wr io.Writer
}
// NewWriterSize returns a new [Writer] whose buffer has at least the specified
// size. If the argument io.Writer is already a [Writer] with large enough
// size, it returns the underlying [Writer].
// NewWriterSize returns a new Writer whose buffer has at least the specified
// size. If the argument io.Writer is already a Writer with large enough
// size, it returns the underlying Writer.
func NewWriterSize(w io.Writer, size int) *Writer {
// Is it already a Writer?
b, ok := w.(*Writer)
@@ -602,9 +601,9 @@ func NewWriterSize(w io.Writer, size int) *Writer {
}
}
// NewWriter returns a new [Writer] whose buffer has the default size.
// If the argument io.Writer is already a [Writer] with large enough buffer size,
// it returns the underlying [Writer].
// NewWriter returns a new Writer whose buffer has the default size.
// If the argument io.Writer is already a Writer with large enough buffer size,
// it returns the underlying Writer.
func NewWriter(w io.Writer) *Writer {
return NewWriterSize(w, defaultBufSize)
}
@@ -614,9 +613,9 @@ func (b *Writer) Size() int { return len(b.buf) }
// Reset discards any unflushed buffered data, clears any error, and
// resets b to write its output to w.
// Calling Reset on the zero value of [Writer] initializes the internal buffer
// Calling Reset on the zero value of Writer initializes the internal buffer
// to the default size.
// Calling w.Reset(w) (that is, resetting a [Writer] to itself) does nothing.
// Calling w.Reset(w) (that is, resetting a Writer to itself) does nothing.
func (b *Writer) Reset(w io.Writer) {
// If a Writer w is passed to NewWriter, NewWriter will return w.
// Different layers of code may do that, and then later pass w
@@ -632,7 +631,7 @@ func (b *Writer) Reset(w io.Writer) {
b.wr = w
}
// Flush writes any buffered data to the underlying [io.Writer].
// Flush writes any buffered data to the underlying io.Writer.
func (b *Writer) Flush() error {
if b.err != nil {
return b.err
@@ -661,7 +660,7 @@ func (b *Writer) Available() int { return len(b.buf) - b.n }
// AvailableBuffer returns an empty buffer with b.Available() capacity.
// This buffer is intended to be appended to and
// passed to an immediately succeeding [Writer.Write] call.
// passed to an immediately succeeding Write call.
// The buffer is only valid until the next write operation on b.
func (b *Writer) AvailableBuffer() []byte {
return b.buf[b.n:][:0]
@@ -778,7 +777,7 @@ func (b *Writer) WriteString(s string) (int, error) {
return nn, nil
}
// ReadFrom implements [io.ReaderFrom]. If the underlying writer
// ReadFrom implements io.ReaderFrom. If the underlying writer
// supports the ReadFrom method, this calls the underlying ReadFrom.
// If there is buffered data and an underlying ReadFrom, this fills
// the buffer and writes it before calling ReadFrom.
@@ -830,14 +829,14 @@ func (b *Writer) ReadFrom(r io.Reader) (n int64, err error) {
// buffered input and output
// ReadWriter stores pointers to a [Reader] and a [Writer].
// It implements [io.ReadWriter].
// ReadWriter stores pointers to a Reader and a Writer.
// It implements io.ReadWriter.
type ReadWriter struct {
*Reader
*Writer
}
// NewReadWriter allocates a new [ReadWriter] that dispatches to r and w.
// NewReadWriter allocates a new ReadWriter that dispatches to r and w.
func NewReadWriter(r *Reader, w *Writer) *ReadWriter {
return &ReadWriter{r, w}
}

View File

@@ -9,7 +9,6 @@ import (
"bytes"
"errors"
"fmt"
"internal/asan"
"io"
"math/rand"
"strconv"
@@ -586,9 +585,6 @@ func TestWriteInvalidRune(t *testing.T) {
}
func TestReadStringAllocs(t *testing.T) {
if asan.Enabled {
t.Skip("test allocates more with -asan; see #70079")
}
r := strings.NewReader(" foo foo 42 42 42 42 42 42 42 42 4.2 4.2 4.2 4.2\n")
buf := NewReader(r)
allocs := testing.AllocsPerRun(100, func() {
@@ -640,7 +636,7 @@ func TestWriter(t *testing.T) {
for l := 0; l < len(written); l++ {
if written[l] != data[l] {
t.Errorf("wrong bytes written")
t.Errorf("want=%q", data[:len(written)])
t.Errorf("want=%q", data[0:len(written)])
t.Errorf("have=%q", written)
}
}
@@ -939,6 +935,7 @@ func (t *testReader) Read(buf []byte) (n int, err error) {
}
func testReadLine(t *testing.T, input []byte) {
//for stride := 1; stride < len(input); stride++ {
for stride := 1; stride < 2; stride++ {
done := 0
reader := testReader{input, stride}

View File

@@ -6,7 +6,6 @@ package bufio_test
import (
"bufio"
"bytes"
"fmt"
"os"
"strconv"
@@ -33,33 +32,6 @@ func ExampleWriter_AvailableBuffer() {
// Output: 1 2 3 4
}
// ExampleWriter_ReadFrom demonstrates how to use the ReadFrom method of Writer.
func ExampleWriter_ReadFrom() {
var buf bytes.Buffer
writer := bufio.NewWriter(&buf)
data := "Hello, world!\nThis is a ReadFrom example."
reader := strings.NewReader(data)
n, err := writer.ReadFrom(reader)
if err != nil {
fmt.Println("ReadFrom Error:", err)
return
}
if err = writer.Flush(); err != nil {
fmt.Println("Flush Error:", err)
return
}
fmt.Println("Bytes written:", n)
fmt.Println("Buffer contents:", buf.String())
// Output:
// Bytes written: 41
// Buffer contents: Hello, world!
// This is a ReadFrom example.
}
// The simplest use of a Scanner, to read standard input as a set of lines.
func ExampleScanner_lines() {
scanner := bufio.NewScanner(os.Stdin)
@@ -165,36 +137,3 @@ func ExampleScanner_emptyFinalToken() {
}
// Output: "1" "2" "3" "4" ""
}
// Use a Scanner with a custom split function to parse a comma-separated
// list with an empty final value but stops at the token "STOP".
func ExampleScanner_earlyStop() {
onComma := func(data []byte, atEOF bool) (advance int, token []byte, err error) {
i := bytes.IndexByte(data, ',')
if i == -1 {
if !atEOF {
return 0, nil, nil
}
// If we have reached the end, return the last token.
return 0, data, bufio.ErrFinalToken
}
// If the token is "STOP", stop the scanning and ignore the rest.
if string(data[:i]) == "STOP" {
return i + 1, nil, bufio.ErrFinalToken
}
// Otherwise, return the token before the comma.
return i + 1, data[:i], nil
}
const input = "1,2,STOP,4,"
scanner := bufio.NewScanner(strings.NewReader(input))
scanner.Split(onComma)
for scanner.Scan() {
fmt.Printf("Got a token %q\n", scanner.Text())
}
if err := scanner.Err(); err != nil {
fmt.Fprintln(os.Stderr, "reading input:", err)
}
// Output:
// Got a token "1"
// Got a token "2"
}

View File

@@ -13,19 +13,19 @@ import (
// Scanner provides a convenient interface for reading data such as
// a file of newline-delimited lines of text. Successive calls to
// the [Scanner.Scan] method will step through the 'tokens' of a file, skipping
// the Scan method will step through the 'tokens' of a file, skipping
// the bytes between the tokens. The specification of a token is
// defined by a split function of type [SplitFunc]; the default split
// function breaks the input into lines with line termination stripped. [Scanner.Split]
// defined by a split function of type SplitFunc; the default split
// function breaks the input into lines with line termination stripped. Split
// functions are defined in this package for scanning a file into
// lines, bytes, UTF-8-encoded runes, and space-delimited words. The
// client may instead provide a custom split function.
//
// Scanning stops unrecoverably at EOF, the first I/O error, or a token too
// large to fit in the [Scanner.Buffer]. When a scan stops, the reader may have
// large to fit in the buffer. When a scan stops, the reader may have
// advanced arbitrarily far past the last token. Programs that need more
// control over error handling or large tokens, or must run sequential scans
// on a reader, should use [bufio.Reader] instead.
// on a reader, should use bufio.Reader instead.
type Scanner struct {
r io.Reader // The reader provided by the client.
split SplitFunc // The function to split the tokens.
@@ -42,23 +42,21 @@ type Scanner struct {
// SplitFunc is the signature of the split function used to tokenize the
// input. The arguments are an initial substring of the remaining unprocessed
// data and a flag, atEOF, that reports whether the [Reader] has no more data
// data and a flag, atEOF, that reports whether the Reader has no more data
// to give. The return values are the number of bytes to advance the input
// and the next token to return to the user, if any, plus an error, if any.
//
// Scanning stops if the function returns an error, in which case some of
// the input may be discarded. If that error is [ErrFinalToken], scanning
// stops with no error. A non-nil token delivered with [ErrFinalToken]
// will be the last token, and a nil token with [ErrFinalToken]
// immediately stops the scanning.
// the input may be discarded. If that error is ErrFinalToken, scanning
// stops with no error.
//
// Otherwise, the [Scanner] advances the input. If the token is not nil,
// the [Scanner] returns it to the user. If the token is nil, the
// Otherwise, the Scanner advances the input. If the token is not nil,
// the Scanner returns it to the user. If the token is nil, the
// Scanner reads more data and continues scanning; if there is no more
// data--if atEOF was true--the [Scanner] returns. If the data does not
// data--if atEOF was true--the Scanner returns. If the data does not
// yet hold a complete token, for instance if it has no newline while
// scanning lines, a [SplitFunc] can return (0, nil, nil) to signal the
// [Scanner] to read more data into the slice and try again with a
// scanning lines, a SplitFunc can return (0, nil, nil) to signal the
// Scanner to read more data into the slice and try again with a
// longer slice starting at the same point in the input.
//
// The function is never called with an empty data slice unless atEOF
@@ -76,7 +74,7 @@ var (
const (
// MaxScanTokenSize is the maximum size used to buffer a token
// unless the user provides an explicit buffer with [Scanner.Buffer].
// unless the user provides an explicit buffer with Scanner.Buffer.
// The actual maximum token size may be smaller as the buffer
// may need to include, for instance, a newline.
MaxScanTokenSize = 64 * 1024
@@ -84,8 +82,8 @@ const (
startBufSize = 4096 // Size of initial allocation for buffer.
)
// NewScanner returns a new [Scanner] to read from r.
// The split function defaults to [ScanLines].
// NewScanner returns a new Scanner to read from r.
// The split function defaults to ScanLines.
func NewScanner(r io.Reader) *Scanner {
return &Scanner{
r: r,
@@ -94,7 +92,7 @@ func NewScanner(r io.Reader) *Scanner {
}
}
// Err returns the first non-EOF error that was encountered by the [Scanner].
// Err returns the first non-EOF error that was encountered by the Scanner.
func (s *Scanner) Err() error {
if s.err == io.EOF {
return nil
@@ -102,36 +100,34 @@ func (s *Scanner) Err() error {
return s.err
}
// Bytes returns the most recent token generated by a call to [Scanner.Scan].
// Bytes returns the most recent token generated by a call to Scan.
// The underlying array may point to data that will be overwritten
// by a subsequent call to Scan. It does no allocation.
func (s *Scanner) Bytes() []byte {
return s.token
}
// Text returns the most recent token generated by a call to [Scanner.Scan]
// Text returns the most recent token generated by a call to Scan
// as a newly allocated string holding its bytes.
func (s *Scanner) Text() string {
return string(s.token)
}
// ErrFinalToken is a special sentinel error value. It is intended to be
// returned by a Split function to indicate that the scanning should stop
// with no error. If the token being delivered with this error is not nil,
// the token is the last token.
//
// returned by a Split function to indicate that the token being delivered
// with the error is the last token and scanning should stop after this one.
// After ErrFinalToken is received by Scan, scanning stops with no error.
// The value is useful to stop processing early or when it is necessary to
// deliver a final empty token (which is different from a nil token).
// One could achieve the same behavior with a custom error value but
// providing one here is tidier.
// deliver a final empty token. One could achieve the same behavior
// with a custom error value but providing one here is tidier.
// See the emptyFinalToken example for a use of this value.
var ErrFinalToken = errors.New("final token")
// Scan advances the [Scanner] to the next token, which will then be
// available through the [Scanner.Bytes] or [Scanner.Text] method. It returns false when
// there are no more tokens, either by reaching the end of the input or an error.
// After Scan returns false, the [Scanner.Err] method will return any error that
// occurred during scanning, except that if it was [io.EOF], [Scanner.Err]
// Scan advances the Scanner to the next token, which will then be
// available through the Bytes or Text method. It returns false when the
// scan stops, either by reaching the end of the input or an error.
// After Scan returns false, the Err method will return any error that
// occurred during scanning, except that if it was io.EOF, Err
// will return nil.
// Scan panics if the split function returns too many empty
// tokens without advancing the input. This is a common error mode for
@@ -152,10 +148,7 @@ func (s *Scanner) Scan() bool {
if err == ErrFinalToken {
s.token = token
s.done = true
// When token is not nil, it means the scanning stops
// with a trailing token, and thus the return value
// should be true to indicate the existence of the token.
return token != nil
return true
}
s.setErr(err)
return false
@@ -205,7 +198,9 @@ func (s *Scanner) Scan() bool {
if newSize == 0 {
newSize = startBufSize
}
newSize = min(newSize, s.maxTokenSize)
if newSize > s.maxTokenSize {
newSize = s.maxTokenSize
}
newBuf := make([]byte, newSize)
copy(newBuf, s.buf[s.start:s.end])
s.buf = newBuf
@@ -260,13 +255,13 @@ func (s *Scanner) setErr(err error) {
}
}
// Buffer sets the initial buffer to use when scanning
// and the maximum size of buffer that may be allocated during scanning.
// The maximum token size must be less than the larger of max and cap(buf).
// If max <= cap(buf), [Scanner.Scan] will use this buffer only and do no allocation.
// Buffer sets the initial buffer to use when scanning and the maximum
// size of buffer that may be allocated during scanning. The maximum
// token size is the larger of max and cap(buf). If max <= cap(buf),
// Scan will use this buffer only and do no allocation.
//
// By default, [Scanner.Scan] uses an internal buffer and sets the
// maximum token size to [MaxScanTokenSize].
// By default, Scan uses an internal buffer and sets the
// maximum token size to MaxScanTokenSize.
//
// Buffer panics if it is called after scanning has started.
func (s *Scanner) Buffer(buf []byte, max int) {
@@ -277,8 +272,8 @@ func (s *Scanner) Buffer(buf []byte, max int) {
s.maxTokenSize = max
}
// Split sets the split function for the [Scanner].
// The default split function is [ScanLines].
// Split sets the split function for the Scanner.
// The default split function is ScanLines.
//
// Split panics if it is called after scanning has started.
func (s *Scanner) Split(split SplitFunc) {
@@ -290,7 +285,7 @@ func (s *Scanner) Split(split SplitFunc) {
// Split functions
// ScanBytes is a split function for a [Scanner] that returns each byte as a token.
// ScanBytes is a split function for a Scanner that returns each byte as a token.
func ScanBytes(data []byte, atEOF bool) (advance int, token []byte, err error) {
if atEOF && len(data) == 0 {
return 0, nil, nil
@@ -300,7 +295,7 @@ func ScanBytes(data []byte, atEOF bool) (advance int, token []byte, err error) {
var errorRune = []byte(string(utf8.RuneError))
// ScanRunes is a split function for a [Scanner] that returns each
// ScanRunes is a split function for a Scanner that returns each
// UTF-8-encoded rune as a token. The sequence of runes returned is
// equivalent to that from a range loop over the input as a string, which
// means that erroneous UTF-8 encodings translate to U+FFFD = "\xef\xbf\xbd".
@@ -346,7 +341,7 @@ func dropCR(data []byte) []byte {
return data
}
// ScanLines is a split function for a [Scanner] that returns each line of
// ScanLines is a split function for a Scanner that returns each line of
// text, stripped of any trailing end-of-line marker. The returned line may
// be empty. The end-of-line marker is one optional carriage return followed
// by one mandatory newline. In regular expression notation, it is `\r?\n`.
@@ -393,7 +388,7 @@ func isSpace(r rune) bool {
return false
}
// ScanWords is a split function for a [Scanner] that returns each
// ScanWords is a split function for a Scanner that returns each
// space-separated word of text, with surrounding spaces deleted. It will
// never return an empty string. The definition of space is set by
// unicode.IsSpace.

View File

@@ -68,7 +68,7 @@ func TestScanRune(t *testing.T) {
var i, runeCount int
var expect rune
// Use a string range loop to validate the sequence of runes.
for i, expect = range test {
for i, expect = range string(test) {
if !s.Scan() {
break
}

View File

@@ -6,12 +6,8 @@
# Usage: buildall.bash [-e] [pattern]
#
# buildall.bash builds the standard library for all Go-supported
# architectures.
#
# Originally the Go build system used it as a smoke test to quickly
# flag portability issues in builders named "misc-compile" or "all-compile".
# As of CL 464955, the build system uses make.bash -compile-only instead,
# so this script no longer runs in any automated fashion.
# architectures. It is used by the "misc-compile" trybot builders,
# as a smoke test to quickly flag portability issues.
#
# Options:
# -e: stop at first failure
@@ -41,7 +37,7 @@ GOROOT="$(cd .. && pwd)"
gettargets() {
../bin/go tool dist list | sed -e 's|/|-|' |
grep -E -v '^(android|ios)' # need C toolchain even for cross-compiling
egrep -v '^(android|ios)' # need C toolchain even for cross-compiling
echo linux-arm-arm5
}

View File

@@ -53,10 +53,10 @@ type int32 int32
// Range: -9223372036854775808 through 9223372036854775807.
type int64 int64
// float32 is the set of all IEEE 754 32-bit floating-point numbers.
// float32 is the set of all IEEE-754 32-bit floating-point numbers.
type float32 float32
// float64 is the set of all IEEE 754 64-bit floating-point numbers.
// float64 is the set of all IEEE-754 64-bit floating-point numbers.
type float64 float64
// complex64 is the set of all complex numbers with float32 real and
@@ -284,10 +284,9 @@ func panic(v any)
// by restoring normal execution and retrieves the error value passed to the
// call of panic. If recover is called outside the deferred function it will
// not stop a panicking sequence. In this case, or when the goroutine is not
// panicking, recover returns nil.
//
// Prior to Go 1.21, recover would also return nil if panic is called with
// a nil argument. See [panic] for details.
// panicking, or if the argument supplied to panic was nil, recover returns
// nil. Thus the return value from recover reports whether the goroutine is
// panicking.
func recover() any
// The print built-in function formats its arguments in an

View File

@@ -98,18 +98,3 @@ func TestIndexNearPageBoundary(t *testing.T) {
}
q[len(q)-1] = 0
}
func TestCountNearPageBoundary(t *testing.T) {
t.Parallel()
b := dangerousSlice(t)
for i := range b {
c := Count(b[i:], []byte{1})
if c != 0 {
t.Fatalf("Count(b[%d:], {1})=%d, want 0\n", i, c)
}
c = Count(b[:i], []byte{0})
if c != i {
t.Fatalf("Count(b[:%d], {0})=%d, want %d\n", i, c, i)
}
}
}

View File

@@ -15,7 +15,7 @@ import (
// smallBufferSize is an initial allocation minimal capacity.
const smallBufferSize = 64
// A Buffer is a variable-sized buffer of bytes with [Buffer.Read] and [Buffer.Write] methods.
// A Buffer is a variable-sized buffer of bytes with Read and Write methods.
// The zero value for Buffer is an empty buffer ready to use.
type Buffer struct {
buf []byte // contents are the bytes buf[off : len(buf)]
@@ -48,21 +48,21 @@ const maxInt = int(^uint(0) >> 1)
// Bytes returns a slice of length b.Len() holding the unread portion of the buffer.
// The slice is valid for use only until the next buffer modification (that is,
// only until the next call to a method like [Buffer.Read], [Buffer.Write], [Buffer.Reset], or [Buffer.Truncate]).
// only until the next call to a method like Read, Write, Reset, or Truncate).
// The slice aliases the buffer content at least until the next buffer modification,
// so immediate changes to the slice will affect the result of future reads.
func (b *Buffer) Bytes() []byte { return b.buf[b.off:] }
// AvailableBuffer returns an empty buffer with b.Available() capacity.
// This buffer is intended to be appended to and
// passed to an immediately succeeding [Buffer.Write] call.
// passed to an immediately succeeding Write call.
// The buffer is only valid until the next write operation on b.
func (b *Buffer) AvailableBuffer() []byte { return b.buf[len(b.buf):] }
// String returns the contents of the unread portion of the buffer
// as a string. If the [Buffer] is a nil pointer, it returns "<nil>".
// as a string. If the Buffer is a nil pointer, it returns "<nil>".
//
// To build strings more efficiently, see the [strings.Builder] type.
// To build strings more efficiently, see the strings.Builder type.
func (b *Buffer) String() string {
if b == nil {
// Special case, useful in debugging.
@@ -102,7 +102,7 @@ func (b *Buffer) Truncate(n int) {
// Reset resets the buffer to be empty,
// but it retains the underlying storage for use by future writes.
// Reset is the same as [Buffer.Truncate](0).
// Reset is the same as Truncate(0).
func (b *Buffer) Reset() {
b.buf = b.buf[:0]
b.off = 0
@@ -160,7 +160,7 @@ func (b *Buffer) grow(n int) int {
// another n bytes. After Grow(n), at least n bytes can be written to the
// buffer without another allocation.
// If n is negative, Grow will panic.
// If the buffer can't grow it will panic with [ErrTooLarge].
// If the buffer can't grow it will panic with ErrTooLarge.
func (b *Buffer) Grow(n int) {
if n < 0 {
panic("bytes.Buffer.Grow: negative count")
@@ -171,7 +171,7 @@ func (b *Buffer) Grow(n int) {
// Write appends the contents of p to the buffer, growing the buffer as
// needed. The return value n is the length of p; err is always nil. If the
// buffer becomes too large, Write will panic with [ErrTooLarge].
// buffer becomes too large, Write will panic with ErrTooLarge.
func (b *Buffer) Write(p []byte) (n int, err error) {
b.lastRead = opInvalid
m, ok := b.tryGrowByReslice(len(p))
@@ -183,7 +183,7 @@ func (b *Buffer) Write(p []byte) (n int, err error) {
// WriteString appends the contents of s to the buffer, growing the buffer as
// needed. The return value n is the length of s; err is always nil. If the
// buffer becomes too large, WriteString will panic with [ErrTooLarge].
// buffer becomes too large, WriteString will panic with ErrTooLarge.
func (b *Buffer) WriteString(s string) (n int, err error) {
b.lastRead = opInvalid
m, ok := b.tryGrowByReslice(len(s))
@@ -193,16 +193,16 @@ func (b *Buffer) WriteString(s string) (n int, err error) {
return copy(b.buf[m:], s), nil
}
// MinRead is the minimum slice size passed to a [Buffer.Read] call by
// [Buffer.ReadFrom]. As long as the [Buffer] has at least MinRead bytes beyond
// what is required to hold the contents of r, [Buffer.ReadFrom] will not grow the
// MinRead is the minimum slice size passed to a Read call by
// Buffer.ReadFrom. As long as the Buffer has at least MinRead bytes beyond
// what is required to hold the contents of r, ReadFrom will not grow the
// underlying buffer.
const MinRead = 512
// ReadFrom reads data from r until EOF and appends it to the buffer, growing
// the buffer as needed. The return value n is the number of bytes read. Any
// error except io.EOF encountered during the read is also returned. If the
// buffer becomes too large, ReadFrom will panic with [ErrTooLarge].
// buffer becomes too large, ReadFrom will panic with ErrTooLarge.
func (b *Buffer) ReadFrom(r io.Reader) (n int64, err error) {
b.lastRead = opInvalid
for {
@@ -247,13 +247,13 @@ func growSlice(b []byte, n int) []byte {
c = 2 * cap(b)
}
b2 := append([]byte(nil), make([]byte, c)...)
i := copy(b2, b)
return b2[:i]
copy(b2, b)
return b2[:len(b)]
}
// WriteTo writes data to w until the buffer is drained or an error occurs.
// The return value n is the number of bytes written; it always fits into an
// int, but it is int64 to match the [io.WriterTo] interface. Any error
// int, but it is int64 to match the io.WriterTo interface. Any error
// encountered during the write is also returned.
func (b *Buffer) WriteTo(w io.Writer) (n int64, err error) {
b.lastRead = opInvalid
@@ -279,9 +279,9 @@ func (b *Buffer) WriteTo(w io.Writer) (n int64, err error) {
}
// WriteByte appends the byte c to the buffer, growing the buffer as needed.
// The returned error is always nil, but is included to match [bufio.Writer]'s
// The returned error is always nil, but is included to match bufio.Writer's
// WriteByte. If the buffer becomes too large, WriteByte will panic with
// [ErrTooLarge].
// ErrTooLarge.
func (b *Buffer) WriteByte(c byte) error {
b.lastRead = opInvalid
m, ok := b.tryGrowByReslice(1)
@@ -294,8 +294,8 @@ func (b *Buffer) WriteByte(c byte) error {
// WriteRune appends the UTF-8 encoding of Unicode code point r to the
// buffer, returning its length and an error, which is always nil but is
// included to match [bufio.Writer]'s WriteRune. The buffer is grown as needed;
// if it becomes too large, WriteRune will panic with [ErrTooLarge].
// included to match bufio.Writer's WriteRune. The buffer is grown as needed;
// if it becomes too large, WriteRune will panic with ErrTooLarge.
func (b *Buffer) WriteRune(r rune) (n int, err error) {
// Compare as uint32 to correctly handle negative runes.
if uint32(r) < utf8.RuneSelf {
@@ -313,7 +313,7 @@ func (b *Buffer) WriteRune(r rune) (n int, err error) {
// Read reads the next len(p) bytes from the buffer or until the buffer
// is drained. The return value n is the number of bytes read. If the
// buffer has no data to return, err is [io.EOF] (unless len(p) is zero);
// buffer has no data to return, err is io.EOF (unless len(p) is zero);
// otherwise it is nil.
func (b *Buffer) Read(p []byte) (n int, err error) {
b.lastRead = opInvalid
@@ -334,7 +334,7 @@ func (b *Buffer) Read(p []byte) (n int, err error) {
}
// Next returns a slice containing the next n bytes from the buffer,
// advancing the buffer as if the bytes had been returned by [Buffer.Read].
// advancing the buffer as if the bytes had been returned by Read.
// If there are fewer than n bytes in the buffer, Next returns the entire buffer.
// The slice is only valid until the next call to a read or write method.
func (b *Buffer) Next(n int) []byte {
@@ -352,7 +352,7 @@ func (b *Buffer) Next(n int) []byte {
}
// ReadByte reads and returns the next byte from the buffer.
// If no byte is available, it returns error [io.EOF].
// If no byte is available, it returns error io.EOF.
func (b *Buffer) ReadByte() (byte, error) {
if b.empty() {
// Buffer is empty, reset to recover space.
@@ -388,10 +388,10 @@ func (b *Buffer) ReadRune() (r rune, size int, err error) {
return r, n, nil
}
// UnreadRune unreads the last rune returned by [Buffer.ReadRune].
// UnreadRune unreads the last rune returned by ReadRune.
// If the most recent read or write operation on the buffer was
// not a successful [Buffer.ReadRune], UnreadRune returns an error. (In this regard
// it is stricter than [Buffer.UnreadByte], which will unread the last byte
// not a successful ReadRune, UnreadRune returns an error. (In this regard
// it is stricter than UnreadByte, which will unread the last byte
// from any read operation.)
func (b *Buffer) UnreadRune() error {
if b.lastRead <= opInvalid {
@@ -424,7 +424,7 @@ func (b *Buffer) UnreadByte() error {
// ReadBytes reads until the first occurrence of delim in the input,
// returning a slice containing the data up to and including the delimiter.
// If ReadBytes encounters an error before finding a delimiter,
// it returns the data read before the error and the error itself (often [io.EOF]).
// it returns the data read before the error and the error itself (often io.EOF).
// ReadBytes returns err != nil if and only if the returned data does not end in
// delim.
func (b *Buffer) ReadBytes(delim byte) (line []byte, err error) {
@@ -452,7 +452,7 @@ func (b *Buffer) readSlice(delim byte) (line []byte, err error) {
// ReadString reads until the first occurrence of delim in the input,
// returning a string containing the data up to and including the delimiter.
// If ReadString encounters an error before finding a delimiter,
// it returns the data read before the error and the error itself (often [io.EOF]).
// it returns the data read before the error and the error itself (often io.EOF).
// ReadString returns err != nil if and only if the returned data does not end
// in delim.
func (b *Buffer) ReadString(delim byte) (line string, err error) {
@@ -460,23 +460,23 @@ func (b *Buffer) ReadString(delim byte) (line string, err error) {
return string(slice), err
}
// NewBuffer creates and initializes a new [Buffer] using buf as its
// initial contents. The new [Buffer] takes ownership of buf, and the
// NewBuffer creates and initializes a new Buffer using buf as its
// initial contents. The new Buffer takes ownership of buf, and the
// caller should not use buf after this call. NewBuffer is intended to
// prepare a [Buffer] to read existing data. It can also be used to set
// prepare a Buffer to read existing data. It can also be used to set
// the initial size of the internal buffer for writing. To do that,
// buf should have the desired capacity but a length of zero.
//
// In most cases, new([Buffer]) (or just declaring a [Buffer] variable) is
// sufficient to initialize a [Buffer].
// In most cases, new(Buffer) (or just declaring a Buffer variable) is
// sufficient to initialize a Buffer.
func NewBuffer(buf []byte) *Buffer { return &Buffer{buf: buf} }
// NewBufferString creates and initializes a new [Buffer] using string s as its
// NewBufferString creates and initializes a new Buffer using string s as its
// initial contents. It is intended to prepare a buffer to read an existing
// string.
//
// In most cases, new([Buffer]) (or just declaring a [Buffer] variable) is
// sufficient to initialize a [Buffer].
// In most cases, new(Buffer) (or just declaring a Buffer variable) is
// sufficient to initialize a Buffer.
func NewBufferString(s string) *Buffer {
return &Buffer{buf: []byte(s)}
}

View File

@@ -7,7 +7,6 @@ package bytes_test
import (
. "bytes"
"fmt"
"internal/testenv"
"io"
"math/rand"
"strconv"
@@ -95,22 +94,6 @@ func TestNewBuffer(t *testing.T) {
check(t, "NewBuffer", buf, testString)
}
var buf Buffer
// Calling NewBuffer and immediately shallow copying the Buffer struct
// should not result in any allocations.
// This can be used to reset the underlying []byte of an existing Buffer.
func TestNewBufferShallow(t *testing.T) {
testenv.SkipIfOptimizationOff(t)
n := testing.AllocsPerRun(1000, func() {
buf = *NewBuffer(testBytes)
})
if n > 0 {
t.Errorf("allocations occurred while shallow copying")
}
check(t, "NewBuffer", &buf, testString)
}
func TestNewBufferString(t *testing.T) {
buf := NewBufferString(testString)
check(t, "NewBufferString", buf, testString)
@@ -213,7 +196,7 @@ func TestLargeByteWrites(t *testing.T) {
func TestLargeStringReads(t *testing.T) {
var buf Buffer
for i := 3; i < 30; i += 3 {
s := fillString(t, "TestLargeReads (1)", &buf, "", 5, testString[:len(testString)/i])
s := fillString(t, "TestLargeReads (1)", &buf, "", 5, testString[0:len(testString)/i])
empty(t, "TestLargeReads (2)", &buf, s, make([]byte, len(testString)))
}
check(t, "TestLargeStringReads (3)", &buf, "")
@@ -222,7 +205,7 @@ func TestLargeStringReads(t *testing.T) {
func TestLargeByteReads(t *testing.T) {
var buf Buffer
for i := 3; i < 30; i += 3 {
s := fillBytes(t, "TestLargeReads (1)", &buf, "", 5, testBytes[:len(testBytes)/i])
s := fillBytes(t, "TestLargeReads (1)", &buf, "", 5, testBytes[0:len(testBytes)/i])
empty(t, "TestLargeReads (2)", &buf, s, make([]byte, len(testString)))
}
check(t, "TestLargeByteReads (3)", &buf, "")
@@ -274,7 +257,7 @@ func TestNil(t *testing.T) {
func TestReadFrom(t *testing.T) {
var buf Buffer
for i := 3; i < 30; i += 3 {
s := fillBytes(t, "TestReadFrom (1)", &buf, "", 5, testBytes[:len(testBytes)/i])
s := fillBytes(t, "TestReadFrom (1)", &buf, "", 5, testBytes[0:len(testBytes)/i])
var b Buffer
b.ReadFrom(&buf)
empty(t, "TestReadFrom (2)", &b, s, make([]byte, len(testString)))
@@ -337,7 +320,7 @@ func TestReadFromNegativeReader(t *testing.T) {
func TestWriteTo(t *testing.T) {
var buf Buffer
for i := 3; i < 30; i += 3 {
s := fillBytes(t, "TestWriteTo (1)", &buf, "", 5, testBytes[:len(testBytes)/i])
s := fillBytes(t, "TestWriteTo (1)", &buf, "", 5, testBytes[0:len(testBytes)/i])
var b Buffer
buf.WriteTo(&b)
empty(t, "TestWriteTo (2)", &b, s, make([]byte, len(testString)))

View File

@@ -8,10 +8,8 @@ package bytes
import (
"internal/bytealg"
"math/bits"
"unicode"
"unicode/utf8"
_ "unsafe" // for linkname
)
// Equal reports whether a and b
@@ -114,7 +112,7 @@ func LastIndex(s, sep []byte) int {
case n == 0:
return len(s)
case n == 1:
return bytealg.LastIndexByte(s, sep[0])
return LastIndexByte(s, sep[0])
case n == len(s):
if Equal(s, sep) {
return 0
@@ -123,21 +121,43 @@ func LastIndex(s, sep []byte) int {
case n > len(s):
return -1
}
return bytealg.LastIndexRabinKarp(s, sep)
// Rabin-Karp search from the end of the string
hashss, pow := bytealg.HashStrRevBytes(sep)
last := len(s) - n
var h uint32
for i := len(s) - 1; i >= last; i-- {
h = h*bytealg.PrimeRK + uint32(s[i])
}
if h == hashss && Equal(s[last:], sep) {
return last
}
for i := last - 1; i >= 0; i-- {
h *= bytealg.PrimeRK
h += uint32(s[i])
h -= pow * uint32(s[i+n])
if h == hashss && Equal(s[i:i+n], sep) {
return i
}
}
return -1
}
// LastIndexByte returns the index of the last instance of c in s, or -1 if c is not present in s.
func LastIndexByte(s []byte, c byte) int {
return bytealg.LastIndexByte(s, c)
for i := len(s) - 1; i >= 0; i-- {
if s[i] == c {
return i
}
}
return -1
}
// IndexRune interprets s as a sequence of UTF-8-encoded code points.
// It returns the byte index of the first occurrence in s of the given rune.
// It returns -1 if rune is not present in s.
// If r is [utf8.RuneError], it returns the first instance of any
// If r is utf8.RuneError, it returns the first instance of any
// invalid UTF-8 byte sequence.
func IndexRune(s []byte, r rune) int {
const haveFastIndex = bytealg.MaxBruteForce > 0
switch {
case 0 <= r && r < utf8.RuneSelf:
return IndexByte(s, byte(r))
@@ -153,64 +173,9 @@ func IndexRune(s []byte, r rune) int {
case !utf8.ValidRune(r):
return -1
default:
// Search for rune r using the last byte of its UTF-8 encoded form.
// The distribution of the last byte is more uniform compared to the
// first byte which has a 78% chance of being [240, 243, 244].
var b [utf8.UTFMax]byte
n := utf8.EncodeRune(b[:], r)
last := n - 1
i := last
fails := 0
for i < len(s) {
if s[i] != b[last] {
o := IndexByte(s[i+1:], b[last])
if o < 0 {
return -1
}
i += o + 1
}
// Step backwards comparing bytes.
for j := 1; j < n; j++ {
if s[i-j] != b[last-j] {
goto next
}
}
return i - last
next:
fails++
i++
if (haveFastIndex && fails > bytealg.Cutover(i)) && i < len(s) ||
(!haveFastIndex && fails >= 4+i>>4 && i < len(s)) {
goto fallback
}
}
return -1
fallback:
// Switch to bytealg.Index, if available, or a brute force search when
// IndexByte returns too many false positives.
if haveFastIndex {
if j := bytealg.Index(s[i-last:], b[:n]); j >= 0 {
return i + j - last
}
} else {
// If bytealg.Index is not available a brute force search is
// ~1.5-3x faster than Rabin-Karp since n is small.
c0 := b[last]
c1 := b[last-1] // There are at least 2 chars to match
loop:
for ; i < len(s); i++ {
if s[i] == c0 && s[i-1] == c1 {
for k := 2; k < n; k++ {
if s[i-k] != b[last-k] {
continue loop
}
}
return i - last
}
}
}
return -1
return Index(s, b[:n])
}
}
@@ -412,20 +377,22 @@ func genSplit(s, sep []byte, sepSave, n int) [][]byte {
// the subslices between those separators.
// If sep is empty, SplitN splits after each UTF-8 sequence.
// The count determines the number of subslices to return:
// - n > 0: at most n subslices; the last subslice will be the unsplit remainder;
// - n == 0: the result is nil (zero subslices);
// - n < 0: all subslices.
//
// To split around the first instance of a separator, see [Cut].
// n > 0: at most n subslices; the last subslice will be the unsplit remainder.
// n == 0: the result is nil (zero subslices)
// n < 0: all subslices
//
// To split around the first instance of a separator, see Cut.
func SplitN(s, sep []byte, n int) [][]byte { return genSplit(s, sep, 0, n) }
// SplitAfterN slices s into subslices after each instance of sep and
// returns a slice of those subslices.
// If sep is empty, SplitAfterN splits after each UTF-8 sequence.
// The count determines the number of subslices to return:
// - n > 0: at most n subslices; the last subslice will be the unsplit remainder;
// - n == 0: the result is nil (zero subslices);
// - n < 0: all subslices.
//
// n > 0: at most n subslices; the last subslice will be the unsplit remainder.
// n == 0: the result is nil (zero subslices)
// n < 0: all subslices
func SplitAfterN(s, sep []byte, n int) [][]byte {
return genSplit(s, sep, len(sep), n)
}
@@ -435,7 +402,7 @@ func SplitAfterN(s, sep []byte, n int) [][]byte {
// If sep is empty, Split splits after each UTF-8 sequence.
// It is equivalent to SplitN with a count of -1.
//
// To split around the first instance of a separator, see [Cut].
// To split around the first instance of a separator, see Cut.
func Split(s, sep []byte) [][]byte { return genSplit(s, sep, 0, -1) }
// SplitAfter slices s into all subslices after each instance of sep and
@@ -450,7 +417,7 @@ var asciiSpace = [256]uint8{'\t': 1, '\n': 1, '\v': 1, '\f': 1, '\r': 1, ' ': 1}
// Fields interprets s as a sequence of UTF-8-encoded code points.
// It splits the slice s around each instance of one or more consecutive white space
// characters, as defined by [unicode.IsSpace], returning a slice of subslices of s or an
// characters, as defined by unicode.IsSpace, returning a slice of subslices of s or an
// empty slice if s contains only white space.
func Fields(s []byte) [][]byte {
// First count the fields.
@@ -581,7 +548,7 @@ func Join(s [][]byte, sep []byte) []byte {
n += len(v)
}
b := bytealg.MakeNoZero(n)[:n:n]
b := bytealg.MakeNoZero(n)
bp := copy(b, s[0])
for _, v := range s[1:] {
bp += copy(b[bp:], sep)
@@ -590,12 +557,12 @@ func Join(s [][]byte, sep []byte) []byte {
return b
}
// HasPrefix reports whether the byte slice s begins with prefix.
// HasPrefix tests whether the byte slice s begins with prefix.
func HasPrefix(s, prefix []byte) bool {
return len(s) >= len(prefix) && Equal(s[:len(prefix)], prefix)
return len(s) >= len(prefix) && Equal(s[0:len(prefix)], prefix)
}
// HasSuffix reports whether the byte slice s ends with suffix.
// HasSuffix tests whether the byte slice s ends with suffix.
func HasSuffix(s, suffix []byte) bool {
return len(s) >= len(suffix) && Equal(s[len(s)-len(suffix):], suffix)
}
@@ -624,18 +591,6 @@ func Map(mapping func(r rune) rune, s []byte) []byte {
return b
}
// Despite being an exported symbol,
// Repeat is linknamed by widely used packages.
// Notable members of the hall of shame include:
// - gitee.com/quant1x/num
//
// Do not remove or change the type signature.
// See go.dev/issue/67401.
//
// Note that this comment is not part of the doc comment.
//
//go:linkname Repeat
// Repeat returns a new byte slice consisting of count copies of b.
//
// It panics if count is negative or if the result of (len(b) * count)
@@ -651,11 +606,10 @@ func Repeat(b []byte, count int) []byte {
if count < 0 {
panic("bytes: negative Repeat count")
}
hi, lo := bits.Mul(uint(len(b)), uint(count))
if hi > 0 || lo > uint(maxInt) {
if len(b) >= maxInt/count {
panic("bytes: Repeat output length overflow")
}
n := int(lo) // lo = len(b) * count
n := len(b) * count
if len(b) == 0 {
return []byte{}
@@ -679,10 +633,13 @@ func Repeat(b []byte, count int) []byte {
chunkMax = len(b)
}
}
nb := bytealg.MakeNoZero(n)[:n:n]
nb := bytealg.MakeNoZero(n)
bp := copy(nb, b)
for bp < n {
chunk := min(bp, chunkMax)
chunk := bp
if chunk > chunkMax {
chunk = chunkMax
}
bp += copy(nb[bp:], nb[:chunk])
}
return nb
@@ -706,7 +663,7 @@ func ToUpper(s []byte) []byte {
// Just return a copy.
return append([]byte(""), s...)
}
b := bytealg.MakeNoZero(len(s))[:len(s):len(s)]
b := bytealg.MakeNoZero(len(s))
for i := 0; i < len(s); i++ {
c := s[i]
if 'a' <= c && c <= 'z' {
@@ -736,7 +693,7 @@ func ToLower(s []byte) []byte {
if !hasUpper {
return append([]byte(""), s...)
}
b := bytealg.MakeNoZero(len(s))[:len(s):len(s)]
b := bytealg.MakeNoZero(len(s))
for i := 0; i < len(s); i++ {
c := s[i]
if 'A' <= c && c <= 'Z' {
@@ -1379,7 +1336,7 @@ func Index(s, sep []byte) int {
// we should cutover at even larger average skips,
// because Equal becomes that much more expensive.
// This code does not take that effect into account.
j := bytealg.IndexRabinKarp(s[i:], sep)
j := bytealg.IndexRabinKarpBytes(s[i:], sep)
if j < 0 {
return -1
}

View File

@@ -1,21 +0,0 @@
// Copyright 2024 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build js && wasm
package bytes_test
import (
"bytes"
"testing"
)
func TestIssue65571(t *testing.T) {
b := make([]byte, 1<<31+1)
b[1<<31] = 1
i := bytes.IndexByte(b, 1)
if i != 1<<31 {
t.Errorf("IndexByte(b, 1) = %d; want %d", i, 1<<31)
}
}

View File

@@ -8,10 +8,9 @@ import (
. "bytes"
"fmt"
"internal/testenv"
"iter"
"math"
"math/rand"
"slices"
"reflect"
"strings"
"testing"
"unicode"
@@ -19,6 +18,18 @@ import (
"unsafe"
)
func eq(a, b []string) bool {
if len(a) != len(b) {
return false
}
for i := 0; i < len(a); i++ {
if a[i] != b[i] {
return false
}
}
return true
}
func sliceOfString(s [][]byte) []string {
result := make([]string, len(s))
for i, v := range s {
@@ -27,37 +38,6 @@ func sliceOfString(s [][]byte) []string {
return result
}
func collect(t *testing.T, seq iter.Seq[[]byte]) [][]byte {
out := slices.Collect(seq)
out1 := slices.Collect(seq)
if !slices.Equal(sliceOfString(out), sliceOfString(out1)) {
t.Fatalf("inconsistent seq:\n%s\n%s", out, out1)
}
return out
}
type LinesTest struct {
a string
b []string
}
var linesTests = []LinesTest{
{a: "abc\nabc\n", b: []string{"abc\n", "abc\n"}},
{a: "abc\r\nabc", b: []string{"abc\r\n", "abc"}},
{a: "abc\r\n", b: []string{"abc\r\n"}},
{a: "\nabc", b: []string{"\n", "abc"}},
{a: "\nabc\n\n", b: []string{"\n", "abc\n", "\n"}},
}
func TestLines(t *testing.T) {
for _, s := range linesTests {
result := sliceOfString(slices.Collect(Lines([]byte(s.a))))
if !slices.Equal(result, s.b) {
t.Errorf(`slices.Collect(Lines(%q)) = %q; want %q`, s.a, result, s.b)
}
}
}
// For ease of reading, the test cases use strings that are converted to byte
// slices before invoking the functions.
@@ -197,11 +177,6 @@ var indexTests = []BinOpTest{
{"oxoxoxoxoxoxoxoxoxoxoxox", "oy", -1},
// test fallback to Rabin-Karp.
{"000000000000000000000000000000000000000000000000000000000000000000000001", "0000000000000000000000000000000000000000000000000000000000000000001", 5},
// test fallback to IndexRune
{"oxoxoxoxoxoxoxoxoxoxox☺", "☺", 22},
// invalid UTF-8 byte sequence (must be longer than bytealg.MaxBruteForce to
// test that we don't use IndexRune)
{"xx0123456789012345678901234567890123456789012345678901234567890120123456789012345678901234567890123456xxx\xed\x9f\xc0", "\xed\x9f\xc0", 105},
}
var lastIndexTests = []BinOpTest{
@@ -450,31 +425,6 @@ func TestIndexRune(t *testing.T) {
{"some_text=some_value", '=', 9},
{"☺a", 'a', 3},
{"a☻☺b", '☺', 4},
{"𠀳𠀗𠀾𠁄𠀧𠁆𠁂𠀫𠀖𠀪𠀲𠀴𠁀𠀨𠀿", '𠀿', 56},
// 2 bytes
{"ӆ", 'ӆ', 0},
{"a", 'ӆ', -1},
{" ӆ", 'ӆ', 2},
{" a", 'ӆ', -1},
{strings.Repeat("ц", 64) + "ӆ", 'ӆ', 128}, // test cutover
{strings.Repeat("ц", 64), 'ӆ', -1},
// 3 bytes
{"Ꚁ", 'Ꚁ', 0},
{"a", 'Ꚁ', -1},
{" Ꚁ", 'Ꚁ', 2},
{" a", 'Ꚁ', -1},
{strings.Repeat("Ꙁ", 64) + "Ꚁ", 'Ꚁ', 192}, // test cutover
{strings.Repeat("Ꙁ", 64) + "Ꚁ", '䚀', -1}, // 'Ꚁ' and '䚀' share the same last two bytes
// 4 bytes
{"𡌀", '𡌀', 0},
{"a", '𡌀', -1},
{" 𡌀", '𡌀', 2},
{" a", '𡌀', -1},
{strings.Repeat("𡋀", 64) + "𡌀", '𡌀', 256}, // test cutover
{strings.Repeat("𡋀", 64) + "𡌀", '𣌀', -1}, // '𡌀' and '𣌀' share the same last two bytes
// RuneError should match any invalid UTF-8 byte sequence.
{"<22>", '<27>', 0},
@@ -488,13 +438,6 @@ func TestIndexRune(t *testing.T) {
{"a☺b☻c☹d\xe2\x98<39>\xff<66>\xed\xa0\x80", -1, -1},
{"a☺b☻c☹d\xe2\x98<39>\xff<66>\xed\xa0\x80", 0xD800, -1}, // Surrogate pair
{"a☺b☻c☹d\xe2\x98<39>\xff<66>\xed\xa0\x80", utf8.MaxRune + 1, -1},
// Test the cutover to bytealg.Index when it is triggered in
// the middle of rune that contains consecutive runs of equal bytes.
{"aaaaa\U000bc104", '\U000bc104', 17}, // cutover: (n + 16) / 8
{"aaaaa鄄", '鄄', 17},
{"aaa\U000bc104", '\U000bc104', 18}, // cutover: 4 + n>>4
{"aaa鄄", '鄄', 18},
}
for _, tt := range tests {
if got := IndexRune([]byte(tt.in), tt.rune); got != tt.want {
@@ -642,21 +585,6 @@ func BenchmarkIndexRuneASCII(b *testing.B) {
benchBytes(b, indexSizes, bmIndexRuneASCII(IndexRune))
}
func BenchmarkIndexRuneUnicode(b *testing.B) {
b.Run("Latin", func(b *testing.B) {
// Latin is mostly 1, 2, 3 byte runes.
benchBytes(b, indexSizes, bmIndexRuneUnicode(unicode.Latin, 'é'))
})
b.Run("Cyrillic", func(b *testing.B) {
// Cyrillic is mostly 2 and 3 byte runes.
benchBytes(b, indexSizes, bmIndexRuneUnicode(unicode.Cyrillic, 'Ꙁ'))
})
b.Run("Han", func(b *testing.B) {
// Han consists only of 3 and 4 byte runes.
benchBytes(b, indexSizes, bmIndexRuneUnicode(unicode.Han, '𠀿'))
})
}
func bmIndexRuneASCII(index func([]byte, rune) int) func(b *testing.B, n int) {
return func(b *testing.B, n int) {
buf := bmbuf[0:n]
@@ -687,61 +615,6 @@ func bmIndexRune(index func([]byte, rune) int) func(b *testing.B, n int) {
}
}
func bmIndexRuneUnicode(rt *unicode.RangeTable, needle rune) func(b *testing.B, n int) {
var rs []rune
for _, r16 := range rt.R16 {
for r := rune(r16.Lo); r <= rune(r16.Hi); r += rune(r16.Stride) {
if r != needle {
rs = append(rs, rune(r))
}
}
}
for _, r32 := range rt.R32 {
for r := rune(r32.Lo); r <= rune(r32.Hi); r += rune(r32.Stride) {
if r != needle {
rs = append(rs, rune(r))
}
}
}
// Shuffle the runes so that they are not in descending order.
// The sort is deterministic since this is used for benchmarks,
// which need to be repeatable.
rr := rand.New(rand.NewSource(1))
rr.Shuffle(len(rs), func(i, j int) {
rs[i], rs[j] = rs[j], rs[i]
})
uchars := string(rs)
return func(b *testing.B, n int) {
buf := bmbuf[0:n]
o := copy(buf, uchars)
for o < len(buf) {
o += copy(buf[o:], uchars)
}
// Make space for the needle rune at the end of buf.
m := utf8.RuneLen(needle)
for o := m; o > 0; {
_, sz := utf8.DecodeLastRune(buf)
copy(buf[len(buf)-sz:], "\x00\x00\x00\x00")
buf = buf[:len(buf)-sz]
o -= sz
}
buf = utf8.AppendRune(buf[:n-m], needle)
n -= m // adjust for rune len
for i := 0; i < b.N; i++ {
j := IndexRune(buf, needle)
if j != n {
b.Fatal("bad index", j)
}
}
for i := range buf {
buf[i] = '\x00'
}
}
}
func BenchmarkEqual(b *testing.B) {
b.Run("0", func(b *testing.B) {
var buf [4]byte
@@ -756,11 +629,6 @@ func BenchmarkEqual(b *testing.B) {
})
sizes := []int{1, 6, 9, 15, 16, 20, 32, 4 << 10, 4 << 20, 64 << 20}
b.Run("same", func(b *testing.B) {
benchBytes(b, sizes, bmEqual(func(a, b []byte) bool { return Equal(a, a) }))
})
benchBytes(b, sizes, bmEqual(Equal))
}
@@ -935,18 +803,10 @@ func TestSplit(t *testing.T) {
}
result := sliceOfString(a)
if !slices.Equal(result, tt.a) {
if !eq(result, tt.a) {
t.Errorf(`Split(%q, %q, %d) = %v; want %v`, tt.s, tt.sep, tt.n, result, tt.a)
continue
}
if tt.n < 0 {
b := sliceOfString(slices.Collect(SplitSeq([]byte(tt.s), []byte(tt.sep))))
if !slices.Equal(b, tt.a) {
t.Errorf(`collect(SplitSeq(%q, %q)) = %v; want %v`, tt.s, tt.sep, b, tt.a)
}
}
if tt.n == 0 || len(a) == 0 {
continue
}
@@ -960,8 +820,8 @@ func TestSplit(t *testing.T) {
t.Errorf(`Join(Split(%q, %q, %d), %q) = %q`, tt.s, tt.sep, tt.n, tt.sep, s)
}
if tt.n < 0 {
b := sliceOfString(Split([]byte(tt.s), []byte(tt.sep)))
if !slices.Equal(result, b) {
b := Split([]byte(tt.s), []byte(tt.sep))
if !reflect.DeepEqual(a, b) {
t.Errorf("Split disagrees withSplitN(%q, %q, %d) = %v; want %v", tt.s, tt.sep, tt.n, b, a)
}
}
@@ -1001,18 +861,11 @@ func TestSplitAfter(t *testing.T) {
}
result := sliceOfString(a)
if !slices.Equal(result, tt.a) {
if !eq(result, tt.a) {
t.Errorf(`Split(%q, %q, %d) = %v; want %v`, tt.s, tt.sep, tt.n, result, tt.a)
continue
}
if tt.n < 0 {
b := sliceOfString(slices.Collect(SplitAfterSeq([]byte(tt.s), []byte(tt.sep))))
if !slices.Equal(b, tt.a) {
t.Errorf(`collect(SplitAfterSeq(%q, %q)) = %v; want %v`, tt.s, tt.sep, b, tt.a)
}
}
if want := tt.a[len(tt.a)-1] + "z"; string(x) != want {
t.Errorf("last appended result was %s; want %s", x, want)
}
@@ -1022,8 +875,8 @@ func TestSplitAfter(t *testing.T) {
t.Errorf(`Join(Split(%q, %q, %d), %q) = %q`, tt.s, tt.sep, tt.n, tt.sep, s)
}
if tt.n < 0 {
b := sliceOfString(SplitAfter([]byte(tt.s), []byte(tt.sep)))
if !slices.Equal(result, b) {
b := SplitAfter([]byte(tt.s), []byte(tt.sep))
if !reflect.DeepEqual(a, b) {
t.Errorf("SplitAfter disagrees withSplitAfterN(%q, %q, %d) = %v; want %v", tt.s, tt.sep, tt.n, b, a)
}
}
@@ -1061,16 +914,11 @@ func TestFields(t *testing.T) {
}
result := sliceOfString(a)
if !slices.Equal(result, tt.a) {
if !eq(result, tt.a) {
t.Errorf("Fields(%q) = %v; want %v", tt.s, a, tt.a)
continue
}
result2 := sliceOfString(collect(t, FieldsSeq([]byte(tt.s))))
if !slices.Equal(result2, tt.a) {
t.Errorf(`collect(FieldsSeq(%q)) = %v; want %v`, tt.s, result2, tt.a)
}
if string(b) != tt.s {
t.Errorf("slice changed to %s; want %s", string(b), tt.s)
}
@@ -1086,7 +934,7 @@ func TestFieldsFunc(t *testing.T) {
for _, tt := range fieldstests {
a := FieldsFunc([]byte(tt.s), unicode.IsSpace)
result := sliceOfString(a)
if !slices.Equal(result, tt.a) {
if !eq(result, tt.a) {
t.Errorf("FieldsFunc(%q, unicode.IsSpace) = %v; want %v", tt.s, a, tt.a)
continue
}
@@ -1109,15 +957,10 @@ func TestFieldsFunc(t *testing.T) {
}
result := sliceOfString(a)
if !slices.Equal(result, tt.a) {
if !eq(result, tt.a) {
t.Errorf("FieldsFunc(%q) = %v, want %v", tt.s, a, tt.a)
}
result2 := sliceOfString(collect(t, FieldsFuncSeq([]byte(tt.s), pred)))
if !slices.Equal(result2, tt.a) {
t.Errorf(`collect(FieldsFuncSeq(%q)) = %v; want %v`, tt.s, result2, tt.a)
}
if string(b) != tt.s {
t.Errorf("slice changed to %s; want %s", b, tt.s)
}
@@ -1394,48 +1237,45 @@ func repeat(b []byte, count int) (err error) {
// See Issue golang.org/issue/16237
func TestRepeatCatchesOverflow(t *testing.T) {
type testCase struct {
tests := [...]struct {
s string
count int
errStr string
}
runTestCases := func(prefix string, tests []testCase) {
for i, tt := range tests {
err := repeat([]byte(tt.s), tt.count)
if tt.errStr == "" {
if err != nil {
t.Errorf("#%d panicked %v", i, err)
}
continue
}
if err == nil || !strings.Contains(err.Error(), tt.errStr) {
t.Errorf("%s#%d got %q want %q", prefix, i, err, tt.errStr)
}
}
}
const maxInt = int(^uint(0) >> 1)
runTestCases("", []testCase{
}{
0: {"--", -2147483647, "negative"},
1: {"", maxInt, ""},
1: {"", int(^uint(0) >> 1), ""},
2: {"-", 10, ""},
3: {"gopher", 0, ""},
4: {"-", -1, "negative"},
5: {"--", -102, "negative"},
6: {string(make([]byte, 255)), int((^uint(0))/255 + 1), "overflow"},
})
const is64Bit = 1<<(^uintptr(0)>>63)/2 != 0
if !is64Bit {
return
}
runTestCases("64-bit", []testCase{
0: {"-", maxInt, "out of range"},
})
for i, tt := range tests {
err := repeat([]byte(tt.s), tt.count)
if tt.errStr == "" {
if err != nil {
t.Errorf("#%d panicked %v", i, err)
}
continue
}
if err == nil || !strings.Contains(err.Error(), tt.errStr) {
t.Errorf("#%d expected %q got %q", i, tt.errStr, err)
}
}
}
func runesEqual(a, b []rune) bool {
if len(a) != len(b) {
return false
}
for i, r := range a {
if r != b[i] {
return false
}
}
return true
}
type RunesTest struct {
@@ -1458,7 +1298,7 @@ func TestRunes(t *testing.T) {
for _, tt := range RunesTests {
tin := []byte(tt.in)
a := Runes(tin)
if !slices.Equal(a, tt.out) {
if !runesEqual(a, tt.out) {
t.Errorf("Runes(%q) = %v; want %v", tin, a, tt.out)
continue
}
@@ -2184,11 +2024,6 @@ func makeBenchInputHard() []byte {
var benchInputHard = makeBenchInputHard()
func benchmarkIndexHard(b *testing.B, sep []byte) {
n := Index(benchInputHard, sep)
if n < 0 {
n = len(benchInputHard)
}
b.SetBytes(int64(n))
for i := 0; i < b.N; i++ {
Index(benchInputHard, sep)
}

View File

@@ -10,7 +10,7 @@ import (
"fmt"
"io"
"os"
"slices"
"sort"
"strconv"
"unicode"
)
@@ -81,9 +81,9 @@ func ExampleBuffer_Next() {
var b bytes.Buffer
b.Grow(64)
b.Write([]byte("abcde"))
fmt.Printf("%s\n", b.Next(2))
fmt.Printf("%s\n", b.Next(2))
fmt.Printf("%s", b.Next(2))
fmt.Printf("%s\n", string(b.Next(2)))
fmt.Printf("%s\n", string(b.Next(2)))
fmt.Printf("%s", string(b.Next(2)))
// Output:
// ab
// cd
@@ -102,7 +102,7 @@ func ExampleBuffer_Read() {
fmt.Println(n)
fmt.Println(b.String())
fmt.Println(string(rdbuf))
// Output:
// Output
// 1
// bcde
// a
@@ -118,7 +118,7 @@ func ExampleBuffer_ReadByte() {
}
fmt.Println(c)
fmt.Println(b.String())
// Output:
// Output
// 97
// bcde
}
@@ -165,8 +165,11 @@ func ExampleCompare_search() {
// Binary search to find a matching byte slice.
var needle []byte
var haystack [][]byte // Assume sorted
_, found := slices.BinarySearchFunc(haystack, needle, bytes.Compare)
if found {
i := sort.Search(len(haystack), func(i int) bool {
// Return haystack[i] >= needle.
return bytes.Compare(haystack[i], needle) >= 0
})
if i < len(haystack) && bytes.Equal(haystack[i], needle) {
// Found it!
}
}
@@ -209,17 +212,6 @@ func ExampleContainsRune() {
// false
}
func ExampleContainsFunc() {
f := func(r rune) bool {
return r >= 'a' && r <= 'z'
}
fmt.Println(bytes.ContainsFunc([]byte("HELLO"), f))
fmt.Println(bytes.ContainsFunc([]byte("World"), f))
// Output:
// false
// true
}
func ExampleCount() {
fmt.Println(bytes.Count([]byte("cheese"), []byte("e")))
fmt.Println(bytes.Count([]byte("five"), []byte(""))) // before & after each rune
@@ -502,10 +494,10 @@ func ExampleTitle() {
func ExampleToTitle() {
fmt.Printf("%s\n", bytes.ToTitle([]byte("loud noises")))
fmt.Printf("%s\n", bytes.ToTitle([]byte("брат")))
fmt.Printf("%s\n", bytes.ToTitle([]byte("хлеб")))
// Output:
// LOUD NOISES
// БРАТ
// ХЛЕБ
}
func ExampleToTitleSpecial() {

View File

@@ -1,148 +0,0 @@
// Copyright 2024 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package bytes
import (
"iter"
"unicode"
"unicode/utf8"
)
// Lines returns an iterator over the newline-terminated lines in the byte slice s.
// The lines yielded by the iterator include their terminating newlines.
// If s is empty, the iterator yields no lines at all.
// If s does not end in a newline, the final yielded line will not end in a newline.
// It returns a single-use iterator.
func Lines(s []byte) iter.Seq[[]byte] {
return func(yield func([]byte) bool) {
for len(s) > 0 {
var line []byte
if i := IndexByte(s, '\n'); i >= 0 {
line, s = s[:i+1], s[i+1:]
} else {
line, s = s, nil
}
if !yield(line[:len(line):len(line)]) {
return
}
}
return
}
}
// explodeSeq returns an iterator over the runes in s.
func explodeSeq(s []byte) iter.Seq[[]byte] {
return func(yield func([]byte) bool) {
for len(s) > 0 {
_, size := utf8.DecodeRune(s)
if !yield(s[:size:size]) {
return
}
s = s[size:]
}
}
}
// splitSeq is SplitSeq or SplitAfterSeq, configured by how many
// bytes of sep to include in the results (none or all).
func splitSeq(s, sep []byte, sepSave int) iter.Seq[[]byte] {
if len(sep) == 0 {
return explodeSeq(s)
}
return func(yield func([]byte) bool) {
for {
i := Index(s, sep)
if i < 0 {
break
}
frag := s[:i+sepSave]
if !yield(frag[:len(frag):len(frag)]) {
return
}
s = s[i+len(sep):]
}
yield(s[:len(s):len(s)])
}
}
// SplitSeq returns an iterator over all substrings of s separated by sep.
// The iterator yields the same strings that would be returned by Split(s, sep),
// but without constructing the slice.
// It returns a single-use iterator.
func SplitSeq(s, sep []byte) iter.Seq[[]byte] {
return splitSeq(s, sep, 0)
}
// SplitAfterSeq returns an iterator over substrings of s split after each instance of sep.
// The iterator yields the same strings that would be returned by SplitAfter(s, sep),
// but without constructing the slice.
// It returns a single-use iterator.
func SplitAfterSeq(s, sep []byte) iter.Seq[[]byte] {
return splitSeq(s, sep, len(sep))
}
// FieldsSeq returns an iterator over substrings of s split around runs of
// whitespace characters, as defined by unicode.IsSpace.
// The iterator yields the same strings that would be returned by Fields(s),
// but without constructing the slice.
func FieldsSeq(s []byte) iter.Seq[[]byte] {
return func(yield func([]byte) bool) {
start := -1
for i := 0; i < len(s); {
size := 1
r := rune(s[i])
isSpace := asciiSpace[s[i]] != 0
if r >= utf8.RuneSelf {
r, size = utf8.DecodeRune(s[i:])
isSpace = unicode.IsSpace(r)
}
if isSpace {
if start >= 0 {
if !yield(s[start:i:i]) {
return
}
start = -1
}
} else if start < 0 {
start = i
}
i += size
}
if start >= 0 {
yield(s[start:len(s):len(s)])
}
}
}
// FieldsFuncSeq returns an iterator over substrings of s split around runs of
// Unicode code points satisfying f(c).
// The iterator yields the same strings that would be returned by FieldsFunc(s),
// but without constructing the slice.
func FieldsFuncSeq(s []byte, f func(rune) bool) iter.Seq[[]byte] {
return func(yield func([]byte) bool) {
start := -1
for i := 0; i < len(s); {
size := 1
r := rune(s[i])
if r >= utf8.RuneSelf {
r, size = utf8.DecodeRune(s[i:])
}
if f(r) {
if start >= 0 {
if !yield(s[start:i:i]) {
return
}
start = -1
}
} else if start < 0 {
start = i
}
i += size
}
if start >= 0 {
yield(s[start:len(s):len(s)])
}
}
}

View File

@@ -10,10 +10,10 @@ import (
"unicode/utf8"
)
// A Reader implements the [io.Reader], [io.ReaderAt], [io.WriterTo], [io.Seeker],
// [io.ByteScanner], and [io.RuneScanner] interfaces by reading from
// A Reader implements the io.Reader, io.ReaderAt, io.WriterTo, io.Seeker,
// io.ByteScanner, and io.RuneScanner interfaces by reading from
// a byte slice.
// Unlike a [Buffer], a Reader is read-only and supports seeking.
// Unlike a Buffer, a Reader is read-only and supports seeking.
// The zero value for Reader operates like a Reader of an empty slice.
type Reader struct {
s []byte
@@ -31,11 +31,11 @@ func (r *Reader) Len() int {
}
// Size returns the original length of the underlying byte slice.
// Size is the number of bytes available for reading via [Reader.ReadAt].
// The result is unaffected by any method calls except [Reader.Reset].
// Size is the number of bytes available for reading via ReadAt.
// The result is unaffected by any method calls except Reset.
func (r *Reader) Size() int64 { return int64(len(r.s)) }
// Read implements the [io.Reader] interface.
// Read implements the io.Reader interface.
func (r *Reader) Read(b []byte) (n int, err error) {
if r.i >= int64(len(r.s)) {
return 0, io.EOF
@@ -46,7 +46,7 @@ func (r *Reader) Read(b []byte) (n int, err error) {
return
}
// ReadAt implements the [io.ReaderAt] interface.
// ReadAt implements the io.ReaderAt interface.
func (r *Reader) ReadAt(b []byte, off int64) (n int, err error) {
// cannot modify state - see io.ReaderAt
if off < 0 {
@@ -62,7 +62,7 @@ func (r *Reader) ReadAt(b []byte, off int64) (n int, err error) {
return
}
// ReadByte implements the [io.ByteReader] interface.
// ReadByte implements the io.ByteReader interface.
func (r *Reader) ReadByte() (byte, error) {
r.prevRune = -1
if r.i >= int64(len(r.s)) {
@@ -73,7 +73,7 @@ func (r *Reader) ReadByte() (byte, error) {
return b, nil
}
// UnreadByte complements [Reader.ReadByte] in implementing the [io.ByteScanner] interface.
// UnreadByte complements ReadByte in implementing the io.ByteScanner interface.
func (r *Reader) UnreadByte() error {
if r.i <= 0 {
return errors.New("bytes.Reader.UnreadByte: at beginning of slice")
@@ -83,7 +83,7 @@ func (r *Reader) UnreadByte() error {
return nil
}
// ReadRune implements the [io.RuneReader] interface.
// ReadRune implements the io.RuneReader interface.
func (r *Reader) ReadRune() (ch rune, size int, err error) {
if r.i >= int64(len(r.s)) {
r.prevRune = -1
@@ -99,7 +99,7 @@ func (r *Reader) ReadRune() (ch rune, size int, err error) {
return
}
// UnreadRune complements [Reader.ReadRune] in implementing the [io.RuneScanner] interface.
// UnreadRune complements ReadRune in implementing the io.RuneScanner interface.
func (r *Reader) UnreadRune() error {
if r.i <= 0 {
return errors.New("bytes.Reader.UnreadRune: at beginning of slice")
@@ -112,7 +112,7 @@ func (r *Reader) UnreadRune() error {
return nil
}
// Seek implements the [io.Seeker] interface.
// Seek implements the io.Seeker interface.
func (r *Reader) Seek(offset int64, whence int) (int64, error) {
r.prevRune = -1
var abs int64
@@ -133,7 +133,7 @@ func (r *Reader) Seek(offset int64, whence int) (int64, error) {
return abs, nil
}
// WriteTo implements the [io.WriterTo] interface.
// WriteTo implements the io.WriterTo interface.
func (r *Reader) WriteTo(w io.Writer) (n int64, err error) {
r.prevRune = -1
if r.i >= int64(len(r.s)) {
@@ -152,8 +152,8 @@ func (r *Reader) WriteTo(w io.Writer) (n int64, err error) {
return
}
// Reset resets the [Reader] to be reading from b.
// Reset resets the Reader to be reading from b.
func (r *Reader) Reset(b []byte) { *r = Reader{b, 0, -1} }
// NewReader returns a new [Reader] reading from b.
// NewReader returns a new Reader reading from b.
func NewReader(b []byte) *Reader { return &Reader{b, 0, -1} }

View File

@@ -12,6 +12,7 @@ import (
"path/filepath"
"runtime"
"strings"
"sync"
"testing"
)
@@ -27,6 +28,26 @@ func TestMain(m *testing.M) {
os.Exit(m.Run())
}
// addr2linePath returns the path to the "addr2line" binary to run.
func addr2linePath(t testing.TB) string {
t.Helper()
testenv.MustHaveExec(t)
addr2linePathOnce.Do(func() {
addr2lineExePath, addr2linePathErr = os.Executable()
})
if addr2linePathErr != nil {
t.Fatal(addr2linePathErr)
}
return addr2lineExePath
}
var (
addr2linePathOnce sync.Once
addr2lineExePath string
addr2linePathErr error
)
func loadSyms(t *testing.T, dbgExePath string) map[string]string {
cmd := testenv.Command(t, testenv.GoToolPath(t), "tool", "nm", dbgExePath)
out, err := cmd.CombinedOutput()
@@ -49,7 +70,7 @@ func loadSyms(t *testing.T, dbgExePath string) map[string]string {
}
func runAddr2Line(t *testing.T, dbgExePath, addr string) (funcname, path, lineno string) {
cmd := testenv.Command(t, testenv.Executable(t), dbgExePath)
cmd := testenv.Command(t, addr2linePath(t), dbgExePath)
cmd.Stdin = strings.NewReader(addr)
out, err := cmd.CombinedOutput()
if err != nil {
@@ -87,22 +108,41 @@ func testAddr2Line(t *testing.T, dbgExePath, addr string) {
// Debug paths are stored slash-separated, so convert to system-native.
srcPath = filepath.FromSlash(srcPath)
fi2, err := os.Stat(srcPath)
// If GOROOT_FINAL is set and srcPath is not the file we expect, perhaps
// srcPath has had GOROOT_FINAL substituted for GOROOT and GOROOT hasn't been
// moved to its final location yet. If so, try the original location instead.
if gorootFinal := os.Getenv("GOROOT_FINAL"); gorootFinal != "" &&
(os.IsNotExist(err) || (err == nil && !os.SameFile(fi1, fi2))) {
// srcPath is clean, but GOROOT_FINAL itself might not be.
// (See https://golang.org/issue/41447.)
gorootFinal = filepath.Clean(gorootFinal)
if strings.HasPrefix(srcPath, gorootFinal) {
fi2, err = os.Stat(runtime.GOROOT() + strings.TrimPrefix(srcPath, gorootFinal))
}
}
if err != nil {
t.Fatalf("Stat failed: %v", err)
}
if !os.SameFile(fi1, fi2) {
t.Fatalf("addr2line_test.go and %s are not same file", srcPath)
}
if want := "102"; srcLineNo != want {
t.Fatalf("line number = %v; want %s", srcLineNo, want)
if srcLineNo != "138" {
t.Fatalf("line number = %v; want 138", srcLineNo)
}
}
// This is line 101. The test depends on that.
// This is line 137. The test depends on that.
func TestAddr2Line(t *testing.T) {
testenv.MustHaveGoBuild(t)
tmpDir := t.TempDir()
tmpDir, err := os.MkdirTemp("", "TestAddr2Line")
if err != nil {
t.Fatal("TempDir failed: ", err)
}
defer os.RemoveAll(tmpDir)
// Build copy of test binary with debug symbols,
// since the one running now may not have them.

View File

@@ -28,7 +28,6 @@ import (
"strings"
"cmd/internal/objfile"
"cmd/internal/telemetry/counter"
)
func printUsage(w *os.File) {
@@ -46,7 +45,6 @@ func usage() {
func main() {
log.SetFlags(0)
log.SetPrefix("addr2line: ")
counter.Open()
// pprof expects this behavior when checking for addr2line
if len(os.Args) > 1 && os.Args[1] == "--help" {
@@ -56,8 +54,6 @@ func main() {
flag.Usage = usage
flag.Parse()
counter.Inc("addr2line/invocations")
counter.CountFlags("addr2line/flag:", *flag.CommandLine)
if flag.NArg() != 1 {
usage()
}

View File

@@ -11,7 +11,7 @@ import (
"internal/testenv"
"os"
"path/filepath"
"slices"
"sort"
"strings"
"sync"
"testing"
@@ -77,7 +77,7 @@ func TestGolden(t *testing.T) {
t.Fatalf("opening golden.txt for package %q: %v", fi.Name(), err)
}
wanted := strings.Split(string(bs), "\n")
slices.Sort(wanted)
sort.Strings(wanted)
for _, feature := range wanted {
if feature == "" {
continue
@@ -285,25 +285,6 @@ func TestIssue41358(t *testing.T) {
}
}
func TestIssue64958(t *testing.T) {
defer func() {
if x := recover(); x != nil {
t.Errorf("expected no panic; recovered %v", x)
}
}()
testenv.MustHaveGoBuild(t)
for _, context := range contexts {
w := NewWalker(context, "testdata/src/issue64958")
pkg, err := w.importFrom("p", "", 0)
if err != nil {
t.Errorf("expected no error importing; got %T", err)
}
w.export(pkg)
}
}
func TestCheck(t *testing.T) {
if !*flagCheck {
t.Skip("-check not specified")

View File

@@ -25,7 +25,7 @@ import (
"path/filepath"
"regexp"
"runtime"
"slices"
"sort"
"strconv"
"strings"
"sync"
@@ -106,7 +106,7 @@ func Check(t *testing.T) {
}
var nextFiles []string
if v := runtime.Version(); strings.Contains(v, "devel") || strings.Contains(v, "beta") {
if strings.Contains(runtime.Version(), "devel") {
next, err := filepath.Glob(filepath.Join(testenv.GOROOT(t), "api/next/*.txt"))
if err != nil {
t.Fatal(err)
@@ -232,8 +232,8 @@ func compareAPI(w io.Writer, features, required, exception []string) (ok bool) {
featureSet := set(features)
exceptionSet := set(exception)
slices.Sort(features)
slices.Sort(required)
sort.Strings(features)
sort.Strings(required)
take := func(sl *[]string) string {
s := (*sl)[0]
@@ -378,7 +378,7 @@ func (w *Walker) Features() (fs []string) {
for f := range w.features {
fs = append(fs, f)
}
slices.Sort(fs)
sort.Strings(fs)
return
}
@@ -431,7 +431,7 @@ func tagKey(dir string, context *build.Context, tags []string) string {
// an indirect imported package. See https://github.com/golang/go/issues/21181
// for more detail.
tags = append(tags, context.GOOS, context.GOARCH)
slices.Sort(tags)
sort.Strings(tags)
for _, tag := range tags {
if ctags[tag] {
@@ -490,8 +490,7 @@ func (w *Walker) loadImports() {
if w.context.Dir != "" {
cmd.Dir = w.context.Dir
}
cmd.Stderr = os.Stderr
out, err := cmd.Output()
out, err := cmd.CombinedOutput()
if err != nil {
log.Fatalf("loading imports: %v\n%s", err, out)
}
@@ -535,7 +534,7 @@ func (w *Walker) loadImports() {
}
}
slices.Sort(stdPackages)
sort.Strings(stdPackages)
imports = listImports{
stdPackages: stdPackages,
importMap: importMap,
@@ -717,7 +716,7 @@ func sortedMethodNames(typ *types.Interface) []string {
for i := range list {
list[i] = typ.Method(i).Name()
}
slices.Sort(list)
sort.Strings(list)
return list
}
@@ -747,7 +746,7 @@ func (w *Walker) sortedEmbeddeds(typ *types.Interface) []string {
list = append(list, buf.String())
}
}
slices.Sort(list)
sort.Strings(list)
return list
}
@@ -843,9 +842,6 @@ func (w *Walker) writeType(buf *bytes.Buffer, typ types.Type) {
buf.WriteString(s)
w.writeType(buf, typ.Elem())
case *types.Alias:
w.writeType(buf, types.Unalias(typ))
case *types.Named:
obj := typ.Obj()
pkg := obj.Pkg()
@@ -854,16 +850,6 @@ func (w *Walker) writeType(buf *bytes.Buffer, typ types.Type) {
buf.WriteByte('.')
}
buf.WriteString(typ.Obj().Name())
if targs := typ.TypeArgs(); targs.Len() > 0 {
buf.WriteByte('[')
for i := 0; i < targs.Len(); i++ {
if i > 0 {
buf.WriteString(", ")
}
w.writeType(buf, targs.At(i))
}
buf.WriteByte(']')
}
case *types.TypeParam:
// Type parameter names may change, so use a placeholder instead.
@@ -970,17 +956,17 @@ func (w *Walker) emitType(obj *types.TypeName) {
if w.isDeprecated(obj) {
w.emitf("type %s //deprecated", name)
}
typ := obj.Type()
if obj.IsAlias() {
w.emitf("type %s = %s", name, w.typeString(typ))
return
}
if tparams := obj.Type().(*types.Named).TypeParams(); tparams != nil {
var buf bytes.Buffer
buf.WriteString(name)
w.writeTypeParams(&buf, tparams, true)
name = buf.String()
}
typ := obj.Type()
if obj.IsAlias() {
w.emitf("type %s = %s", name, w.typeString(typ))
return
}
switch typ := typ.Underlying().(type) {
case *types.Struct:
w.emitStructType(name, typ)
@@ -1019,7 +1005,7 @@ func (w *Walker) emitType(obj *types.TypeName) {
func (w *Walker) emitStructType(name string, typ *types.Struct) {
typeStruct := fmt.Sprintf("type %s struct", name)
w.emitf("%s", typeStruct)
w.emitf(typeStruct)
defer w.pushScope(typeStruct)()
for i := 0; i < typ.NumFields(); i++ {
@@ -1083,7 +1069,7 @@ func (w *Walker) emitIfaceType(name string, typ *types.Interface) {
return
}
slices.Sort(methodNames)
sort.Strings(methodNames)
w.emitf("type %s interface { %s }", name, strings.Join(methodNames, ", "))
}

View File

@@ -1,3 +0,0 @@
package p
type BasicAlias = uint8

View File

@@ -1,4 +1,4 @@
pkg p4, func NewPair[$0 interface{ M }, $1 interface{ ~int }]($0, $1) Pair[$0, $1]
pkg p4, func NewPair[$0 interface{ M }, $1 interface{ ~int }]($0, $1) Pair
pkg p4, method (Pair[$0, $1]) Second() $1
pkg p4, method (Pair[$0, $1]) First() $0
pkg p4, type Pair[$0 interface{ M }, $1 interface{ ~int }] struct

Some files were not shown because too many files have changed in this diff Show More