It is ok to clobber registers that have a copy of a fixedreg value,
as that value is always available in its original location later
if we need it. (See 14 lines below the change.)
This CL will fix the regalloc infinite loop that CL 678620 introduced.
That CL requests that the stack pointer value be materialized in a
non-stack-pointer register, which is atypical. That condition
triggered the infinite loop that this CL fixes. The infinite loop is
the compiler trying to reuse that non-stack-pointer register for
something else, but then refusing to give it up because it thought
that non-stack-pointer register held the last copy of the original SP
value.
Fixes#75844
Change-Id: Id604d0937fb9d3753ee273bf1917753d3ef2d5d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/696035
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 9bbea0f21a)
Reviewed-on: https://go-review.googlesource.com/c/go/+/710875
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Constructing the zip index (which is done once when first opening
a file in an archive) can consume large amounts of CPU when
processing deeply-nested directory paths.
Switch to a less inefficient algorithm.
Thanks to Jakub Ciolek for reporting this issue.
goos: darwin
goarch: arm64
pkg: archive/zip
cpu: Apple M4 Pro
│ /tmp/bench.0 │ /tmp/bench.1 │
│ sec/op │ sec/op vs base │
ReaderOneDeepDir-14 25983.62m ± 2% 46.01m ± 2% -99.82% (p=0.000 n=8)
ReaderManyDeepDirs-14 16.221 ± 1% 2.763 ± 6% -82.96% (p=0.000 n=8)
ReaderManyShallowFiles-14 130.3m ± 1% 128.8m ± 2% -1.20% (p=0.003 n=8)
geomean 3.801 253.9m -93.32%
Fixes#77102
Fixes CVE-2025-61728
Change-Id: I2c9c864be01b2a2769eb67fbab1b250aeb8f6c42
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/3060
Reviewed-by: Nicholas Husin <husin@google.com>
Reviewed-by: Neal Patel <nealpatel@google.com>
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/3327
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/736724
TryBot-Bypass: Michael Pratt <mpratt@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
net/url does not currently limit the number of query parameters parsed by
url.ParseQuery or URL.Query.
When parsing a application/x-www-form-urlencoded form,
net/http.Request.ParseForm will parse up to 10 MB of query parameters.
An input consisting of a large number of small, unique parameters can
cause excessive memory consumption.
We now limit the number of query parameters parsed to 10000 by default.
The limit can be adjusted by setting GODEBUG=urlmaxqueryparams=<n>.
Setting urlmaxqueryparams to 0 disables the limit.
Thanks to jub0bs for reporting this issue.
Fixes#77101
Fixes CVE-2025-61726
Change-Id: Iee3374c7ee2d8586dbf158536d3ade424203ff66
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/3020
Reviewed-by: Nicholas Husin <husin@google.com>
Reviewed-by: Neal Patel <nealpatel@google.com>
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/3325
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/736723
Reviewed-by: Junyang Shao <shaojunyang@google.com>
TryBot-Bypass: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
In various situations, the toolchain invokes VCS commands. Some of these
commands take arbitrary input, either provided by users or fetched from
external sources. To prevent potential command injection vulnerabilities
or misinterpretation of arguments as flags, this change updates the VCS
commands to use various techniques to separate flags from positional
arguments, and to directly associate flags with their values.
Additionally, we update the environment variable for Mercurial to use
`HGPLAIN=+strictflags`, which is the more explicit way to disable user
configurations (intended or otherwise) that might interfere with command
execution.
We also now disallow version strings from being prefixed with '-' or
'/', as doing so opens us up to making the same mistake again in the
future. As far as we know there are currently ~0 public modules affected
by this.
While I was working on cmd/go/internal/vcs, I also noticed that a
significant portion of the commands being implemented were dead code.
In order to reduce the maintenance burden and surface area for potential
issues, I removed the dead code for unused commands.
We should probably follow up with a more structured change to make it
harder to accidentally re-introduce these issues in the future, but for
now this addresses the issue at hand.
Thanks to splitline (@splitline) from DEVCORE Research Team for
reporting this issue.
Fixes CVE-2025-68119
Fixes#77099
Change-Id: I9d9f4ee05b95be49fe14edf71a1b8e6c0784378e
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/3260
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Nicholas Husin <husin@google.com>
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/3342
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/736721
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Bypass: Michael Pratt <mpratt@google.com>
Once a tls.Config is used, it is not safe to mutate. We provide the
Clone method in order to allow users to copy and modify a Config that
is in use.
If Config.SessionTicketKey is not populated, and if
Config.SetSessionTicketKeys has not been called, we automatically
populate and rotate session ticket keys. Clone was previously copying
these keys into the new Config, meaning that two Configs could share
the same auto-rotated session ticket keys. This could allow sessions to
be resumed across different Configs, which may have completely different
configurations.
This change updates Clone to not copy the auto-rotated session ticket
keys.
Additionally, when resuming a session, check that not just that the leaf
certificate is unexpired, but that the entire certificate chain is still
unexpired.
Fixes#77113
Fixes CVE-2025-68121
Change-Id: I011df7329de83068d11b3f0c793763692d018a98
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/3300
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Nicholas Husin <husin@google.com>
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/3321
Reviewed-on: https://go-review.googlesource.com/c/go/+/736720
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Bypass: Michael Pratt <mpratt@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
For TLS 1.3, after procesesing the server/client hello, if there isn't a
CCS message, reject the trailing messages which were appended to the
hello messages. This prevents an on-path attacker from injecting
plaintext messages into the handshake.
Additionally, check that we don't have any buffered messages before we
switch the read traffic secret regardless, since any buffered messages
would have been under an old key which is no longer appropriate.
We also invert the ordering of setting the read/write secrets so that if
we fail when changing the read secret we send the alert using the
correct write secret.
Updates #76443Fixes#76855
Fixes CVE-2025-61730
Change-Id: If6ba8ad16f48d5cd5db5574824062ad4244a5b52
Reviewed-on: https://go-review.googlesource.com/c/go/+/724120
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Coia Prant <coiaprant@gmail.com>
(cherry picked from commit 5046bdf8a6)
Reviewed-on: https://go-review.googlesource.com/c/go/+/731960
Reviewed-by: Damien Neil <dneil@google.com>
The ppc64x TSAN wrappers for atomic And/Or did not initialize R6 with the Go argument frame before calling racecallatomic. Since racecallatomic expects R6 to point to the argument list and dereferences it unconditionally, this led to a nil-pointer dereference under -race.
Other atomic TSAN wrappers (Load/Store/Add/Swap/CAS) already set up R6 in the expected way. This change aligns the And/Or wrappers with the rest by adding the missing R6 initialisation.
This keeps the behavior consistent across all atomic operations on ppc64x.
Updates #76776.
Change-Id: Iaf578449a6171a0c6f7c33ec6f64c1251297ae6d
Reviewed-on: https://go-review.googlesource.com/c/go/+/718560
Reviewed-by: Mark Freeman <markfreeman@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Paul Murphy <paumurph@redhat.com>
(cherry picked from commit 44cb82449e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/728900
Reviewed-by: David Chase <drchase@google.com>
Constructing HostnameError.Error() takes O(N^2) runtime due to using a
string concatenation in a loop. Additionally, there is no limit on how
many names are included in the error message. As a result, a malicious
attacker could craft a certificate with an infinite amount of names to
unfairly consume resource.
To remediate this, we will now use strings.Builder to construct the
error message, preventing O(N^2) runtime. When a certificate has 100 or
more names, we will also not print each name individually.
Thanks to Philippe Antoine (Catena cyber) for reporting this issue.
Updates #76445Fixes#76461
Fixes CVE-2025-61729
Change-Id: I6343776ec3289577abc76dad71766c491c1a7c81
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/3000
Reviewed-by: Neal Patel <nealpatel@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/3200
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/725800
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Mark Freeman <markfreeman@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
CL 641955 changes the Unified IR reader to not doing shapify when
reading reshaping expression, prevent losing of the original type.
This is an oversight, as the main problem isn't about shaping during the
reshaping process itself, but about the specific case of shaping a
pointer shape type. This bug occurs when instantiating a generic
function within another generic function with a pointer shape type as
type parameter, which will convert `*[]go.shape.T` to `*go.shape.uint8`,
resulting in the loss of the original expression's type.
This commit changes Unified IR reader to avoid pointer shaping for
`*[]go.shape.T`, ensures that the original type is preserved when
processing reshaping expressions.
Fixes#75480
Change-Id: Icede6b73247d0d367bb485619f2dafb60ad66806
Reviewed-on: https://go-review.googlesource.com/c/go/+/704095
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/706216
Reviewed-by: Ed Schouten <ed@nuxi.nl>
Reviewed-by: Cherry Mui <cherryyz@google.com>
In CL 709854 we enabled strict validation for a number of properties of
domain names (and their constraints). This caused significant breakage,
since we didn't previously disallow the creation of certificates which
contained these malformed domains.
Rollback a number of the properties we enforced, making domainNameValid
only enforce the same properties that domainToReverseLabels does. Since
this also undoes some of the DoS protections our initial fix enabled,
this change also adds caching of constraints in isValid (which perhaps
is the fix we should've initially chosen).
Updates #75835
Updates #75828Fixes#75861
Change-Id: Ie6ca6b4f30e9b8a143692b64757f7bbf4671ed0e
Reviewed-on: https://go-review.googlesource.com/c/go/+/710735
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
(cherry picked from commit 1cd71689f2)
Reviewed-on: https://go-review.googlesource.com/c/go/+/710677
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Sparse files in tar archives contain only the non-zero components
of the file. There are several different encodings for sparse
files. When reading GNU tar pax 1.0 sparse files, archive/tar did
not set a limit on the size of the sparse region data. A malicious
archive containing a large number of sparse blocks could cause
archive/tar to read an unbounded amount of data from the archive
into memory.
Since a malicious input can be highly compressable, a small
compressed input could cause very large allocations.
Cap the size of the sparse block data to the same limit used
for PAX headers (1 MiB).
Thanks to Harshit Gupta (Mr HAX) (https://www.linkedin.com/in/iam-harshit-gupta/)
for reporting this issue.
Fixes CVE-2025-58183
For #75677Fixes#75711
Change-Id: I70b907b584a7b8676df8a149a1db728ae681a770
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2800
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Nicholas Husin <husin@google.com>
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2987
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/709852
TryBot-Bypass: Michael Pratt <mpratt@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
Because Decode scanned the input first for the first BEGIN line, and
then the first END line, the complexity of Decode is quadratic. If the
input contained a large number of BEGINs and then a single END right at
the end of the input, we would find the first BEGIN, and then scan the
entire input for the END, and fail to parse the block, so move onto the
next BEGIN, scan the entire input for the END, etc.
Instead, look for the first END in the input, and then the first BEGIN
that precedes the found END. We then process the bytes between the BEGIN
and END, and move onto the bytes after the END for further processing.
This gives us linear complexity.
Fixes CVE-2025-61723
For #75676Fixes#75709
Change-Id: I813c4f63e78bca4054226c53e13865c781564ccf
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2921
Reviewed-by: Nicholas Husin <husin@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2985
Reviewed-on: https://go-review.googlesource.com/c/go/+/709851
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Bypass: Michael Pratt <mpratt@google.com>
Within parseSequenceOf, reflect.MakeSlice is being used to pre-allocate
a slice that is needed in order to fully validate the given DER payload.
The size of the slice allocated are also multiple times larger than the
input DER:
- When using asn1.Unmarshal directly, the allocated slice is ~28x
larger.
- When passing in DER using x509.ParseCertificateRequest, the allocated
slice is ~48x larger.
- When passing in DER using ocsp.ParseResponse, the allocated slice is
~137x larger.
As a result, a malicious actor can craft a big empty DER payload,
resulting in an unnecessary large allocation of memories. This can be a
way to cause memory exhaustion.
To prevent this, we now use SliceCapWithSize within internal/saferio to
enforce a memory allocation cap.
Thanks to Jakub Ciolek for reporting this issue.
For #75671Fixes#75705
Fixes CVE-2025-58185
Change-Id: Id50e76187eda43f594be75e516b9ca1d2ae6f428
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2700
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2966
Reviewed-by: Nicholas Husin <husin@google.com>
Commit-Queue: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/709850
TryBot-Bypass: Michael Pratt <mpratt@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
When handling HTTP headers, net/http does not currently limit the number
of cookies that can be parsed. The only limitation that exists is for
the size of the entire HTTP header, which is controlled by
MaxHeaderBytes (defaults to 1 MB).
Unfortunately, this allows a malicious actor to send HTTP headers which
contain a massive amount of small cookies, such that as much cookies as
possible can be fitted within the MaxHeaderBytes limitation. Internally,
this causes us to allocate a massive number of Cookie struct.
For example, a 1 MB HTTP header with cookies that repeats "a=;" will
cause an allocation of ~66 MB in the heap. This can serve as a way for
malicious actors to induce memory exhaustion.
To fix this, we will now limit the number of cookies we are willing to
parse to 3000 by default. This behavior can be changed by setting a new
GODEBUG option: GODEBUG=httpcookiemaxnum. httpcookiemaxnum can be set to
allow a higher or lower cookie limit. Setting it to 0 will also allow an
infinite number of cookies to be parsed.
Thanks to jub0bs for reporting this issue.
For #75672Fixes#75707
Fixes CVE-2025-58186
Change-Id: Ied58b3bc8acf5d11c880f881f36ecbf1d5d52622
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2720
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2965
Reviewed-by: Nicholas Husin <husin@google.com>
Commit-Queue: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/709849
TryBot-Bypass: Michael Pratt <mpratt@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
Don't use domainToReverseLabels to check if domain names are valid,
since it is not particularly performant, and can contribute to DoS
vectors. Instead just iterate over the name and enforce the properties
we care about.
This also enforces that DNS names, both in SANs and name constraints,
are valid. We previously allowed invalid SANs, because some
intermediates had these weird names (see #23995), but there are
currently no trusted intermediates that have this property, and since we
target the web PKI, supporting this particular case is not a high
priority.
Thank you to Jakub Ciolek for reporting this issue.
Fixes CVE-2025-58187
For #75681Fixes#75715
Change-Id: I6ebce847dcbe5fc63ef2f9a74f53f11c4c56d3d1
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2820
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2981
Commit-Queue: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Nicholas Husin <husin@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/709848
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Bypass: Michael Pratt <mpratt@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
- Previously, url.Parse did not enforce validation of hostnames within
square brackets.
- RFC 3986 stipulates that only IPv6 hostnames can be embedded within
square brackets in a URL.
- Now, the parsing logic should strictly enforce that only IPv6
hostnames can be resolved when in square brackets. IPv4, IPv4-mapped
addresses and other input will be rejected.
- Update url_test to add test cases that cover the above scenarios.
Thanks to Enze Wang, Jingcheng Yang and Zehui Miao of Tsinghua
University for reporting this issue.
Fixes CVE-2025-47912
For #75678Fixes#75713
Change-Id: Iaa41432bf0ee86de95a39a03adae5729e4deb46c
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2680
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2988
Commit-Queue: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/709847
TryBot-Bypass: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
RFC 5322 domain-literal parsing built the dtext value one character
at a time with string concatenation, resulting in excessive
resource consumption when parsing very large domain-literal values.
Replace with a subslice.
Benchmark not included in this CL because it's too narrow to be
of general ongoing use, but for:
ParseAddress("alice@[" + strings.Repeat("a", 0x40000) + "]")
goos: darwin
goarch: arm64
pkg: net/mail
cpu: Apple M4 Pro
│ /tmp/bench.0 │ /tmp/bench.1 │
│ sec/op │ sec/op vs base │
ParseAddress-14 1987.732m ± 9% 1.524m ± 5% -99.92% (p=0.000 n=10)
│ /tmp/bench.0 │ /tmp/bench.1 │
│ B/op │ B/op vs base │
ParseAddress-14 33692.767Mi ± 0% 1.282Mi ± 0% -100.00% (p=0.000 n=10)
│ /tmp/bench.0 │ /tmp/bench.1 │
│ allocs/op │ allocs/op vs base │
ParseAddress-14 263711.00 ± 0% 17.00 ± 0% -99.99% (p=0.000 n=10)
Thanks to Philippe Antoine (Catena cyber) for reporting this issue.
Fixes CVE-2025-61725
For #75680Fixes#75701
Change-Id: Id971c2d5b59882bb476e22fceb7e01ec08234bb7
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2840
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Nicholas Husin <husin@google.com>
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2961
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/709844
TryBot-Bypass: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Remove a race condition in counting the number of connections per host,
which can cause a connCount underflow and a panic.
The race occurs when:
- A RoundTrip call attempts to use a HTTP/2 roundtripper (pconn.alt != nil)
and receives an isNoCachedConn error. The call removes the pconn from
the idle conn pool and decrements the connCount for its host.
- A second RoundTrip call on the same pconn succeeds,
and delivers the pconn to a third RoundTrip waiting for a conn.
- The third RoundTrip receives the pconn at the same moment its request
context is canceled. It places the pconn back into the idle conn pool.
At this time, the connCount is incorrect, because the conn returned to
the idle pool is not matched by an increment in the connCount.
Fix this by not adding HTTP/2 pconns back to the idle pool in
wantConn.cancel.
For #61474Fixes#75539
Change-Id: I104d6cf85a54d0382eebf3fcf5dda99c69a7c3f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/703936
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Nicholas Husin <husin@google.com>
Reviewed-by: Nicholas Husin <nsh@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 3203a5da29)
Reviewed-on: https://go-review.googlesource.com/c/go/+/705376
Reviewed-by: Cherry Mui <cherryyz@google.com>
The Context.Err documentation states that it returns nil if the
context's done channel is not closed. Fix a race condition introduced
by CL 653795 where Err could return a non-nil error slightly before
the Done channel is closed.
No impact on Err performance when returning nil.
Slows down Err when returning non-nil by about 3x,
but that's still almost 2x faster than before CL 653795
and the performance of this path is less important.
(A tight loop checking Err for doneness will be terminated
by the first Err call to return a non-nil result.)
goos: darwin
goarch: arm64
pkg: context
cpu: Apple M4 Pro
│ /tmp/bench.0 │ /tmp/bench.1 │
│ sec/op │ sec/op vs base │
ErrOK-14 1.806n ± 1% 1.774n ± 0% -1.77% (p=0.000 n=8)
ErrCanceled-14 1.821n ± 1% 7.525n ± 3% +313.23% (p=0.000 n=8)
geomean 1.813n 3.654n +101.47%
Fixes#75533Fixes#75537
Change-Id: Iea22781a199ace7e7f70cf65168c36e090cd2e2a
Reviewed-on: https://go-review.googlesource.com/c/go/+/705235
TryBot-Bypass: Damien Neil <dneil@google.com>
Reviewed-by: Nicholas Husin <husin@google.com>
Reviewed-by: Nicholas Husin <nsh@golang.org>
Auto-Submit: Damien Neil <dneil@google.com>
(cherry picked from commit 8ca209ec39)
Reviewed-on: https://go-review.googlesource.com/c/go/+/705375
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Exceptionally, we decided to make a compliance-related change following
CMVP's updated Implementation Guidance on September 2nd.
The Security Policy will be updated to reflect the new zip hash.
mkzip.go has been modified to accept versions of the form vX.Y.Z-hash,
where the -hash suffix is ignored for fips140.Version() but used to
name the zip file and the unpacked cache directory.
The new zip is generated with
go run ../../src/cmd/go/internal/fips140/mkzip.go -b c2097c7c v1.0.0-c2097c7c
from c2097c7c which is the current release-branch.go1.24 head.
The full diff between the zip file contents is included below.
Fixes#75524
For #74947
Updates #69536
$ diff -ru golang.org/fips140@v1.0.0/fips140/v1.0.0/ golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c
diff -ru golang.org/fips140@v1.0.0/fips140/v1.0.0/cast.go golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/cast.go
--- golang.org/fips140@v1.0.0/fips140/v1.0.0/cast.go 1980-01-10 00:00:00.000000000 +0100
+++ golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/cast.go 1980-01-10 00:00:00.000000000 +0100
@@ -56,9 +56,10 @@
}
// PCT runs the named Pairwise Consistency Test (if operated in FIPS mode) and
-// returns any errors. If an error is returned, the key must not be used.
+// aborts the program (stopping the module input/output and entering the "error
+// state") if the test fails.
//
-// PCTs are mandatory for every key pair that is generated/imported, including
+// PCTs are mandatory for every generated (but not imported) key pair, including
// ephemeral keys (which effectively doubles the cost of key establishment). See
// Implementation Guidance 10.3.A Additional Comment 1.
//
@@ -66,17 +67,23 @@
//
// If a package p calls PCT during key generation, an invocation of that
// function should be added to fipstest.TestConditionals.
-func PCT(name string, f func() error) error {
+func PCT(name string, f func() error) {
if strings.ContainsAny(name, ",#=:") {
panic("fips: invalid self-test name: " + name)
}
if !Enabled {
- return nil
+ return
}
err := f()
if name == failfipscast {
err = errors.New("simulated PCT failure")
}
- return err
+ if err != nil {
+ fatal("FIPS 140-3 self-test failed: " + name + ": " + err.Error())
+ panic("unreachable")
+ }
+ if debug {
+ println("FIPS 140-3 PCT passed:", name)
+ }
}
diff -ru golang.org/fips140@v1.0.0/fips140/v1.0.0/ecdh/ecdh.go golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/ecdh/ecdh.go
--- golang.org/fips140@v1.0.0/fips140/v1.0.0/ecdh/ecdh.go 1980-01-10 00:00:00.000000000 +0100
+++ golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/ecdh/ecdh.go 1980-01-10 00:00:00.000000000 +0100
@@ -161,6 +161,27 @@
if err != nil {
continue
}
+
+ // A "Pairwise Consistency Test" makes no sense if we just generated the
+ // public key from an ephemeral private key. Moreover, there is no way to
+ // check it aside from redoing the exact same computation again. SP 800-56A
+ // Rev. 3, Section 5.6.2.1.4 acknowledges that, and doesn't require it.
+ // However, ISO 19790:2012, Section 7.10.3.3 has a blanket requirement for a
+ // PCT for all generated keys (AS10.35) and FIPS 140-3 IG 10.3.A, Additional
+ // Comment 1 goes out of its way to say that "the PCT shall be performed
+ // consistent [...], even if the underlying standard does not require a
+ // PCT". So we do it. And make ECDH nearly 50% slower (only) in FIPS mode.
+ fips140.PCT("ECDH PCT", func() error {
+ p1, err := c.newPoint().ScalarBaseMult(privateKey.d)
+ if err != nil {
+ return err
+ }
+ if !bytes.Equal(p1.Bytes(), privateKey.pub.q) {
+ return errors.New("crypto/ecdh: public key does not match private key")
+ }
+ return nil
+ })
+
return privateKey, nil
}
}
@@ -188,28 +209,6 @@
panic("crypto/ecdh: internal error: public key is the identity element")
}
- // A "Pairwise Consistency Test" makes no sense if we just generated the
- // public key from an ephemeral private key. Moreover, there is no way to
- // check it aside from redoing the exact same computation again. SP 800-56A
- // Rev. 3, Section 5.6.2.1.4 acknowledges that, and doesn't require it.
- // However, ISO 19790:2012, Section 7.10.3.3 has a blanket requirement for a
- // PCT for all generated keys (AS10.35) and FIPS 140-3 IG 10.3.A, Additional
- // Comment 1 goes out of its way to say that "the PCT shall be performed
- // consistent [...], even if the underlying standard does not require a
- // PCT". So we do it. And make ECDH nearly 50% slower (only) in FIPS mode.
- if err := fips140.PCT("ECDH PCT", func() error {
- p1, err := c.newPoint().ScalarBaseMult(key)
- if err != nil {
- return err
- }
- if !bytes.Equal(p1.Bytes(), publicKey) {
- return errors.New("crypto/ecdh: public key does not match private key")
- }
- return nil
- }); err != nil {
- panic(err)
- }
-
k := &PrivateKey{d: bytes.Clone(key), pub: PublicKey{curve: c.curve, q: publicKey}}
return k, nil
}
diff -ru golang.org/fips140@v1.0.0/fips140/v1.0.0/ecdsa/cast.go golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/ecdsa/cast.go
--- golang.org/fips140@v1.0.0/fips140/v1.0.0/ecdsa/cast.go 1980-01-10 00:00:00.000000000 +0100
+++ golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/ecdsa/cast.go 1980-01-10 00:00:00.000000000 +0100
@@ -51,8 +51,8 @@
}
}
-func fipsPCT[P Point[P]](c *Curve[P], k *PrivateKey) error {
- return fips140.PCT("ECDSA PCT", func() error {
+func fipsPCT[P Point[P]](c *Curve[P], k *PrivateKey) {
+ fips140.PCT("ECDSA PCT", func() error {
hash := testHash()
drbg := newDRBG(sha512.New, k.d, bits2octets(P256(), hash), nil)
sig, err := sign(c, k, drbg, hash)
diff -ru golang.org/fips140@v1.0.0/fips140/v1.0.0/ecdsa/ecdsa.go golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/ecdsa/ecdsa.go
--- golang.org/fips140@v1.0.0/fips140/v1.0.0/ecdsa/ecdsa.go 1980-01-10 00:00:00.000000000 +0100
+++ golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/ecdsa/ecdsa.go 1980-01-10 00:00:00.000000000 +0100
@@ -166,11 +166,6 @@
return nil, err
}
priv := &PrivateKey{pub: *pub, d: d.Bytes(c.N)}
- if err := fipsPCT(c, priv); err != nil {
- // This can happen if the application went out of its way to make an
- // ecdsa.PrivateKey with a mismatching PublicKey.
- return nil, err
- }
return priv, nil
}
@@ -203,10 +198,7 @@
},
d: k.Bytes(c.N),
}
- if err := fipsPCT(c, priv); err != nil {
- // This clearly can't happen, but FIPS 140-3 mandates that we check it.
- panic(err)
- }
+ fipsPCT(c, priv)
return priv, nil
}
diff -ru golang.org/fips140@v1.0.0/fips140/v1.0.0/ecdsa/hmacdrbg.go golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/ecdsa/hmacdrbg.go
--- golang.org/fips140@v1.0.0/fips140/v1.0.0/ecdsa/hmacdrbg.go 1980-01-10 00:00:00.000000000 +0100
+++ golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/ecdsa/hmacdrbg.go 1980-01-10 00:00:00.000000000 +0100
@@ -121,7 +121,7 @@
//
// This should only be used for ACVP testing. hmacDRBG is not intended to be
// used directly.
-func TestingOnlyNewDRBG(hash func() fips140.Hash, entropy, nonce []byte, s []byte) *hmacDRBG {
+func TestingOnlyNewDRBG[H fips140.Hash](hash func() H, entropy, nonce []byte, s []byte) *hmacDRBG {
return newDRBG(hash, entropy, nonce, plainPersonalizationString(s))
}
diff -ru golang.org/fips140@v1.0.0/fips140/v1.0.0/ed25519/cast.go golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/ed25519/cast.go
--- golang.org/fips140@v1.0.0/fips140/v1.0.0/ed25519/cast.go 1980-01-10 00:00:00.000000000 +0100
+++ golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/ed25519/cast.go 1980-01-10 00:00:00.000000000 +0100
@@ -12,8 +12,8 @@
"sync"
)
-func fipsPCT(k *PrivateKey) error {
- return fips140.PCT("Ed25519 sign and verify PCT", func() error {
+func fipsPCT(k *PrivateKey) {
+ fips140.PCT("Ed25519 sign and verify PCT", func() error {
return pairwiseTest(k)
})
}
diff -ru golang.org/fips140@v1.0.0/fips140/v1.0.0/ed25519/ed25519.go golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/ed25519/ed25519.go
--- golang.org/fips140@v1.0.0/fips140/v1.0.0/ed25519/ed25519.go 1980-01-10 00:00:00.000000000 +0100
+++ golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/ed25519/ed25519.go 1980-01-10 00:00:00.000000000 +0100
@@ -69,10 +69,7 @@
fips140.RecordApproved()
drbg.Read(priv.seed[:])
precomputePrivateKey(priv)
- if err := fipsPCT(priv); err != nil {
- // This clearly can't happen, but FIPS 140-3 requires that we check.
- panic(err)
- }
+ fipsPCT(priv)
return priv, nil
}
@@ -88,10 +85,6 @@
}
copy(priv.seed[:], seed)
precomputePrivateKey(priv)
- if err := fipsPCT(priv); err != nil {
- // This clearly can't happen, but FIPS 140-3 requires that we check.
- panic(err)
- }
return priv, nil
}
@@ -137,12 +130,6 @@
copy(priv.prefix[:], h[32:])
- if err := fipsPCT(priv); err != nil {
- // This can happen if the application messed with the private key
- // encoding, and the public key doesn't match the seed anymore.
- return nil, err
- }
-
return priv, nil
}
diff -ru golang.org/fips140@v1.0.0/fips140/v1.0.0/fips140.go golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/fips140.go
--- golang.org/fips140@v1.0.0/fips140/v1.0.0/fips140.go 1980-01-10 00:00:00.000000000 +0100
+++ golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/fips140.go 1980-01-10 00:00:00.000000000 +0100
@@ -62,6 +62,10 @@
return "Go Cryptographic Module"
}
+// Version returns the formal version (such as "v1.0.0") if building against a
+// frozen module with GOFIPS140. Otherwise, it returns "latest".
func Version() string {
- return "v1.0"
+ // This return value is replaced by mkzip.go, it must not be changed or
+ // moved to a different file.
+ return "v1.0.0"
}
diff -ru golang.org/fips140@v1.0.0/fips140/v1.0.0/mlkem/mlkem1024.go golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/mlkem/mlkem1024.go
--- golang.org/fips140@v1.0.0/fips140/v1.0.0/mlkem/mlkem1024.go 1980-01-10 00:00:00.000000000 +0100
+++ golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/mlkem/mlkem1024.go 1980-01-10 00:00:00.000000000 +0100
@@ -118,10 +118,7 @@
var z [32]byte
drbg.Read(z[:])
kemKeyGen1024(dk, &d, &z)
- if err := fips140.PCT("ML-KEM PCT", func() error { return kemPCT1024(dk) }); err != nil {
- // This clearly can't happen, but FIPS 140-3 requires us to check.
- panic(err)
- }
+ fips140.PCT("ML-KEM PCT", func() error { return kemPCT1024(dk) })
fips140.RecordApproved()
return dk, nil
}
@@ -149,10 +146,6 @@
d := (*[32]byte)(seed[:32])
z := (*[32]byte)(seed[32:])
kemKeyGen1024(dk, d, z)
- if err := fips140.PCT("ML-KEM PCT", func() error { return kemPCT1024(dk) }); err != nil {
- // This clearly can't happen, but FIPS 140-3 requires us to check.
- panic(err)
- }
fips140.RecordApproved()
return dk, nil
}
diff -ru golang.org/fips140@v1.0.0/fips140/v1.0.0/mlkem/mlkem768.go golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/mlkem/mlkem768.go
--- golang.org/fips140@v1.0.0/fips140/v1.0.0/mlkem/mlkem768.go 1980-01-10 00:00:00.000000000 +0100
+++ golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/mlkem/mlkem768.go 1980-01-10 00:00:00.000000000 +0100
@@ -177,10 +177,7 @@
var z [32]byte
drbg.Read(z[:])
kemKeyGen(dk, &d, &z)
- if err := fips140.PCT("ML-KEM PCT", func() error { return kemPCT(dk) }); err != nil {
- // This clearly can't happen, but FIPS 140-3 requires us to check.
- panic(err)
- }
+ fips140.PCT("ML-KEM PCT", func() error { return kemPCT(dk) })
fips140.RecordApproved()
return dk, nil
}
@@ -208,10 +205,6 @@
d := (*[32]byte)(seed[:32])
z := (*[32]byte)(seed[32:])
kemKeyGen(dk, d, z)
- if err := fips140.PCT("ML-KEM PCT", func() error { return kemPCT(dk) }); err != nil {
- // This clearly can't happen, but FIPS 140-3 requires us to check.
- panic(err)
- }
fips140.RecordApproved()
return dk, nil
}
diff -ru golang.org/fips140@v1.0.0/fips140/v1.0.0/rsa/keygen.go golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/rsa/keygen.go
--- golang.org/fips140@v1.0.0/fips140/v1.0.0/rsa/keygen.go 1980-01-10 00:00:00.000000000 +0100
+++ golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/rsa/keygen.go 1980-01-10 00:00:00.000000000 +0100
@@ -105,7 +105,28 @@
// negligible chance of failure we can defer the check to the end of key
// generation and return an error if it fails. See [checkPrivateKey].
- return newPrivateKey(N, 65537, d, P, Q)
+ k, err := newPrivateKey(N, 65537, d, P, Q)
+ if err != nil {
+ return nil, err
+ }
+
+ if k.fipsApproved {
+ fips140.PCT("RSA sign and verify PCT", func() error {
+ hash := []byte{
+ 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08,
+ 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10,
+ 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18,
+ 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x20,
+ }
+ sig, err := signPKCS1v15(k, "SHA-256", hash)
+ if err != nil {
+ return err
+ }
+ return verifyPKCS1v15(k.PublicKey(), "SHA-256", hash, sig)
+ })
+ }
+
+ return k, nil
}
}
diff -ru golang.org/fips140@v1.0.0/fips140/v1.0.0/rsa/rsa.go golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/rsa/rsa.go
--- golang.org/fips140@v1.0.0/fips140/v1.0.0/rsa/rsa.go 1980-01-10 00:00:00.000000000 +0100
+++ golang.org/fips140@v1.0.0-c2097c7c/fips140/v1.0.0-c2097c7c/rsa/rsa.go 1980-01-10 00:00:00.000000000 +0100
@@ -310,26 +310,6 @@
return errors.New("crypto/rsa: d too small")
}
- // If the key is still in scope for FIPS mode, perform a Pairwise
- // Consistency Test.
- if priv.fipsApproved {
- if err := fips140.PCT("RSA sign and verify PCT", func() error {
- hash := []byte{
- 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08,
- 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10,
- 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18,
- 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x20,
- }
- sig, err := signPKCS1v15(priv, "SHA-256", hash)
- if err != nil {
- return err
- }
- return verifyPKCS1v15(priv.PublicKey(), "SHA-256", hash, sig)
- }); err != nil {
- return err
- }
- }
-
return nil
}
Change-Id: I6a6a6964b1780f19ec2b5202052de58b47d9342c
Reviewed-on: https://go-review.googlesource.com/c/go/+/701520
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Commit-Queue: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/706719
CMVP clarified with the September 2nd changes to IG 10.3.A that PCTs
don't need to run on imported keys.
However, PCT failure must enter the error state (which for us is fatal).
Thankfully, now that PCTs only run on key generation, we can be assured
they will never fail.
This change should only affect FIPS 140-3 mode.
While at it, make the CAST/PCT testing more robust, checking
TestConditional is terminated by a fatal error (and not by t.Fatal).
Updates #75524
Updates #74947
Updates #69536
Change-Id: I6a6a696439e1560c10f3cce2cb208fd40c5bc641
Reviewed-on: https://go-review.googlesource.com/c/go/+/706718
TryBot-Bypass: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
We are re-sealing the .zip file anyway for another reason, might as well
take the opportunity to fix the "v1.0" mistake.
Note that the actual returned version change will happen when re-sealing
the .zip, as the latest mkzip.go will inject "v1.0.0" instead of "v1.0".
This reapplies CL 701518, reverted in CL 702255.
Updates #75524
Change-Id: Ib5b3721bda35c32dd48293b3d1193c12661662dd
Reviewed-on: https://go-review.googlesource.com/c/go/+/706717
Reviewed-by: Roland Shoemaker <roland@golang.org>
TryBot-Bypass: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Delve and viewcore use DWARF type DIEs to display and explore the
runtime value of interface variables.
This has always been slightly problematic since the runtime type of an
interface variable might only be reachable through interfaces and thus
be missing from debug_info (see issue #46670).
Prior to commit f4de2ecf this was not a severe problem since a struct
literal caused the allocation of a struct into an autotemp variable,
which was then used by dwarfgen to make sure that the DIE for that type
would be generated.
After f4de2ecf such autotemps are no longer being generated and
go1.25.0 ends up having many more instances of interfaces with
unreadable runtime type (https://github.com/go-delve/delve/issues/4080).
This commit fixes this problem by scanning the relocation of the
function symbol and adding to the function's DIE symbol references to
all types used by the function to create interfaces.
Fixes go-delve/delve#4080
Updates #46670Fixes#75255
Change-Id: I3e9db1c0d1662905373239816a72604ac533b09e
Reviewed-on: https://go-review.googlesource.com/c/go/+/696955
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Than McIntosh <thanm@golang.org>
Reviewed-by: Florian Lehner <lehner.florian86@gmail.com>
(cherry picked from commit 80038586ed)
Reviewed-on: https://go-review.googlesource.com/c/go/+/704335
Reviewed-by: Alessandro Arzilli <alessandro.arzilli@gmail.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Windows doesn't keep the file pointer for overlapped file handles. To
work around this, we keep track of the current offset ourselves and use
it on every Read/Write operation.
When the user calls File.Seek with whence == io.SeekCurrent, it expects
that the offset we keep track of is also accounted for, else the the
seek'ed value won't match the file pointer seen by the user.
Fixes#75083.
Change-Id: Ieca7c3779e5349292883ffc293a8474088a4dec7
Reviewed-on: https://go-review.googlesource.com/c/go/+/697275
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
(cherry picked from CL 697275)
Reviewed-on: https://go-review.googlesource.com/c/go/+/697995
Unblocking a bubbled goroutine from outside the bubble is an error
and panics. Currently, some of those panics are regular panics
and some are fatal. We use fatal panics in cases where its difficult
to panic without leaving something in an inconsistent state.
Change the regular panics (channel and timer operations) to be fatal.
This makes our behavior more consistent: All bubble violations are
always fatal.
More importantly, it avoids introducing new, recoverable panics.
A motivating example for this change is the context package,
which performs channel operations with a mutex held in the
expectation that those operations can never panic. These operations
can now panic as a result of a bubble violation, potentially
leaving a context.Context in an inconsistent state.
For #74837Fixes#75021
Change-Id: Ie6efd916b7f505c0f13dde42de1572992401f15c
Reviewed-on: https://go-review.googlesource.com/c/go/+/696195
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
(cherry picked from commit a8564bd412)
Reviewed-on: https://go-review.googlesource.com/c/go/+/696196
A database/sql/driver.Rows can return database-owned data
from Rows.Next. The driver.Rows documentation doesn't explicitly
document the lifetime guarantees for this data, but a reasonable
expectation is that the caller of Next should only access it
until the next call to Rows.Close or Rows.Next.
Avoid violating that constraint when a query is cancelled while
a call to database/sql.Rows.Scan (note the difference between
the two different Rows types!) is in progress. We previously
took care to avoid closing a driver.Rows while the user has
access to driver-owned memory via a RawData, but we could still
close a driver.Rows while a Scan call was in the process of
reading previously-returned driver-owned data.
Update the fake DB used in database/sql tests to invalidate
returned data to help catch other places we might be
incorrectly retaining it.
Updates #74831Fixes#74834
Change-Id: Ice45b5fad51b679c38e3e1d21ef39156b56d6037
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2540
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Neal Patel <nealpatel@google.com>
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2600
Reviewed-on: https://go-review.googlesource.com/c/go/+/693559
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Mark Freeman <markfreeman@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Fix incorrect expansion of "" and "." when $PATH contains an executable
file or, on Windows, a parent directory of a %PATH% element contains an
file with the same name as the %PATH% element but with one of the
%PATHEXT% extension (ex: C:\utils\bin is in PATH, and C:\utils\bin.exe
exists).
Fix incorrect expansion of ".." when $PATH contains an element which is
an the concatenation of the path to an executable file (or on Windows
a path that can be expanded to an executable by appending a %PATHEXT%
extension), a path separator and a name.
"", "." and ".." are now rejected early with ErrNotFound.
Fixes CVE-2025-47906
Fixes#74466
Change-Id: Ie50cc0a660fce8fbdc952a7f2e05c36062dcb50e
Reviewed-on: https://go-review.googlesource.com/c/go/+/685755
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
(cherry picked from commit e0b07dc22e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/691775
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Merge List:
+ 2025-07-23 489868f776 cmd/link: scope test to linux & net.sendFile
+ 2025-07-22 71c2bf5513 cmd/compile: fix loclist for heap return vars without optimizations
+ 2025-07-22 c74399e7f5 net: correct comment for ListenConfig.ListenPacket
+ 2025-07-22 4ed9943b26 all: go fmt
+ 2025-07-22 1aaf7422f1 cmd/internal/objabi: remove redundant word in comment
+ 2025-07-21 d5ec0815e6 runtime: relax TestMemoryLimitNoGCPercent a bit
+ 2025-07-21 f7cc61e7d7 cmd/compile: for arm64 epilog, do SP increment with a single instruction
+ 2025-07-21 5dac42363b runtime: fix asan wrapper for riscv64
+ 2025-07-21 e5502e0959 cmd/go: check subcommand properties
+ 2025-07-19 2363897932 cmd/internal/obj: enable got pcrel itype in fips140 for riscv64
+ 2025-07-19 e32255fcc0 cmd/compile/internal/ssa: restrict architectures for TestDebugLines_74576
+ 2025-07-18 0451816430 os: revert the use of AddCleanup to close files and roots
+ 2025-07-18 34b70684ba go/types: infer correct type for y in append(bytes, y...)
+ 2025-07-17 66536242fc cmd/compile/internal/escape: improve DWARF .debug_line numbering for literal rewriting optimizations
+ 2025-07-16 385000b004 runtime: fix idle time double-counting bug
+ 2025-07-16 f506ad2644 cmd/compile/internal/escape: speed up analyzing some functions with many closures
+ 2025-07-16 9c507e7942 cmd/link, runtime: on Wasm, put only function index in method table and func table
+ 2025-07-16 9782dcfd16 runtime: use 32-bit function index on Wasm
+ 2025-07-16 c876bf9346 cmd/internal/obj/wasm: use 64-bit instructions for indirect calls
+ 2025-07-15 b4309ece66 cmd/internal/doc: upgrade godoc pkgsite to 01b046e
+ 2025-07-15 75a19dbcd7 runtime: use memclrNoHeapPointers to clear inline mark bits
+ 2025-07-15 6d4a91c7a5 runtime: only clear inline mark bits on span alloc if necessary
+ 2025-07-15 0c6296ab12 runtime: have mergeInlineMarkBits also clear the inline mark bits
+ 2025-07-15 397d2117ec runtime: merge inline mark bits with gcmarkBits 8 bytes at a time
+ 2025-07-15 7dceabd3be runtime/maps: fix typo in group.go comment (instrinsified -> intrinsified)
+ 2025-07-15 d826bf4d74 os: remove useless error check
+ 2025-07-14 bb07e55aff runtime: expand GOMAXPROCS documentation
+ 2025-07-14 9159cd4ec6 encoding/json: decompose legacy options
+ 2025-07-14 c6556b8eb3 encoding/json/v2: add security section to doc
+ 2025-07-11 6ebb5f56d9 runtime: gofmt after CL 643897 and CL 662455
+ 2025-07-11 1e48ca7020 encoding/json: remove legacy option to EscapeInvalidUTF8
+ 2025-07-11 a0a99cb22b encoding/json/v2: report wrapped io.ErrUnexpectedEOF
+ 2025-07-11 9d04122d24 crypto/rsa: drop contradictory promise to keep PublicKey modulus secret
+ 2025-07-11 1ca23682dd crypto/rsa: fix documentation formatting
+ 2025-07-11 4bc3373c8e runtime: turn off large memmove tests under asan/msan
+ 2025-07-11 88cf0c5d55 cmd/link: do size fixups after symbol references are loaded
+ 2025-07-10 7a38975a48 os: trivial comment fix
+ 2025-07-10 aa5de9ebb5 synctest: fix comments for time.Now() in synctests
+ 2025-07-10 63ec70d4e1 crypto/cipher: Fix comment punctuation
+ 2025-07-09 8131635e5a runtime: run TestSignalDuringExec in its own process group
+ 2025-07-09 67c1704444 crypto/tls: empty server_name conf. ext. from server
+ 2025-07-08 54c9d77630 cmd/go: disable support for multiple vcs in one module
+ 2025-07-08 fca43a8436 internal: make struct comment match struct name
+ 2025-07-08 bb917bb030 cmd/compile: document that nosplit directive is unsafe
+ 2025-07-08 a5bda585d5 cmd/compile: run fmt on ssa
+ 2025-07-07 86b5ba7310 internal/trace: only test for sync preemption if async preemption is off
+ 2025-07-07 ef46e1b164 cmd/internal/doc: fix GOROOT skew and path joining bugs
+ 2025-07-07 75b43f9a97 runtime: make traceStack testable and add a benchmark
+ 2025-07-07 20978f46fd crypto/rsa: remove another forgotten note to future self
+ 2025-07-07 33fb4819f5 cmd/compile/internal/ssa: skip EndSequence entries in TestStmtLines
+ 2025-07-07 a995269a93 sort: clarify Less doc
+ 2025-07-03 6c3b5a2798 runtime: correct vdsoSP on S390X
+ 2025-07-03 dd687c3860 hash: document that Clone may only return ErrUnsupported or a nil error
+ 2025-07-02 b325151453 cmd/cgo/internal/testsanitizers: skip asan tests when FIPS140 mode is on
+ 2025-07-02 15d9fe43d6 testing/synctest: explicitly state Run will be removed in Go 1.26
+ 2025-07-01 de646d94f7 cmd/go/internal/modindex: apply changes in CL 502615 to modindex package
Change-Id: I0420eec24c176a76a0ae51ddf6e34ee3fe4ae8ba
CL 687815 recently added TestDebugLines_74576.
The pre-existing debug_lines_test.go file generally restricts the
tested architectures and contains multiple warnings that the
testing approach is useful but fragile, such as:
"These files must all be short because this is super-fragile."
Despite that, initially I wanted to see what happened on the
different architectures on the trybots in case it might show something
surprising, and I let TestDebugLines_74576 run on all architectures.
That seemed to initially work, but the test is now failing on a
linux/risc64 builder (#74669), so it is likely more prudent to be
more conservative and restrict the platforms like many of the
other pre-existing tests, which is what this CL now does.
Fixes#74669
Change-Id: I9e5a7d3ee901f58253cf72e03c2239df338479e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/688856
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
The type-checking logic for append has a special case for
append(bytes, s...) where the typeset for s contains string.
However, this case was triggering even when the typeset contained
only []byte, causing the creation of Signature types of the form
func([]byte, Y) []byte, with the variadic flag set, where Y
is the type of Y (e.g. a type parameter constrained to ~[]byte).
This is an illegal combination: a variadic signature's last
parameter must be a slice, or its typeset must contain string.
This caused x/tools/go/ssa to crash.
This CL narrows the special case to only typesets that contain
string, and adds a test for the inferred signature.
(There's little point in testing that a subsequent NewSignatureType
call would succeed, because the inferred type plainly has
no free type parameters.)
Fixes#73871
Change-Id: Id7641104133371dd6b0077f281cdaa9db84cc1c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/688815
Reviewed-by: Mark Freeman <mark@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The literal rewriting optimizations to reduce user allocations
that were implemented in CL 649079 and related CLs like CL 684116
could produce .debug_line tables such that the line numbers
sometimes jumped back to a variable's declaration.
This CL adjusts the positions of the IR nodes to avoid this.
For the first test added here in i74576a.go:
11 func main() {
12 a := 1
13 runtime.Breakpoint()
14 sink = a
15 }
Without this fix, the test reports debug lines of 12, 13, 13, 12, 14.
Note it goes backwards from 13 to a second 12.
With this fix, the test reports debug lines of 12, 13, 13, 14
without going backwards.
The test added in i74576b.go creates a slice via make with a
non-constant argument, which similarly shows debug lines going backwards
before this fix but not after. To address the slice make case, we
create a new BasicLit node to then set its position.
There were some related allocation optimizations for struct literals
such as CL 649555 during the Go 1.25 dev cycle, but at least in some
basic test cases, those optimizations did not seem to produce
debug line issues. The struct literal interface conversion test
added in i74576c.go has the same behavior before and after this change.
Finally, running 'go test ./...' in the delve repo root (4a2a6e1aeb)
seems to have many failures with go1.25rc2, but seems to pass with
this CL.
Fixes#74576
Updates #71359
Change-Id: I31faf5fe4bb352fdcd06bdc8606dbdbc4bbd65f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/687815
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Alessandro Arzilli <alessandro.arzilli@gmail.com>
Reviewed-by: Keith Randall <khr@google.com>
Escape analysis examines functions in batches. In some cases, closures
are in the same batch as their parent function, but in other cases,
the closures are in different batches. This can mean the per-batch
ir.ReassignOracle cache is not as effective.
For example, #74615 has 4,000 closures in a single function that
are all in different batches. For that example, these caches
had an ~80% hit rate.
This CL makes the ir.ReassignOracle cache more broadly scoped, instead
of per batch.
This speeds up escape analysis when a function has many closures
that end up in different batches, including this resolves#74615.
For that example, this cache now has a ~100% hit rate.
In addition, in (*batch).rewriteWithLiterals, we also slightly delay
checking the ir.ReassignOracle cache, which is more natural to do now
compared to when rewriteWithLiterals was first merged. This means we can
avoid consulting or populating the cache in more cases. (We also leave
a new type-related TODO there. If we were to also implement that TODO, a
quick test suggests we could independently resolve the specific example
in #74615 even without making the cache more broadly scoped,
though other conceivable examples would not be helped; the scoping of
the cache is the more fundamental improvement.)
If we look at cumulative time spent via pprof for the #74615 example
using this CL, the work of ir.ReassignOracle escape analysis
cache now typically shows zero cpu samples.
This CL passes "go build -toolexec 'toolstash -cmp' -a std cmd".
Fixes#74615
Change-Id: I3c17c527fbb546ffb8a4fa52cd61e41ff3cdb869
Reviewed-on: https://go-review.googlesource.com/c/go/+/688075
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
In the type descriptor's method table, it contains relative PCs of
the methods (relative to the start of the text section) stored as
32-bit offsets. On Wasm, a PC is PC_F<<16 + PC_B, where PC_F is
the function index, and PC_B is the block index. When there are
more than 65536 functions, the PC will not fit into 32-bit (and
relative to the section start doesn't help). Since there are no
more bits for the function index, and the method table always
targets the entry of a method, we put just the PC_F there, and
rewrite back to a full PC at run time when we need the PC. This
way we can have more than 65536 functions.
The func table also contains 32-bit relative PCs, and it also
always points to function entries. Do the same there, as well
as other places where we use relative text offsets.
Also add the relocation type in the relocation overflow error
message.
Also add check for function too big on Wasm. If a function has
more than 65536 blocks, PC_B will overflow and PC = PC_F<<16 + PC_B
will points to the wrong function.
Fixes#64856.
Change-Id: If9c307e9fb1641f367a5f19c39f88f455805d0bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/552835
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently, on Wasm, an indirect call is compiled to
// function index = PC>>16, PC is already on stack
I32WrapI64
I32Const $16
ShrU
// set PC_B to 0
...
// actual call
CallIndirect
Specifically, the function index is extracted from bits 16-31 of
the "PC". When there are more than 65536 functions, this will
overflow and wrap around, causing wrong function being called.
This CL changes it to use 64-bit operations to extract the
function index from the "PC", so there are enough bits to for it.
For #64856.
Change-Id: I83c11db4b78cf66250e88ac02a82bd13730a8914
Reviewed-on: https://go-review.googlesource.com/c/go/+/567896
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Clearing the inline mark bits with memclrNoHeapPointers is slightly
better than having the compiler insert, e.g. duffzero, since it can take
advantage of wider SIMD instructions. duffzero is likely going away, but
we know things the compiler doesn't, such as the fact that this memory
is nicely aligned. In this particular case, memclrNoHeapPointers does a
better job.
For #73581.
Change-Id: I3918096929acfe6efe6f469fb089ebe04b4acff5
Reviewed-on: https://go-review.googlesource.com/c/go/+/687938
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
This is conceptually simpler, as the sweeper doesn't have to worry about
clearing them separately. It also doesn't have a use for them.
This will also be useful to avoiding unnecessary zeroing in
initInlineMarkBits at allocation time. Currently, because it's used in
both span allocation and at sweep time, we cannot blindly trust
needzero.
This change also renames mergeInlineMarkBits to moveInlineMarkBits to
make this change in semantics clearer from the name.
For #73581.
Change-Id: Ib154738a945633b7ff5b2ae27235baa310400139
Reviewed-on: https://go-review.googlesource.com/c/go/+/687936
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Currently, with Green Tea GC, we need to copy (really bitwise-or) mark
bits back into mspan.gcmarkBits, so that it can propagate to
mspan.allocBits at sweep time. This function does actually seem to make
sweeping small spans a good bit more expensive, though sweeping is still
relatively cheap. There's some low-hanging fruit here though, in that
the merge is performed one byte at a time, but this is pretty
inefficient. We can almost as easily perform this merge one word at a
time instead, which seems to make this operation about 33% faster.
For #73581.
Change-Id: I170d36e7a2193199c423dcd556cba048ebd698af
Reviewed-on: https://go-review.googlesource.com/c/go/+/687935
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Expand the GOMAXPROCS documentation to include details of how defaults
are selected, as this is something that inquisitive minds will want to
know. I've added an additional warning that these details may changed.
While we are here, add a bit more structure to make it easier to find
the relevant parts of the documentation.
For #73193.
Change-Id: I6a6a636cae93237e3e3174822490d51805e70990
Reviewed-on: https://go-review.googlesource.com/c/go/+/685318
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Sean Liao <sean@liao.dev>
Reviewed-by: Sean Liao <sean@liao.dev>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
WARNING: This commit contains breaking changes
for those already using GOEXPERIMENT=jsonv2.
This decomposes FormatBytesWithLegacySemantics as:
* FormatBytesWithLegacySemantics
* FormatByteArrayAsArray
* ParseBytesWithLooseRFC4648
This decomposes FormatTimeWithLegacySemantics as:
* FormatDurationAsNano
* ParseTimeWithLooseRFC3339
In particular, it splits out specific behaviors from the option
that may need to be specified on a finer-grain level.
FormatByteArrayAsArray and FormatDurationAsNano are targeted
to just the default representation of a [N]byte or time.Duration type.
Both of these are not necessary if the `format` tag is explicitly specified.
However, we want to isolate their behavior from other behaviors that used to
be part of FormatBytesWithLegacySemantics and FormatTimeWithLegacySemantics.
ParseBytesWithLooseRFC4648 and ParseTimeWithLooseRFC3339 are targeted
to just historically buggy parsing according to the relevant RFCs,
which may need to be enabled by some services for backwards compatibility.
While FormatTimeWithLegacySemantics is deleted, we still need
FormatBytesWithLegacySemantics to configure highly esoteric
aspects of how v1 used to handle byte slices.
We rename OmitEmptyWithLegacyDefinition as OmitEmptyWithLegacySemantics
to be consistent with other options with the WithLegacySemantics suffix.
Updates #71497
Change-Id: Ic660515fb086fe3af237135f195736de99c2bd33
Reviewed-on: https://go-review.googlesource.com/c/go/+/685395
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
In the presence of invalid UTF-8, the AllowInvalidUTF8 option
allows such bytes to be present, but silently mangles them
using the Unicode replacement character.
The v2 default is to emit the replacement character verbatim
(which is valid UTF-8 and exactly what it is for).
However, the v1 behavior has historically been to emit
the escaped form of the replacement character.
This behavior was introduced in https://go.dev/cl/11211045
where the documentation says that it is:
replacing invalid bytes with the Unicode replacement rune U+FFFD
but the implementation actually replaces it with
the escaped form of the Unicode replacement rune.
Given that the documentation differs from the implementation,
the actual behavior is likely an oversight.
Given how esoteric of behavior this is,
we change the v1in2 behavior to avoid the unnecesary escaping
and drop support for EscapeInvalidUTF8.
This does not violate the Go compatibility agreement since
we do not document what the exact syntactic output is.
Also, there has already been prior precedence for changing the output:
* [encoding/json: encode \b and \f as '\b' and '\f' in JSON strings](https://go.dev/cl/521675)
* [encoding/json: encode \n in strings as "\n", not "\u000A"](https://go.dev/cl/4678046)
* [encoding/json: encode \t as \t instead of \u0009](https://go.dev/cl/162340043)
* [encoding/json: use standard ES6 formatting for numbers during marshal](https://go.dev/cl/30371)
Fixes#74551
Change-Id: Ib59a873c44713d302f1f6ab103ffba2520d63276
Reviewed-on: https://go-review.googlesource.com/c/go/+/687116
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
In the event that the input is just JSON whitespace,
the underlying jsontext.Decoder treats this as an empty stream
and reports io.EOF.
The logic in unmarshalFull simply casted io.EOF as io.ErrUnexpectedEOF,
which is inconsistent with how all other io.ErrUnexpectedEOF are reported,
which are wrapped within a jsontext.SyntacticError.
Do the same thing for consistency.
We add a v1 test (without goexperiment.jsonv2) to verify that
the behavior is identical to how v1 has always behaved.
We add a v1in2 test (with goexperiment.jsonv2) to verify that
the v1in2 behavior correctly replicates historical v1 behavior.
We also fix a faulty check in v1 Decoder.Decode,
where it tried to detect errUnexpectedEnd and
return an unwrapped io.ErrUnexpectedEOF error.
This is the exact semantic that v1 has always done
in streaming Decoder.Decode (but not non-streaming Unmarshal).
There is a prior bug reported in #25956 about this inconsistency,
but we aim to preserve historical v1 behavior to reduce
the probability of churn when v1 is re-implemented in terms of v2.
Fixes#74548
Change-Id: Ibca52c3699ff3c09141e081c85f853781a86ec8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/687115
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
We claim to treat N as secret (and indeed bigmod is constant time in
relation to the modulus) but at the same time we warn that all inputs to
VerifyPKCS1v15 and Verify are public:
> The inputs are not considered confidential, and may leak through
> timing side channels, or if an attacker has control of part of the
> inputs.
See #67043 (which focuses on the inverse, recovering signatures by
controlling the public key input to Verify), and in particular
https://github.com/golang/go/issues/67043#issuecomment-2079335804.
Stopping the Verify adaptive attack would require significantly more
complexity, the kind that has caused vulnerabilities in the past (e.g.
CVE-2016-2107). On the other hand, assuming that a public key is
confidential is unlikely to work in practice, since it can be recovered
from just two valid (message, signature) pairs. See for example
https://keymaterial.net/2024/06/15/reconstructing-public-keys-from-signatures/.
This comment was introduced in CL 552935, not really due to a need to
specify that N was secret, but rather to clarify that E is not (so it
could be used in variable-time exponentiation).
Change-Id: I6a6a6964f3f8d2dc2fcc13ce938b271c9de9666b
Reviewed-on: https://go-review.googlesource.com/c/go/+/687616
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
When we do a size fixup, we need to clone the symbol to an
external symbol so we can modify it. This includes cloning the
relocations, which includes resolving the relocations. If the
symbol being fixed has a relocation referencing a non-Go symbol,
that symbol has not yet been created, it will be resolved to an
empty symbol. Load the references first, so the referenced symbol,
even if it is a non-Go symbol, exists.
Fixes#74537.
Change-Id: I81525bd7c3e232b80eefeb0f18e13ba5331e1510
Reviewed-on: https://go-review.googlesource.com/c/go/+/687315
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
TestSignalDuringExec sends a SIGWINCH to the whole process group.
However, it may execute concurrently with other copies of the runtime
tests, especially through `go tool dist`, and gdb version <12.1 has a
bug in non-interactive mode where recieving a SIGWINCH causes a crash.
This change modifies SignalDuringExec in the testprog to first fork
itself into a new process group. To avoid issues with Ctrl+C and the new
process group hanging, the new process blocks on a pipe that is passed
down to it. This pipe is automatically closed when its parent exits,
which should ensure that the subprocess also exits.
Fixes#58932.
Change-Id: I3906afa28cf8b15d22ae612d071bce7f30fc3e6c
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest-noswissmap,gotip-linux-amd64-longtest-aliastypeparams,gotip-linux-amd64-longtest,gotip-linux-386-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/686875
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
When a TLS server uses the information from the server_name extension in
a client hello, and the connection isn't resuming, it should return an
empty server_name extension in its server hello (or encrypted extensions
for TLS 1.3).
For TLS <1.3 we we do this in doFullHandshake(), by setting the
pre-existing serverHelloMsg.serverNameAck bool. We know that the
connection isn't resuming based on the context where this function is
called.
For TLS 1.3, a new encryptedExtensionsMsg.serverNameAck bool is added,
and populated as appropriate in sendServerParameters() based on whether
the conn was resumed or not. The encryptedExtensionsMsg marshalling is
updated to emit the encrypted extension based on that field.
These changes allow enabling the ServerNameExtensionServer-* bogo tests
that verify both the presence and absence of the server_name extension
based on the relevant specifications.
Resolves#74282
Updates #72006
Change-Id: I703bc2ec916b50906bdece7b7483a7faed7aa8e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/684795
TryBot-Bypass: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Daniel McCarney <daniel@binaryparadox.net>
Removes the somewhat redundant vcs.FromDir, "allowNesting" argument,
which was always enabled, and disallow multiple VCS metadata folders
being present in a single directory. This makes VCS injection attacks
much more difficult.
Also adds a GODEBUG, allowmultiplevcs, which re-enables this behavior.
Thanks to RyotaK (https://ryotak.net) of GMO Flatt Security Inc for reporting this issue.
Fixes#74380
Fixes CVE-2025-4674
Change-Id: I5787d90cdca8deb3aca6f154efb627df1e7d2789
Reviewed-on: https://go-review.googlesource.com/c/go/+/686515
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Commit-Queue: Carlos Amedee <carlos@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Removes the somewhat redundant vcs.FromDir, "allowNesting" argument,
which was always enabled, and disallow multiple VCS metadata folders
being present in a single directory. This makes VCS injection attacks
much more difficult.
Also adds a GODEBUG, allowmultiplevcs, which re-enables this behavior.
Thanks to RyotaK (https://ryotak.net) of GMO Flatt Security Inc for
reporting this issue.
Fixes#74380
Fixes CVE-2025-4674
Change-Id: I95b619588ecb6661770aa4e1d6023d6cb22e2263
Reviewed-on: https://go-review.googlesource.com/c/go/+/686338
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Carlos Amedee <carlos@golang.org>
TryBot-Bypass: Carlos Amedee <carlos@golang.org>
Currently, the test change made for the fix to #68090 is flaky. This is
because the sync-point-only goroutine that we expect to be sync
preempted might only ever get async preempted in some circumstances.
This change adds a variant to all trace tests to run with
asyncpreemptoff=1, and the stacks test, the flaky one, only actually
checks for the sync-point in the trace when async preemption is
disabled.
Fixes#74417.
Change-Id: Ib6341bbc26921574b8f0fff6dd521ce83f85499c
Reviewed-on: https://go-review.googlesource.com/c/go/+/686055
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Use the goCmd() function to get the go command to invoke, so that when
GOROOT is set, the go command that's invoked uses the same GOROOT.
Otherwise there will be skew between the go command and the tools and
runtime. Also use the environment when determining GOPROXY and
GOMODCACHE, and use url.Join so the slashes in 'http://' aren't
collapsed into one.
Change-Id: Ie36ca2fffdb015a7f5f9bd7f514850e41fad2c1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/685319
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
In CL 226937 I refactored the RSA-PSS implementation, and apparently
left behind a note to think a bit harder about whether this bytes.Equal
check should be constant time or not. It snuck through code review, so
it's 2018 again, no one is worried about pandemics, I have just joined
Google, and I am mailing CL 147637 again.
Anyway, as discussed in #67043 and documented in CL 587277, the inputs
to signature verification functions are not secret, and are allowed to
leak through timing side channels. This means an attacker can already
compute h (from signature and public key) and h0 (from message hash and
public key). What the attacker can't do is produce a signature that
yields the correct h (since that requires the private key).
Change-Id: I6a6a4656d6255bdad628a94f48f7ea878a304263
Reviewed-on: https://go-review.googlesource.com/c/go/+/685255
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
The TestStmtLines test has been accessing a nil pointer when it
tries to look up LineEntry.File.Name on a line entry with
EndSequence set to true. The doc for EndSequence specifies that if
EndSequence is set, only it and the Address field are meaningful. Skip
the entries with EndSequence set when building the set of files.
I've reproduced this issue locally.
Probably also fixes#49372, but will leave that for a follow-up CL.
Fixes#74475
Updates #49372
Change-Id: Ic0664f7652b52a0a20239d13fe16454622740821
Reviewed-on: https://go-review.googlesource.com/c/go/+/685835
Reviewed-by: Than McIntosh <thanm@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Quim Muntal <quimmuntal@gmail.com>
CL 502615 modified go/build to check for invalid import paths, but did
not make those changes to the corresponding code in the modindex
package. Apply those changes here.
We should try to deduplicate the code to prevent this from happening
again.
For #73976
For #74446
Change-Id: I69fc5e2c829efb818c9974ec8126807a1c8f7913
Reviewed-on: https://go-review.googlesource.com/c/go/+/685317
TryBot-Bypass: Michael Matloob <matloob@google.com>
Auto-Submit: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>
The jsonopts.Struct.join method unfortunately escapes
the receiver because it is passed to JoinUnknownOption,
which is a dynamically implemented function.
This affects jsontext.Encoder.reset and jsontext.Decoder.reset,
which relied on a local jsonopts.Struct to temporarily store
prior options such that it would have to be heap allocated.
Adjust the signature of JoinUnknownOption to avoid pointers
so that nothing escape.
This is a regression from
https://github.com/go-json-experiment/json/pull/163
Performance:
name old time/op new time/op delta
Marshal/Bool-32 72.1ns ± 2% 51.3ns ± 1% -28.77% (p=0.000 n=10+9)
name old allocs/op new allocs/op delta
Marshal/Bool-32 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=10+10)
Updates #71845
Change-Id: Ife500d82d3d2beb13652553a4ffdf882c136f5a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/685135
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Several CLs earlier in this stack added optimizations to reduce
user allocations by recognizing and taking advantage of literals,
including CL 649555, CL 649079, and CL 649035.
This CL adds debug hashing of those changes, which enables use of the
bisect tool, such as 'bisect -compile=literalalloc go test -run=Foo'.
This also allows these optimizations to be manually disabled via
'-gcflags=all=-d=literalallochash=n'.
Updates #71359
Change-Id: I854f7742a6efa5b17d914932d61a32b2297f0c88
Reviewed-on: https://go-review.googlesource.com/c/go/+/675415
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
CL 649079 and CL 649035 updated escape analysis to rewrite certain
expressions in OMAKE and OCONVIFACE nodes as optimizations to
reduce user allocations.
Part of the change in CL 649079 disabled those optimzations when
coverage instrumentation was enabled under an incorrect possible theory
of how those optimizations might be "expected" to change coverage
results -- in particular, the cover_build_pkg_select.txt testscript
failed with different coverage results. I now realize that the proper
explanation is that my fix in CL 684116 was needed.
Now that CL 684116 is merged, we should no longer disable these
optimizations when coverage is enabled, which is what this CL does.
This has not been reported as a problem to my knowledge, but without
this CL, one could imagine for example a test using testing.AllocsPerRun
might start failing when coverage was enabled if the result relied on
these optimizations.
As expected, if we place this CL just before the necessary fix in
CL 684116, the cover_build_pkg_select.txt testscript fails with a
changed coverage result. If we place this CL just after CL 684116,
the test passes, also as expected.
Updates #71359
Change-Id: Ib5ff00c267acd85dd423c238d177e91a4d881f9e
Reviewed-on: https://go-review.googlesource.com/c/go/+/684777
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
This commit updates the pinned revision of BoringSSL that's used for the
BoGo integration test.
Doing this requires a few categories of config changes:
* ignoring a few new tests for features crypto/tls doesn't implement
* ignoring a few new tests that require further
investigation/classification, or that already have an associated
tracking issue
* updating the error map syntax to accommodate the upstream change that
allows a one-to-many mapping
One code change is required in the shim test process to adjust how we
tear down a connection after an error to account for an upstream change
in the test runner.
Previously, for error conditions we would immediately close the
connection when exiting the shim process. We instead need to do this in
a multi-step process:
1. Flush any pending TLS writes to surface any alerts the error
condition may have generated.
2. Close the write side of the TCP connection to signal we're not
writing anymore.
3. Read and discard any pending data from the peer.
4. Close the read side of the TCP connection to fully close the socket.
Without doing this unpredictable timing factors may result in spurious
test failures where:
1. The runner sends us data that produces an error.
2. We send an alert, and immediately tear down the connection.
3. The runner tries to perform a write, and hits an error because the
pipe is closed.
4. The runner fails the test with the pipe write error, before it reads
from the connection to see the expected alert.
With the new code we instead swallow the unrelated writes and the runner
sees our alert after its ignored write when it tries to read from the
conn. The alert is the expected test outcome, and so the test passes.
This was previously not an issue because the runner was discarding the
write errors.
Updates #72006
Change-Id: Ib72a1c5e693aac92144696c8bae888d5f3f6c32f
Reviewed-on: https://go-review.googlesource.com/c/go/+/683456
Auto-Submit: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
findRunnable takes a snapshot of allp prior to dropping the P because
afterwards procresize may mutate allp without synchronization.
procresize is careful to never mutate the contents up to cap(allp), so
findRunnable can still safely access the Ps in the slice.
Unfortunately, growing allp is problematic. If procresize grows the allp
backing array, it drops the reference to the old array. allpSnapshot
still refers to the old array, but allpSnapshot is on the system stack
in findRunnable, which also likely no longer has a P at all.
This means that a future GC will not find the reference and can free the
array and use it for another allocation. This would corrupt later reads
that findRunnable does from the array.
The fix is simple: the M struct itself is reachable by the GC, so we can
stash the snapshot in the M to ensure it is visible to the GC.
The ugliest part of the CL is the cleanup when we are done with the
snapshot because there are so many return/goto top sites. I am tempted
to put mp.clearAllpSnapshot() in the caller and at top to make this less
error prone, at the expensive of extra unnecessary writes.
Fixes#74414.
Change-Id: I6a6a636c484e4f4b34794fd07910b3fffeca830b
Reviewed-on: https://go-review.googlesource.com/c/go/+/684460
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Fix a race condition in disassociating a WaitGroup in a synctest
bubble from its bubble. We previously disassociated the WaitGroup
when count becomes 0, but this causes problems when an Add call
setting count to 0 races with one incrementing the count.
Instead, disassociate a WaitGroup from its bubble when Wait returns.
Wait must not be called concurrently with an Add call with a
positive delta and a 0 count, so we know that the disassociation
will not race with an Add call trying to create a new association.
Fixes#74386
Change-Id: I9b519519921f7691869a64a245a5ee65d071d054
Reviewed-on: https://go-review.googlesource.com/c/go/+/684635
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
On Windows, GOMODCACHE almost never starts with a slash, and
"go doc -http" constructs a GOPROXY URL by doing "file://" + GOMODCACHE,
resulting in an invalid file URI.
For example, if GOMODCACHE is "C:\foo", then the file URI should be
"file:///C:/foo", but it becomes "file://C:/foo" instead, where "C:" is
understood as a host name, not a drive letter.
Fixes#74137.
Change-Id: I23e776e0f649a0062e01d1a4a6ea8268ba467331
Reviewed-on: https://go-review.googlesource.com/c/go/+/684575
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>
The NeedmDeadlock test program currently has a 5-second timeout,
which is sort of arbitrary. It is long enough in regular mode
(which usually takes 0.0X seconds), but not quite so for
configurations like ASAN. Instead of using an arbitrary timeout,
just use the test's deadline. The test program is invoked with
testenv.Command, which will send it a SIGQUIT before the deadline
expires.
Fixes#56420 (at least for the asan builder).
Change-Id: I0b13651cb07241401837ca2e60eaa1b83275b093
Reviewed-on: https://go-review.googlesource.com/c/go/+/684697
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
In the Go language a type assertion of a nil interface value
will always report false:
var err error
v, ok := err.(error) // always reports (nil, false)
Consequently, assertion on a reflect.Value.Interface()
will also report false:
var err error
rv := ValueOf(&err).Elem()
v, ok := rv.Interface().(error) // reports (nil, false)
However, prior to this change, a TypeAssert would report true:
var err error
rv := ValueOf(&err).Elem()
v, ok := TypeAssert[error](rv) // reports (nil, true)
when it should report false.
This fixes TypeAssert to match the Go language by
pushing the typ != v.typ check to the very end after
we have validated that neither v nor T are interface kinds.
Fixes#74404
Change-Id: Ie14d5cf18c8370c3e27ce4bdf4570c89519d8a16
Reviewed-on: https://go-review.googlesource.com/c/go/+/684675
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Mateusz Poliwczak <mpoliwczak34@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
On Windows, the process might not have read permission on the parent
directory, but still can delete files in it. This change allows
RemoveAll to open the parent directory with minimal permissions, which
is sufficient for deleting child files.
Fixes#74134.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-arm64
Change-Id: I5d5c5977caaebf6e0f93fb2313b0ceb346f70e05
Reviewed-on: https://go-review.googlesource.com/c/go/+/684515
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Add a section to the package doc which details the security
considerations of using encoding/json, in particular with respect to
parser misalignment issues.
Additionally, clarify previously ambiguous statement in the Unmarshal
doc about how case is used when matching keys in objects, and add a note
about how duplicate keys are handled.
Fixes#14750
Change-Id: I66f9b845efd98c86a684d7333b3aa8a456564922
Reviewed-on: https://go-review.googlesource.com/c/go/+/684315
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Roland Shoemaker <roland@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
If a goroutine is synchronously preempted, then taking a
frame-pointer-based stack trace at that preemption will skip PC of the
caller of the function which called into morestack. This happens because
the frame pointer is pushed to the stack after the preamble, leaving the
stack in an odd state for frame pointer unwinding.
Deal with this by marking a goroutine as synchronously preempted and
using that signal to load the missing PC from the stack. On LR platforms
this is available in gp.sched.lr. On non-LR platforms like x86, it's at
gp.sched.sp, because there are no args, no locals, and no frame pointer
pushed to the SP yet.
For #68090.
Change-Id: I73a1206d8b84eecb8a96dbe727195da30088f288
Reviewed-on: https://go-review.googlesource.com/c/go/+/684435
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Nick Ripley <nick.ripley@datadoghq.com>
The existing js/wasm implementation of RoundTrip calls abort() on the
fetch() call when the context is canceled but does not wait for for the
resulting promise to be rejected. The result is the failure callback for the
promise will be called at some later point in time when the promise
rejection is handled. In some case this callback may be called after the Go
program has exited resulting in "Go program has already exited" errors.
Fixes#57098
Change-Id: Ia37fd22cb9f667dbb0805ff5db0ceb8fdba7246b
Reviewed-on: https://go-review.googlesource.com/c/go/+/680937
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Currently, if there is a BSS reference and a DATA symbol
definition with the same name, we pick the DATA symbol, as it
contains the right content. In this case, if the BSS reference
has a larger size, we error out, because it is not safe to access
a smaller symbol as if it has a larger size.
Sometimes code declares a global variable in Go and defines it
in assembly with content. They are expected to be of the same
size. However, in ASAN mode, we insert a red zone for the variable
on the Go side, making it have a larger size, whereas the assembly
symbol is unchanged. This causes the Go reference (BSS) has a
larger size than the assembly definition (DATA). It results in an
error currently. This code is valid and safe, so we should permit
that.
We support this case by increasing the symbol size to match the
larger size (of the BSS side). The symbol content (from the DATA
side) is kept. In some sense, we merge the two symbols. When
loading symbols, it is not easy to change its size (as the object
file may be mapped read-only), so we add it to a fixup list, and
fix it up later after all Go symbols are loaded. This is a very
rare case, so the list should not be long.
We could limit this to just ASAN mode. But it seems okay to allow
this in general. As long as the symbol has the larger size, it is
safe to access it with the larger size.
Fixes#74314.
Change-Id: I3ee6e46161d8f59500e2b81befea11e563355a57
Reviewed-on: https://go-review.googlesource.com/c/go/+/684236
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
CL 649035 and CL 649079 updated escape analysis to rewrite
certain operands in OMAKE and OCONVIFACE nodes from non-constant
expressions to basic literals that evaluate to the same value.
However, when doing that rewriting, we need to evaluate any
side effects prior to replacing the expression, which is what
this CL now does.
Issue #74379 reported a problem with OCONVIFACE nodes due to CL 649079.
CL 649035 has essentially the same issue with OMAKE nodes. To illustrate
that, we add a test for the OMAKE case in fixedbugs/issue74379b.go,
which fails without this change. To avoid introducing an unnecessary
temporary for OMAKE nodes, we also conditionalize the main work of
CL 649035 on whether the OMAKE operand is already an OLITERAL.
CL 649555 and CL 649078 were related changes that created read-only
global storage for composite literals used in an interface conversion.
This CL adds a test in fixedbugs/issue74379c.go to illustrate
that they do not have the same problem.
Updates #71359Fixes#74379
Change-Id: I6645575ef34f1fe2b0241a22dc205875d66b7ada
Reviewed-on: https://go-review.googlesource.com/c/go/+/684116
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Keith Randall <khr@google.com>
This follows the precedent set by:
bufio.Writer.AvailableBuffer
bytes.Buffer.AvailableBuffer
both with methods that return a zero-length buffer that
is intended to only be used with a following Write call.
This keeps the older UnusedBuffer method around so that
at least one commit that has both methods for migration purposes.
Updates #71497
Change-Id: I3815f593e09f645280ae5ad9cbdd63a6c147123b
Reviewed-on: https://go-review.googlesource.com/c/go/+/683896
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
The hugo binary gets slower, potentially dramatically so, with
GOEXPERIMENT=greenteagc. The root cause is page mapping churn. The Green
Tea code introduced a new implicit nil check on value in a
freshly-allocated span to clear some new heap metadata. This nil check
would read the fresh memory, causing Linux to back that virtual address
space with an RO page. This would then be almost immediately written to,
causing Linux to possibly flush the TLB and find memory to replace that
read-only page (likely deduplicated as just the zero page).
This CL fixes the issue by replacing the implicit nil check, which is a
memory read expected to fault if it's truly nil, with an explicit one.
The explicit nil check is a branch, and thus makes no reads to memory.
The result is that the hugo binary no longer gets slower.
No regression test because it doesn't seem possible without access to OS
internals, like Linux tracepoints. We briefly experimented with RSS
metrics, but they're inconsistent. Some system RSS metrics count the
deduplicated zero page, while others (like those produced by
/proc/self/smaps) do not.
Instead, we'll add a new benchmark to our benchmark suite, separately.
For #73581.
Fixes#74375.
Change-Id: I708321c14749a94ccff55072663012eba18b3b91
Reviewed-on: https://go-review.googlesource.com/c/go/+/684015
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
When an application calls runtime.GOMAXPROCS(runtime.GOMAXPROCS(0)), the
runtime does not need to change the actual GOMAXPROCS value (via STW).
However, this call must still transition from "automatic" to "custom"
GOMAXPROCS state, thus disabling background updates.
Thus this case shouldn't return quite as early as it currently does.
Change-Id: I6a6a636c42f73996532bd9f7beb95e933256c9e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/683815
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
There is a non-public zstd decoder in the stdlib (CL 473356) and
also zstd compressed testdata already present.
Delete testdata/code.json.gz and
instead use internal/jsontest/testdata/golang_source.json.zst,
which has exactly the same content:
$ cat internal/jsontest/testdata/golang_source.json.zst | zstd -d | sha1sum
3f70b6fd429f4aba3e8e1c3e5a294c8f2e219a6e -
$ cat testdata/code.json.gz | zstd -d | sha1sum
3f70b6fd429f4aba3e8e1c3e5a294c8f2e219a6e -
This will reduce the size of the final Go release by 118KB.
Updates #71845
Change-Id: I6da2df27bd260befc0a44c6bc0255365be0a5b0f
Reviewed-on: https://go-review.googlesource.com/c/go/+/525516
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Bypass: Damien Neil <dneil@google.com>
Based on the discussion in #71631, it is hotly contested
whether the default JSON representation for a Go time.Duration
should be the time.Duration.String format or
a particular profile of ISO 8601.
Regardless of the default, it seems clear that we should
at least support ISO 8601 if specified via a format flag.
Note that this CL does not alter the default representation.
Unfortunately, ISO 8601 is a large and evolving standard
with many optional extensions and optional restrictions.
Thus, the term "ISO 8601 duration" unfortunately does not
resolve to a particular grammar, nor one that is stable.
However, there is precedence that we can follow in this matter.
JSON finds its heritage in JavaScript and
JavaScript is adding a Temporal.Duration type whose default
JSON representation is ISO 8601.
There is a well-specified grammar for their particular
profile of ISO 8601, which is documented at:
https://tc39.es/proposal-temporal/#prod-Duration
This particular CL adds support for ISO 8601 according to
the exact same grammar that JavaScript uses.
While Temporal.Duration is technically still a proposal,
it is already in stage 3 of the TC39 proposal process
(i.e., "no changes to the proposal are expected"
and "has been recommended for implementation")
and therefore close to final adoption.
One major concern with ISO 8601 is that it supports
nominal date units like years, months, weeks, and days
that do not have an accurate meaning without being
anchored to a particular point in time and place on Earth.
Fortunately, JavaScript (by default) avoids producing
Temporal.Duration values with nominal units unless
arithmetic in JavaScript explicitly sets a largestUnits
value that is larger than "hours". In the Go implementation,
we support syntactically parsing the full ISO 8601 grammar
(according to JavaScript), but semantically report an error if
nominal units are present. This ensures that ISO 8601 durations
remain accurate so long as they only use the accurate units
of hours, minutes, or seconds.
Updates #71631
Change-Id: I983593662f2150461ebc486a5acfeb72f0286939
Reviewed-on: https://go-review.googlesource.com/c/go/+/682403
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
In this blog:
https://blog.trailofbits.com/2025/06/17/unexpected-security-footguns-in-gos-parsers/
the concern was raised that whenever "-" is combined with other options,
the "-" is intepreted as as a name, rather than an ignored field,
which may go contrary to user expectation.
Static analysis demonstrates that there are ~2k instances of `json:"-,omitempty"
in the wild, where almost all of them intended for the field to be ignored.
To prevent this footgun, reject any tags that has "-," as a prefix
and warn the user to choose one of the reasonable alternatives.
The documentation of json/v2 already suggests `json:"'-'"`
as the recommended way to explicitly specify dash as the name.
See Example_fieldNames for example usages of the single-quoted literal.
Update the v1 json documentation to suggest the same thing.
Updates #71497
Change-Id: I7687b6eecdf82a5d894d057c78a4a90af4f5a6e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/683175
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
This also updates wasip1_wasm to use a 8MiB stack, which is
the same stack size as what is used by go_js_wasm_exec.
The increase of stack size is necessary because the jsonv2
tests exercise that the jsonv2 and jsontext packages support
a hard limit of a maximum JSON nesting depth of 10000.
However, even with a depth limit of 10000, this still exceeds
the previously specified maximum stack size of 1 MiB.
For use of JSON with untrusted inputs in WASM,
we really need to support #56733 as there is no right answer
for the default max depth limit to use since the max wasm
stack size is determined on a per-system basis.
Updates #71845
Change-Id: I3b32c58cc9f594a5c59bb3e4b20f5e86d85d8209
Reviewed-on: https://go-review.googlesource.com/c/go/+/683575
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The gc-stress test is useful for trying to exercise GC-related trace
events by producing a lot of them in many different situations.
Unfortunately this test is flaky, because allocating in a loop can
easily out-run the GC when it's trying to preempt the allocating
goroutine.
It's been a long standing problem that a program that allocates in a
loop can outrun a GC. The problem isn't the GC persay, it's consistently
correlated with a high STW time (likely a high 'stopping' time, not a
'stopped' time), suggesting that in the window of time when the garbage
collector is trying to stop all goroutines, they continue to allocate.
This should probably be fixed in general, but for now, let's focus on
this flaky test.
This CL changes the gc-stress test to (1) set a memory limit and (2) do
more work in between allocations. (2) is really what makes things less
flaky, but (2) unfortunately also means the GC is less exercised. That's
where (1) comes in. By setting a low memory limit, we increase GC
activity (in particular, assist activity). The memory limit also helps
prevent the heap from totally blowing up due to the heap goal inflating
from floating garbage, but it's not perfect.
After this change, under stress2, this test exceeds a heap size of 500
MiB only 1 in 5000 runs on my 64-vCPU VM. Before this change, it got
that big about 1/4th of the time.
Fixes#74052.
Change-Id: I49233c914c8b65b1d593d3953891fddda6685aec
Reviewed-on: https://go-review.googlesource.com/c/go/+/683515
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
During toolchain selection, the GOEXPERIMENT value may not be valid for
the current version (but it is valid for the selected version). In this
case, cfg.ExperimentErr is set and cfg.Experiment is nil.
Normally cmd/go main exits when ExperimentErr is set, so Experiment is
~never nil. But that is skipped during toolchain selection, and
fips140.Init is used during toolchain selection.
Fixes#74111.
Change-Id: I6a6a636c65ee5831feaf3d29993a60613bbec6f2
Reviewed-on: https://go-review.googlesource.com/c/go/+/680976
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Currently the mspan limit field is set after allocSpan returns, *after*
the span has already been published to the GC (including the
conservative scanner). But the limit field is load-bearing, because it's
checked to filter out invalid pointers. A stale limit value could cause
a crash by having the conservative scanner access allocBits out of
bounds.
Fix this by setting the mspan limit field before publishing the span.
For large objects and arena chunks, we adjust the limit down after
allocSpan because we don't have access to the true object's size from
allocSpan. However this is safe, since we first initialize the limit to
something definitely safe (the actual span bounds) and only adjust it
down after. Adjusting it down has the benefit of more precise debug
output, but the window in which it's imprecise is also fine because a
single object (logically, with arena chunks) occupies the whole span, so
the 'invalid' part of the memory will just safely point back to that
object. We can't do this for smaller objects because the limit will
include space that does *not* contain any valid objects.
Fixes#74288.
Change-Id: I0a22e5b9bccc1bfdf51d2b73ea7130f1b99c0c7c
Reviewed-on: https://go-review.googlesource.com/c/go/+/682655
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Almost everywhere we stop the world we casGToWaitingForGC to prevent
mutual deadlock with the GC trying to scan our stack. This historically
was only necessary if we weren't stopping the world to change the GC
phase, because what we were worried about was mutual deadlock with mark
workers' use of suspendG. And, they were the only users of suspendG.
In Go 1.22 this changed. The execution tracer began using suspendG, too.
This leads to the possibility of mutual deadlock between the execution
tracer and a goroutine trying to start or end the GC mark phase. The fix
is simple: make the stop-the-world calls for the GC also call
casGToWaitingForGC. This way, suspendG is guaranteed to make progress in
this circumstance, and once it completes, the stop-the-world can
complete as well.
We can take this a step further, though, and move casGToWaitingForGC
into stopTheWorldWithSema, since there's no longer really a place we can
afford to skip this detail.
While we're here, rename casGToWaitingForGC to casGToWaitingForSuspendG,
since the GC is now not the only potential source of mutual deadlock.
Fixes#72740.
Change-Id: I5e3739a463ef3e8173ad33c531e696e46260692f
Reviewed-on: https://go-review.googlesource.com/c/go/+/681501
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Our attempt to evenly distribute tests across shards struggles a bit
because certain long-running targets are very difficult to distinguish
in ResultDB, namely racebench and the test directory tests. These are
the only tests where the JSON output from dist omits the variant from
the package, making it impossible to distinguish them in the test result
data. My current suspicion is that this is preventing the load balancing
from being effective for the race builders in particular, though I worry
the longtest builders have a similar situation with the test directory
tests.
For #65814.
Change-Id: I5804c2af092ff9aa4a3f0f6897b4a57c4628f837
Reviewed-on: https://go-review.googlesource.com/c/go/+/681955
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Issue #74045 describes a scenario in which gopark is inlined into
readTrace, such that there are no preemption points. This is only a
problem because readTrace spins if trace.shutdown is set, through
traceReaderAvailable. However, trace.shutdown is almost certainly
overkill for traceReaderAvailable. The first condition, checking whether
the reader gen and the flushed gen match, should be sufficient to ensure
the reader wakes up and finishes flushing all buffers. The first
condition is also safe because it guarantees progress. In the case of
shutdown, all the trace work that will be flushed has been flushed, and
so the trace reader will exit into a regular goroutine context when
it's finished. If not shutting down, then the trace reader will release
doneSema, increase readerGen, and then the gopark unlockf will let it
block until new work actually comes in.
Fixes#74045.
Change-Id: Id9b15c277cb731618488771bd484577341b68675
Reviewed-on: https://go-review.googlesource.com/c/go/+/680738
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Nick Ripley <nick.ripley@datadoghq.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
A previous change renamed Value.Uint64 to Value.ToUint64 to accomodate
string values. The method for a string value is then Value.ToString,
while the method for a debug string (for example, for fmt) is just
called String, as per fmt.Stringer.
This change follows a request from Dominik Honnef, maintainer of
gotraceui, to make Value follow the conventions of the reflect package.
The Value type there has a method String which fulfills both purposes:
getting the string for a String Value, and as fmt.Stringer. It's
not exactly pretty, but it does make sense to just stick to convention.
Change-Id: I55b364be88088d2121527bffc833ef03dbdb9764
Reviewed-on: https://go-review.googlesource.com/c/go/+/680978
Reviewed-by: Florian Lehner <lehner.florian86@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
This change switches from using testenv.Command to
testenv.CommandContext which is a little bit friendlier. It also
switches away from using 'go run' to 'go build' and running the
resulting binary explicitly. This helps eliminate any questions about
signal handling and propagation.
For #72740.
Change-Id: Ife8010da89a7bc439e061fe0c9c6b1f5620d90f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/680977
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
This runs the ssa/_gen generator writing files into
a temporary directory, and then checks that there are
no differences with what is currently in the ssa directory,
and also checks that any file with the "generated from
_gen/..." header was actually generated, and checks that
the headers on the generated file match the expected
header prefix.
Change-Id: Ic8eeb0b06cf6f2e576a013e865b331a12d3a77aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/680615
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Merge List:
+ 2025-06-10 7fa2c736b3 os: disallow Root.Remove(".") on Plan 9, js, and Windows
+ 2025-06-10 281cfcfc1b runtime: handle system goroutines later in goroutine profiling
+ 2025-06-10 4f86f22671 testing/synctest, runtime: avoid panic when using linker-alloc WG from bubble
+ 2025-06-10 773701a853 internal/trace: pass GOTRACEBACK=crash to testprogs
+ 2025-06-10 fb0c27c514 os: do not follow dangling symlinks in Root when O_CREATE|O_EXCL on AIX
+ 2025-06-10 1cafdfb63b net/http: make the zero value of CrossOriginProtection work
+ 2025-06-10 a35701b352 cmd/dist: only install necessary tools when doing local test
+ 2025-06-10 a189516d3a runtime: don't do a direct G handoff in semrelease on systemstack
+ 2025-06-10 f18d046568 all.{bash,rc}: use "../bin/go tool dist" instead of "%GOTOOLDIR%/dist" print build info
+ 2025-06-09 ee7bfbdbcc cmd/compile/internal/ssa: fix PPC64 merging of (AND (S[RL]Dconst ...)
+ 2025-06-09 985d600f3a runtime: use small struct TestSynctest to ensure cleanups run
+ 2025-06-09 848a768ba7 runtime: clarify stack traces for bubbled goroutines
+ 2025-06-09 049a5e6036 runtime: return a different bubble deadlock error when main goroutine is done
+ 2025-06-09 ac1686752b cmd/internal/doc: increase version of pkgsite doc command that's run
Change-Id: Iba7b2c2f06e91a39fa039c08170e6054e50de3c6
Windows already forbids this, since removing the root causes a
sharing violation (can't delete the directory while the os.Root
has a handle open to it), but add a more explicit check for
attempts to delete "." and return EINVAL.
Note that this change to Windows doesn't affect operations like
Root.Remove("dir/."), since the path is cleaned into just "dir"
before attempting the deletion.
Fixes#73863
Change-Id: I0f45ccb6c9f171d3a52831632c134150388d77b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/679377
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Before CL 650697, there was only one system goroutine that could
dynamically change between being a user goroutine and a system
goroutine, and that was the finalizer/cleanup goroutine. In goroutine
profiles, it was handled explicitly. It's status would be checked during
the first STW, and its stack would be recorded. This let the goroutine
profiler completely ignore system goroutines once the world was started
again.
CL 650697 added dedicated cleanup goroutines (there may be more than
one), and with this, the logic for finalizer goroutines no longer
scaled. In that CL, I let the isSystemGoroutine check be dynamic and
dropped the special case, but this was based on incorrect assumptions.
Namely, it's possible for the scheduler to observe, for example, the
finalizer goroutine as a system goroutine and ignore it, but then later
the goroutine profiler itself sees it as a user goroutine. At that point
it's too late and already running. This violates the invariant of the
goroutine profile that all goroutines are handled by the profiler before
they start executing. In practice, the result is that the goroutine
profiler can crash when it checks this invariant (not checking the
invariant means racily reading goroutine stack memory).
The root cause of the problem is that these system goroutines do not
participate in the goroutine profiler's state machine. Normally, when
profiling, goroutines transition from 'absent' to 'in-progress' to
'satisfied'. However with system goroutines, the state machine is
ignored entirely. They always stay in the 'absent' state. This means
that if a goroutine transitions from system to user, it is eligible for
a profile record when it shouldn't be. That transition shouldn't be
allowed to occur with respect to the goroutine profiler, because the
goroutine profiler is trying to snapshot the state of every goroutine.
The fix to this problem is simple: don't ignore system goroutines. Let
them participate in the goroutine profile state machine. Instead, decide
whether or not to record the stack after the goroutine has been acquired
for goroutine profiling. This means if the scheduler observes the
finalizer goroutine as a system goroutine, it will get promoted in the
goroutine profiler's state machine, and no other part of the goroutine
profiler will observe the goroutine again. Simultaneously, the
stack record for the goroutine will be correctly skipped.
Fixes#74090.
Change-Id: Icb9a164a033be22aaa942d19e828e895f700ca74
Reviewed-on: https://go-review.googlesource.com/c/go/+/680477
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
We associate WaitGroups with synctest bubbles by attaching a
special to the WaitGroup. It is not possible to attach a special
to a linker-allocated value, such as:
var wg sync.WaitGroup
Avoid panicking when accessing a linker-allocated WaitGroup
from a bubble. We have no way to associate these WaitGroups
with a bubble, so just treat them as always unbubbled.
This is probably fine, since the WaitGroup was always
created outside the bubble in this case.
Fixes#74005
Change-Id: Ic71514b0b8d0cecd62e45cc929ffcbeb16f54a55
Reviewed-on: https://go-review.googlesource.com/c/go/+/679695
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The failures in #70310 are hard to decipher. The cases where the lock is
being held either don't really make sense (the STW failures) or the
goroutine that fails is 'running on another thread' and we don't get a
stack trace. In fact, such a goroutine exists even in the STW cases.
Since reproducing this is going to be hard (very few failures over a 2
year span) let's set GOTRACEBACK=crash for these testprogs so next time
it happens we can see why.
For #70310.
Change-Id: I81a780aa82b173d42973f06911cb243f33352be1
Reviewed-on: https://go-review.googlesource.com/c/go/+/680476
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
OpenFile with O_CREATE|O_EXCL should not follow dangling symlinks.
On AIX it does, because AIX's openat(2) apparently returns ELOOP
in this case. Most Unices return EEXIST.
Ensure that we never follow symlinks in the final component of
the path when opening a file with O_CREATE|O_EXCL.
Fixes#73924
Change-Id: I869afb7faefccb0bb29d155553a7d7e5be80467d
Reviewed-on: https://go-review.googlesource.com/c/go/+/677735
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Currently, CrossOriginProtection must be constructed by
NewCrossOriginProtection. If you try to use the zero value, most
methods will panic with a nil dereference.
This CL makes CrossOriginProtection use on-demand initialization
instead, so the zero value has the same semantics as the value
currently returned by NewCrossOriginProtection. Now,
NewCrossOriginProtection just constructs the zero value.
We keep NewCrossOriginProtection by analogy to NewServeMux.
Updates #73626Fixes#74089.
Change-Id: Ia80183eb6bfdafb0e002271c0b25c2d6230a159a
Reviewed-on: https://go-review.googlesource.com/c/go/+/680396
Auto-Submit: Austin Clements <austin@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
semrelease is safe to call on the system stack (since it just readies
goroutines) except for the fact that it might perform a direct G
handoff and call into the scheduler. If handoff is set to false this is
exceptionally rare, but could happen, and has happened for the trace
reader goroutine which releases a trace.doneSema.
Fixes#73469.
Change-Id: I37ece678bc4721bbb6e5879d74daac762b7d742a
Reviewed-on: https://go-review.googlesource.com/c/go/+/680315
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
After CL 677558, when running all.bash, the binaries of commands such
as dist, nm, and pprof are no longer built by default, so when running
all.bash, "./all.bash: line 13: /home/golang/pkg/tool/linux_amd64/dist:
No such file or directory" will be printed, and the return result of
the all.bash script is non-zero.
Although the "dist" command won't be installed in $GOTOOLDIR anymore,
but it will be built and cached, and ../bin/go tool dist will reuse the
cached binary.
For #71867
Change-Id: I802eeafdb866e7d80c42da3e0955bb32def7b037
Reviewed-on: https://go-review.googlesource.com/c/go/+/680135
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Use the synctest bubble ID to identify bubbles in traces,
rather than the goroutine ID of the bubble's root goroutine.
Some waitReasons include a "(synctest)" suffix to distinguish
a durably blocking state from a non-durable one. For example,
"chan send" vs. "chan send (synctest)". Change this suffix
to "(durable)".
Always print a "(durable)" sufix for the state of durably
blocked bubbled goroutines. For example, print "sleep (durable)".
Drop the "[not] durably blocked" text from goroutine states,
since this is now entirely redundant with the waitReason.
Old:
goroutine 8 [chan receive (synctest), synctest bubble 7, durably blocked]:
goroutine 9 [select (no cases), synctest bubble 7, durably blocked]:
New:
goroutine 8 [chan receive (durable), synctest bubble 1]:
goroutine 9 [select (no cases) (durable), synctest bubble 1]:
Change-Id: I89112efb25150a98a2954f54d1910ccec52a5824
Reviewed-on: https://go-review.googlesource.com/c/go/+/679376
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
The synctest.Test function waits for all goroutines in a bubble to
exit before returning. If there is ever a point when all goroutines
in a bubble are durably blocked, it panics and reports a deadlock.
Panic with a different message depending on whether the bubble's
main goroutine has returned or not. The main goroutine returning
stops the bubble clock, so knowing whether it is running or not
is useful debugging information.
The new panic messages are:
deadlock: all goroutines in bubble are blocked
deadlock: main bubble goroutine has exited but blocked goroutines remain
Change-Id: I94a69e79121c272d9c86f412c1c9c7de57ef27ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/679375
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The Go 1.25 RC is due soon. This is the time to once again update all
golang.org/x/... module versions that contribute packages to the std and
cmd modules in the standard library to latest master versions.
For #36905.
[git-generate]
go install golang.org/x/build/cmd/updatestd@latest
go install golang.org/x/tools/cmd/bundle@latest
updatestd -goroot=$(pwd) -branch=master
cat << EOF | patch
diff --git a/src/cmd/go/testdata/script/test_json_build.txt b/src/cmd/go/testdata/script/test_json_build.txt
index df8863ae03..2a572ace72 100644
--- a/src/cmd/go/testdata/script/test_json_build.txt
+++ b/src/cmd/go/testdata/script/test_json_build.txt
@@ -56,7 +56,7 @@ stdout '"Action":"fail","Package":"m/cycle/p","Elapsed":.*,"FailedBuild":"m/cycl
! go test -json -o=$devnull ./veterror
stdout '"ImportPath":"m/veterror \[m/veterror.test\]","Action":"build-output","Output":"# m/veterror\\n"'
stdout '"ImportPath":"m/veterror \[m/veterror.test\]","Action":"build-output","Output":"# \[m/veterror\]\\n"'
-stdout '"ImportPath":"m/veterror \[m/veterror.test\]","Action":"build-output","Output":"veterror(/|\\\\)main_test.go:9:9: fmt.Printf format %s reads arg #1, but call has 0 args\\n"'
+stdout '"ImportPath":"m/veterror \[m/veterror.test\]","Action":"build-output","Output":"veterror(/|\\\\)main_test.go:9:21: fmt.Printf format %s reads arg #1, but call has 0 args\\n"'
stdout '"ImportPath":"m/veterror \[m/veterror.test\]","Action":"build-fail"'
stdout '"Action":"start","Package":"m/veterror"'
stdout '"Action":"output","Package":"m/veterror","Output":"FAIL\\tm/veterror \[build failed\]\\n"'
EOF
Change-Id: I6a8d35acdeab90c3bbd6395b8b1abb021673b5cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/678556
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
This CL makes two changes to reduce the predictability
with which bubbled timers fire.
When asynctimerchan=0 (the default), regular timers with an associated
channel are only added to a timer heap when some channel operation
is blocked on that channel. This allows us to garbage collect
unreferenced, unstopped timers. Timers in a synctest bubble, in
contrast, are always added to the bubble's timer heap.
This CL changes bubbled timers with a channel to be handled the
same as unbubbled ones, adding them to the bubble's timer heap only
when some channel operation is blocked on the timer's channel.
This permits unstopped bubbled timers to be garbage collected,
but more importantly it makes all timers past their deadline
behave identically, regardless of whether they are in a bubble.
This CL also changes timer scheduling to execute bubbled timers
immediately when possible rather than adding them to a heap.
Timers in a bubble's heap are executed when the bubble is idle.
Executing timers immediately avoids creating a predictable
order of execution.
For #73850Fixes#73934
Change-Id: If82e441546408f780f6af6fb7f6e416d3160295d
Reviewed-on: https://go-review.googlesource.com/c/go/+/678075
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
There are two additional sources of memory overhead per P that come from
greenteagc. One is for ptrBuf, but on platforms other than Windows it
doesn't actually cost anything due to demand-paging (Windows also
demand-pages, but the memory is 'committed' so it still counts against
OS RSS metrics). The other is for per-sizeclass scan stats. However when
greenteagc is disabled, most of these scan stats are completely unused.
The worst-case memory overhead from these two sources is relatively
small (about 10 KiB per P), but for programs with a small memory
footprint running on a machine with a lot of cores, this can be
significant (single-digit percent).
This change does two things. First, it puts ptrBuf initialization behind
the greenteagc experiment, so now that memory is never allocated by
default. Second, it abstracts the implementation details of scan stat
collection and emission, such that we can have two different
implementations depending on the build tag. This lets us remove all the
unused stats when the greenteagc experiment is disabled, reducing the
memory overhead of the stats from ~2.6 KiB per P to 536 bytes per P.
This is enough to make the difference no longer noticable in our
benchmark suite.
Fixes#73931.
Change-Id: I4351f1cbb3f6743d8f5922d757d73442c6d6ad3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/678535
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
We shouldn't be installing these tools because we will remove them in
distpack. Installing the tools will also prevent us from testing what
happens when the tools are missing.
The changes below this on the stack, CL 677775 (cmd/doc: build cmd/doc
directly into the go command) and CL 677636 (cmd/go/internal/cfg: fix
GOROOT setting when forcing host config) are needed for this change to
pass tests. The doc change is being done so we preserve the properties
in the tests that doc can be invoked without doing a build. It's not
strictly necessary (we could just remove the tests) but it's nice to
have. The GOROOT setting is a significant bug in switching the
configuration to host mode: the value of GOROOT wasn't being reset,
which caused issues for go commands built with trimpath, because
runtime.GOROOT wouldn't have the correct goroot value.
For #71867
Change-Id: I4181711ba117066b7d62d7d013ad4b186871cfb7
Reviewed-on: https://go-review.googlesource.com/c/go/+/677558
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>
There are a couple of places where our tests expect that 'go doc'
doesn't need to do a build. Invoke the cmd/doc code directly by the go
command instead of starting the doc tool in a separate process so we can
preserve that property.
This change moves most of the doc code into the package
cmd/internal/doc, and exposes a Main function from that function that's
called both by the cmd/doc package, and by go doc.
This change makes couple of additional changes to intergrate doc into
the go command:
The counter.Open call and the increment of invocations counter are only
needed by cmd/doc. The go command will open the counters file and
increment a counter for the doc subcommand.
We add a cmd_go_bootstrap tagged variant of the file that defines go doc
so that we don't end up linking net into the bootstrap version of the go
command. We don't need doc in that version of the command.
We create a new flagSet rather than using flag.CommandLine because when
running as part of the go command, the flags to "go doc" won't be the top
level flags.
We change TestGoListTest in go_test.go to use gofmt instead of doc as an
example of a main package in cmd with an in-package test.
For #71867
Change-Id: I3e3df83e5fa266559606fdc086b461165e09f037
Reviewed-on: https://go-review.googlesource.com/c/go/+/677775
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>
We manage the state using a bunch of global config, so we need to make
sure we're doing things in the right order. In this case, the SetGOROOT
function was being called in init, setting the GOROOT on the global
Context, but when we reset the context in ForceHost we lost the goroot
configuration. We need to call SetGOROOT in ForceHost to re-set the
GOROOT on the new context.
This was uncovered by CL 677558 because a go command that was built with
trimpath would try to use its runtime.GOROOT(), which wouldn't be valid
in trimpath mode. Setting GOROOT properly with SetGOROOT will use the
value from findGOROOT, assuming GOROOT isn't set in the environment,
and findGOROOT will try to determine GOROOT using the path of the go
command executable.
For #71867
Change-Id: I731b6c5d859b4504fc128b29ab904e3a2886ff3c
Reviewed-on: https://go-review.googlesource.com/c/go/+/677636
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
This CL changes the representation of FileSet from a slice
to a tree, specifically an AVL tree keyed by the File's
base-end range. This makes a sequence of insertions using
AddExistingFiles much more efficient: creating a FileSet
of size n by a sequence of calls costs O(n log n), whereas
before it was O(n^2 log n) because of the repeated sorting.
The AVL tree is based on Russ' github.com/rsc/omap,
simplified for clarity and to reduce unnecessary dynamism.
We use an AVL tree as it is more strongly balanced than an
RB tree, optimising lookups at the expense of insertions.
The CL includes a basic unit test of the tree using
operations on pseudorandom values.
Benchmarks of Position lookups actually improve because
the tree avoids BinarySearchFunc's dynamic dispatch to cmp,
and the benchmark of AddExistingFiles is about 1000x (!) faster:
goos: darwin
goarch: arm64
pkg: go/token
cpu: Apple M1 Pro
│ old.txt │ new.txt │
│ sec/op │ sec/op vs base │
FileSet_Position/random-8 51.60n ± 1% 39.99n ± 1% -22.50% (p=0.000 n=9)
FileSet_Position/file-8 27.10n ± 3% 26.64n ± 1% ~ (p=0.168 n=9)
FileSet_Position/manyfiles-8 209.9n ± 17% 154.1n ± 9% -26.58% (p=0.000 n=9)
FileSet_AddExistingFiles/sequence-8 395930.3µ ± 4% 280.8µ ± 10% -99.93% (p=0.000 n=9)
Updates #73205
Change-Id: Iea59c624a6cedadc2673987a5eb0ebece67af9e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/675736
Reviewed-by: Robert Findley <rfindley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
In synctest bubbles, fire timers scheduled for the same instant
in a randomized order.
Pending timers are added to a heap ordered by the timer's wakeup time.
Add a per-timer random value, set when the timer is added to a heap,
to break ties between timers scheduled for the same instant.
Only inject this randomness in synctest bubbles. We could do so
for all timers at the cost of one cheaprand call per timer,
but given that it's effectively impossible to create two timers
scheduled for the same instant outside of a fake-time environment,
don't bother.
Fixes#73876
For #73850
Change-Id: Ie96c86a816f548d4c31e4e014bf9293639155bd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/677276
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
This change adds a function to expose the version set by the trace
reader after reading the trace header (in tests). The trace validator
needs to be able to determine what version of the trace it needs to
validate against. Clock snapshot checks have been disabled for
Windows and WASM.
For #63185
Change-Id: Ia3d63e6ed7a5ecd87e63292b84cc417d982aaa5a
Reviewed-on: https://go-review.googlesource.com/c/go/+/677695
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
This String method can potentially recurse infinitely, since %#x will
apparently call String if the method exists. This isn't well documented,
but cmd/vet will be updated soon to check this (when we update the
vendored x/tools dependency) so cut off the recursion by converting to
the underlying type first.
Change-Id: Ia6fc046c9eb56a5dd6a33772afd23da443a06116
Reviewed-on: https://go-review.googlesource.com/c/go/+/677261
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
sync.Cond.Wait is durably blocking. Waking a goroutine out of Cond.Wait
from outside its bubble panics.
Make this panic a fatal panic, since it leaves the notifyList in an
inconsistent state. We could do some work to make this a recoverable
panic, but the complexity doesn't seem worth the outcome.
For #67434
Change-Id: I88874c1519c2e5c0063175297a9b120cedabcd07
Reviewed-on: https://go-review.googlesource.com/c/go/+/675617
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
The primary benefit of reference tables is to the linker, though they
are also reasonably compact as compared to absolute element indices. It
is worth also checking if reference table structure is similarly
exploited past the IR linking stage.
Ideally, the reference table definition would live in / near the linker.
As it stands, it's a bit hard to infer the purpose of the reference
tables when looking at pkgbits in isolation.
Change-Id: I496aca5a4edcf28e66fa7863ddfa4d825e1b2e89
Reviewed-on: https://go-review.googlesource.com/c/go/+/675596
Auto-Submit: Mark Freeman <mark@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
We already guarantee that no automatic updates to GOMAXPROCS occur after
a GOMAXPROCS call returns. This is easily achieved by having the update
goroutine double-check that updates are still allowed during STW before
committing the new value.
However, it is possible for sysmon to concurrently run defaultGOMAXPROCS
to compute a new GOMAXPROCS value after GOMAXPROCS returns. This new
value will be discarded later, but we'll still perform the system calls
necessary to compute the new value.
Normally this distinction doesn't matter, but if you want to sandbox a
Go program, then you may want to disable GOMAXPROCS updates to reduce
the system call footprint. A call to GOMAXPROCS will disable updates,
but without a guarantee on when sysmon will observe the change it is
somewhat fragile.
Add explicit synchronization between GOMAXPROCS and sysmon to guarantee
that sysmon won't run defaultGOMAXPROCS after GOMAXPROCS returns.
The synchronization is a bit complex because we can't hold a mutex
across STW, nor take a semaphore from sysmon, but the result isn't too
bad.
One oddity is that sched.customGOMAXPROCS and gomaxprocs are no longer
updated in lockstep (even though both are protected by sched.lock), but
I don't believe anything should depend on that.
For #73193.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-staticlockranking
Change-Id: I6a6a636cff243a9b69ac1b5d2f98925648e60236
Reviewed-on: https://go-review.googlesource.com/c/go/+/677037
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Add support to internal/synctest for managing associations between
arbitrary pointers and synctest bubbles. (Implemented internally to
the runtime package by attaching a special to the pointer.)
Associate WaitGroups with bubbles.
Since WaitGroups don't have a constructor,
perform the association when Add is called.
All Add calls must be made from within the same bubble,
or outside any bubble.
When a bubbled goroutine calls WaitGroup.Wait,
the wait is durably blocking iff the WaitGroup is associated
with the current bubble.
Change-Id: I77e2701e734ac2fa2b32b28d5b0c853b7b2825c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/676656
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
The updatemaxprocs metric logic is currently backwards. We only
increment the metric when we update GOMAXPROCS, but that only occurs if
updatemaxprocs is enabled.
Instead, the metric is supposed to increment when updatemaxprocs is
disabled and there would be different behavior if it were enabled.
Theoretically we should run the entire update system in a dry run mode,
and only bail out right before committing updates. But that is an awful
lot of effort for a feature that is disabled. Plus some users (like
sandboxes) want to completely disable the update syscalls
(sched_getaffinity and pread64). If we still do dry run updates then we
need an additional GODEBUG for completely disabling functionality.
This CL also avoids starting the update goroutine at all if disabled,
since it isn't needed.
For #73193.
Change-Id: I6a6a636ceec8fced44e36cb27dcb1b4ba51fce33
Reviewed-on: https://go-review.googlesource.com/c/go/+/677036
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
This CL fixes a panic in NewFromFiles when it is provided files
produced by the parser in SkipObjectResolution mode, which skips
the step of connecting ast.Idents to (deprecated) ast.Objects.
Instead of calling ast.NewPackage, which performs a number of
unnecessary steps, we just construct the ast.Package directly.
Fixes#66290
Change-Id: Id55bd30d8afb9d396c3901070e7607c5a22030d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/675036
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
On these platforms, we set up a frame pointer record below
the current stack pointer, so when we're in duffcopy or duffzero,
we get a reasonable traceback. See #73753.
But because this frame pointer record is below SP, it is vulnerable.
Anything that adds a new stack frame to the stack might clobber it.
Which actually happens in #73748 on amd64. I have not yet come across
a repro on arm64, but might as well be safe here.
The only real situation this could happen is when duffzero or duffcopy
is passed a nil pointer. So we can just avoid the problem by doing the
nil check outside duffzero/duffcopy. That way we never add a frame
below duffzero/duffcopy. (Most other ways to get a new frame below the
current one, like async preempt or debugger-generated calls, don't
apply to duffzero/duffcopy because they are runtime functions; we're
not allowed to preempt there.)
Longer term, we should stop putting stuff below SP. #73753 will
include that as part of its remit. But that's not for 1.25, so we'll
do the simple thing for 1.25 for this issue.
Fixes#73748
Change-Id: I913c49ee46dcaee8fb439415a4531f7b59d0f612
Reviewed-on: https://go-review.googlesource.com/c/go/+/676916
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@google.com>
In CL 660696, we made the linker to choose the symbol of the
larger size in case there are multiple contentless declarations of
the same symbol. We also made it emit an error in the case that
there are a contentless declaration of a larger size and a
definition with content of a smaller size. In this case, we should
choose the definition with content, but the code accesses it
through the declaration of the larger size could fall into the
next symbol, potentially causing data corruption. So we disallowed
it.
There is one spcial case, though, that some code uses a linknamed
variable declaration to reference a function in assembly, in order
to take its address. The variable is often declared as uintptr.
The function symbol is the definition, which could sometimes be
shorter. This would trigger the error case above, causing existing
code failing to build.
This CL allows it as a special case. It is still not safe to
access the variable's content. But it is actually okay to just
take its address, which the existing code often do.
Fixes#73617.
Change-Id: I467381bc5f6baa16caee6752a0a824c7185422f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/676636
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Cleanup functions and finalizers must not run in a synctest bubble.
If they did, a function run by the GC at an unpredictable time
could unblock a bubble that synctest believes is durably
blocked.
Add a test verifying that cleanups and finalizers are always
run by non-bubbled goroutines. (This is already the case because
we never add system goroutines to a bubble.)
For #67434
Change-Id: I5a48db2b26f9712c3b0dc1f425d99814031a2fc1
Reviewed-on: https://go-review.googlesource.com/c/go/+/675257
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
When creating a new *ir.Name or *ir.LinksymOffsetExpr to represent
a composite literal stored in the read-only data section, we should
use the original type of the expression that was found via
ir.ReassignOracle.StaticValue. (This is needed because the StaticValue
method can traverse through OCONVNOP operations to find its final
result.)
Otherwise, the compilation may succeed, but the linker might erroneously
conclude that a type is not used and prune an itab when it should not,
leading to a call at execution-time to runtime.unreachableMethod, which
throws "fatal error: unreachable method called. linker bug?".
The tests exercise both the case of a zero value struct literal that
can be represented by the read-only runtime.zeroVal, which was the case
of the simplified example from #73888, and also modifies that example to
test the non zero value struct literal case.
This CL makes two similar changes for those two cases. We can get either
of the tests we are adding to fail independently if we only make
a single corresponding change.
Fixes#73888
Updates #71359
Change-Id: Ifd91f445cc168ab895cc27f7964a6557d5cc32e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/676517
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
This adds a test which validates the traces generated by the execution
tracer and the flight recorder depending on the order where they are
stopped and started. This test uncovered that under certain
circumstances, the traces which were produced would possibly be
missing the trace header. All traces have the trace headers included
now. Clock snapshot checks have been disabled for Windows and WASM.
Change-Id: I5be719d228300469891fc56817fbce4ba5453fff
Reviewed-on: https://go-review.googlesource.com/c/go/+/675975
Auto-Submit: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Before this change, go get didn't have support for the work pattern. The
work pattern is new in Go 1.25 and evaluates to the packages in the work
(also called main) modules. 'go get work' would cause a panic because
'work' would be incorrectly considered a path pattern and then queryPath
would would try to query a metapackage pattern (resulting in the
internal error panic). This change properly supports the work pattern in
go get.
It's pretty simple: First, we need to seprate the work pattern from the
other patterns. Then in performWorkQueries, which maps queries to the
modules that satisfy them, we return the single main module because by
definition the work pattern is the set of packages in the work modules,
and go get always runs in single module mode. (The exception is when the
work module contains no packages, in which case we report a warning, and
return no candidates because nothing is needed to resolve nothing).
The rest of the work is already done by loading the packages matching
the query and finding missing imports in the call to
findAndUpgradeImports in runGet.
Change-Id: I3c4610878b3d930a1d106cc59d9a0be194d966cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/675895
Reviewed-by: Michael Matloob <matloob@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
cleanupQueue.Flush is reachable from mallocgc via sweepAssist. Normally
allp will continue all valid Ps, but procresize itself increases the
size of allp and then allocates new Ps to place in allp. If we get
perfectly unlucky, the new(p) allocations will complete sweeping and
cleanupQueue.Flush will dereference a nil pointer from allp. Avoid this
by skipping nil Ps.
I've looked through every other use of allp and none of them appear to
be reachable from procresize.
Change-Id: I6a6a636cab49ef268eb8fcd9ff9a96790d9c5685
Reviewed-on: https://go-review.googlesource.com/c/go/+/676515
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
CL 585399 fixed an initialization loop during IR contruction that
involving alias type, by avoiding publishing alias declarations until
the RHS type expression has been constructed.
There's an assertion to ensure that the alias's type must be the same
during the initialization. However, that assertion is too strict, since
we may construct different instances of the same type, if the type is an
instantination of generic type.
To fix this, we could use types.IdenticalStrict to ensure that these
types matching exactly.
Updates #66873.
Updates #73309.
Change-Id: I2559bed37e21615854333fb1057d7349406e6a1b
Reviewed-on: https://go-review.googlesource.com/c/go/+/668175
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
When transforming for loop variables, the compiler does roughly
following steps:
(1) prebody = {z := z' for z in leaked}
...
(4) init' = (init : s/z/z' for z in leaked)
However, the definition of z is not updated to `z := z'` statement,
causing ReassignOracle incorrectly use the new init statement with z'
instead of z, trigger the ICE.
Fixing this by updating the correct/new definition statement for z
during the prebody initialization.
Fixes#73823
Change-Id: Ice2a6741be7478506c58f4000f591d5582029136
Reviewed-on: https://go-review.googlesource.com/c/go/+/675475
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: David Chase <drchase@google.com>
unique.Make always copies strings passed into it, so it's safe to not
copy byte slices converted to strings either. Handle this just like map
accesses with string(b) as keys.
This CL only handles unique.Make(string(b)), not nested cases like
unique.Make([2]string{string(b1), string(b2)}); this could be done in a
followup CL but the map lookup code in walk is sufficiently different
than the call handling code that I didn't attempt it. (SSA is much
easier).
Fixes#71926
Change-Id: Ic2f82f2f91963d563b4ddb1282bd49fc40da8b85
Reviewed-on: https://go-review.googlesource.com/c/go/+/672135
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently, hash/maphash.Comparable escapes its parameter if it
contains non-string pointers, but does not escape strings or types
that contain strings but no other pointers. This is achieved by a
compiler intrinsic.
unique.Make does something similar: it stores its parameter to a
central map, with strings cloned. So from the escape analysis's
perspective, the non-string pointers are passed through, whereas
string pointers are not. We currently cannot model this type of
type-dependent data flow directly in Go. So we do this with a
compiler intrinsic. In fact, we can unify this and the intrinsic
above.
Tests are from Jake Bailey's CL 671955 (thanks!).
Fixes#73680.
Change-Id: Ia6a78e09dee39f8d9198a16758e4b5322ee2c56a
Reviewed-on: https://go-review.googlesource.com/c/go/+/675156
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Jake Bailey <jacob.b.bailey@gmail.com>
This was the first CL in a series of CLs aimed at reducing
how often interface arguments escape for the print functions in fmt.
This CL makes some small improvements to the escape analysis logging.
Here is a sample snippet of the current -m=2 logs:
./print.go:587:7: parameter p leaks to {heap} with derefs=0:
./print.go:587:7: flow: p = p:
./print.go:587:7: from (*pp).printArg(p, err, 'v') (call parameter) at ./print.go:613:13
./print.go:587:7: flow: p = p:
./print.go:587:7: from (*pp).handleMethods(p, verb) (call parameter) at ./print.go:749:22
[..]
If we attempt to tease apart some reasons why the -m=2 logs can be
challenging to understand for the uninitiated:
- The "flow" lines are very useful, but contain more-or-less abstracted
pseudocode. The "from" lines most often use actual code. When first
looking at the logs, that distinction might not be apparent, which can
result in looking back to the original code to hunt for pseudocode
that doesn't exist there. (The log example shows 'p = p', but there is
no 'p = p' in the original source).
- Escape analysis can be most interesting with inlining, but that can
result in seeing overlapping short variable names (e.g., p, b, v...).
- The directionality of the "flow" lines might not be obvious,
including whether they build top-to-bottom or bottom-to-top.
- The use of '{' and '}' in the -m=2 logs somewhat intersects with Go
literals (e.g., if the log says "{temp}", an initial thought might
be that represents some temp inside of some Go literal).
- And of course, escape analysis itself is subtle.
This CL:
- Adds the function name to the first -m=2 line to provide more context
and reduce how often the reader needs to lookup line numbers.
- Uses the Unicode left arrow '←' rather than '=' on the flow lines
to make it clearer that these lines are abstracted away from the
original Go code and to help the directionality jump out.
In the future, we can consider changing "{heap}", "{temp}",
"{storage for foo}" to something else, but we leave them as is for now.
Two examples with the modifications:
./f1.go:3:9: parameter inptr leaks to outptr for func1 with derefs=0:
./f1.go:3:9: flow: localptr ← inptr:
./f1.go:3:9: from localptr := inptr (assign) at ./f1.go:4:11
./f1.go:3:9: flow: outptr ← localptr:
./f1.go:3:9: from return localptr (return) at ./f1.go:5:2
./b.go:14:20: []byte{...} escapes to heap in byteOrderExample:
./b.go:14:20: flow: b ← &{storage for []byte{...}}:
./b.go:14:20: from []byte{...} (spill) at ./byteorder.go:14:20
./b.go:14:20: from b := []byte{...} (assign) at ./byteorder.go:14:11
./b.go:14:20: flow: <heap> ← b:
./b.go:14:20: from byteOrder.Uint32(b) (call parameter) at ./byteorder.go:15:32
These changes only affect the -m=2 output and leave the -m=1 output
as is.
Updates #8618
Updates #62653
Change-Id: Ic082a371c3d3fa0d8fd8bfbe4d64ec3e1e53c173
Reviewed-on: https://go-review.googlesource.com/c/go/+/524937
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
This change adds the flight recorder to the trace package.
Flight recording is a technique in which trace data is kept
in a circular buffer and can be flushed upon request. The
implementation will be added in follow-up CLs.
The flight recorder has already been implemented inside of the
golang.org/x/exp/trace package. This copies the current implementation
and modifies it to work within the runtime/trace package.
The changes include:
This adds the ability for multiple consumers (both the execution
tracer and the flight recorder) to subscribe to tracing events. This
change allows us to add multiple consumers without making major
modifications to the runtime. Future optimizations are planned
for this functionality.
This removes the use of byte readers from the process that
parses and processes the trace batches.
This modifies the flight recorder to not parse out the trace
clock frequency, since that requires knowledge of the format that's
unfortunate to encode in yet another place. Right now, the trace clock
frequency is considered stable for the lifetime of the program, so just
grab it directly from the runtime.
This change adds an in-band end-of-generation signal to the internal
implementation of runtime.ReadTrace. The internal implementation is
exported via linkname to runtime/trace, so the flight recorder can
identify exactly when a generation has ended. This signal is also useful
for ensuring that subscribers to runtime trace data always see complete
generations, by starting or stopping data streaming only at generation
boundaries.
For #63185
Change-Id: I5c15345981a6bbe9764a3d623448237e983c64ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/673116
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
When a TLS 1.3 client processes the server's encryptedExtensionsMsg it
should reject instances that contain duplicate extension types.
RFC 8446 §4.2 says:
There MUST NOT be more than one extension of the same type in a given
extension block.
This update matches enforcement done in the client hello unmarshalling,
but applied to the TLS 1.3 encrypted extensions message unmarshalling.
Making this change also allows enabling the
DuplicateExtensionClient-TLS-TLS13 BoGo test.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5d2
Reviewed-on: https://go-review.googlesource.com/c/go/+/673757
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Previously if instances of the handshakeMessage interface returned false
from unmarshal(), indicating an umarshalling error, the crypto/tls
package would emit an unexpected_message alert. This commit changes to
use a decode_error alert for this condition instead.
The usage-pattern of the handshakeMessage interface is that we switch on
the message type, invoke a specific concrete handshakeMessage type's
unmarshal function, and then return it to the caller on success. At this
point the caller looks at the message type and can determine if the
message was unexpected or not. If it was unexpected, the call-sites emit
the correct error for that case. Only the caller knows the current
protocol state and allowed message types, not the generic handshake
decoding logic.
With the above in mind, if we find that within the unmarshal logic for
a specific message type that the data we have in hand doesn't match the
protocol syntax we should emit a decode_error. An unexpected_message
error isn't appropriate because we don't yet know if the message is
unexpected or not, only that the message can't be decoded based on the
spec's syntax for the type the message claimed to be.
Notably one unit test, TestQUICPostHandshakeKeyUpdate, had to have its
test data adjusted because it was previously not testing the right
thing: it was double-encoding the type & length prefix data for a key
update message and expecting the QUIC logic to reject it as an
inappropriate post-handshake message. In reality it was being rejected
sooner as an invalid key update message from the double-encoding and
this was masked by the previous alert for this condition matching the
expected alert.
Finally, changing our alert allows enabling a handful of BoGo tests
related to duplicate extensions of the form
"DuplicateExtension[Server|Client]-TLS-[TLS1|TLS11|TLS12|TLS13]". One
test remains skipped (DuplicateExtensionClient-TLS-TLS13), as it
requires additional follow-up.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/673738
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
This implements RFC 9155 by removing support for SHA-1 algorithms:
- we don't advertise them in ClientHello and CertificateRequest
(where supportedSignatureAlgorithms is used directly)
- we don't select them in our ServerKeyExchange and CertificateVerify
(where supportedSignatureAlgorithms filters signatureSchemesForCertificate)
- we reject them in the peer's ServerKeyExchange and CertificateVerify
(where we check against the algorithms we advertised in ClientHello
and CertificateRequest)
Fixes#72883
Change-Id: I6a6a4656e2aafd2c38cdd32090d3d8a9a8047818
Reviewed-on: https://go-review.googlesource.com/c/go/+/658216
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
WORD and BYTE usage in crypto assembly cores is an anti-pattern which
makes extremely sensitive code significantly harder to understand, and
can result in unexpected behavior.
Because of this, we've decided to ban their usage in the crypto/ tree
(as part of the cryptography assembly policy).
This test walks the crypto/ tree looking for assembly files (those with
the filetype .s) and look for lines that match the regular rexpression
"(^|;)\s(BYTE|WORD)\s".
Change-Id: I60b5283e05e8588fa53273904a9611a411741f72
Reviewed-on: https://go-review.googlesource.com/c/go/+/671099
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Auto-Submit: Roland Shoemaker <roland@golang.org>
If a ClientHello only supports TLS 1.3, or if a CertificateRequest is
sent after selecting TLS 1.3, we should not advertise TLS 1.2-only
signature_algorithms like PKCS#1 v1.5 or SHA-1.
However, since crypto/x509 still supports PKCS#1 v1.5, and a direct
CertPool match might not care about the signature in the certificate at
all, start sending a separate signature_algorithms_cert extension to
indicate support for PKCS#1 v1.5 and SHA-1 in certificates.
We were already correctly rejecting these algorithms if the peer
selected them in a TLS 1.3 connection.
Updates #72883
Change-Id: I6a6a4656ab60e1b7fb20fdedc32604dc156953ae
Reviewed-on: https://go-review.googlesource.com/c/go/+/658215
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Go 1.25 will require macOS 12 Monterey or later, so macOS 11 will be
unsupported. The comment here suggests using a supported macOS version,
and that it can be the most recent one.
For now, make a minimal change of going from 11.0.0 to 12.0.0 so that
the chosen version is a supported one (although not the most recent).
However, it looks like even in CL 460476 (where the comment was added)
we were staying with the macOS version that matched Go's oldest, so we
might not have have recent experience with going beyond that. Update
the comment accordingly.
For #69839.
Change-Id: I90908971b0d5a8235ce77dc6bc9649e86008270a
Cq-Include-Trybots: luci.golang.try:gotip-darwin-amd64-longtest,gotip-darwin-arm64-longtest,gotip-darwin-amd64_12,gotip-darwin-amd64_14,gotip-darwin-arm64_12,gotip-darwin-arm64_15
Reviewed-on: https://go-review.googlesource.com/c/go/+/675095
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Allow skipping the deprecation check when GOPROXY=off. The deprecation
check is an informational message so this doesn't affect the success of
the command. We should probably skip the check in more cases when
GOPROXY=off but that's a bigger change that should be made in a later
release.
There are still some deps.dev log messages that we should try to
suppress.
For #68106
Change-Id: Ifa0efd01ed623bb68c7ad7c5cfb6705547d157a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/675155
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Matloob <matloob@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Just like we do in cmd/doc when we start pkgsite, ignore SIGINT (and
SIGQUIT on unix) when we start cmd/doc so that it's handled by cmd/doc
(if pkgsite is not started, and before it is started) or pkgsite, if it
is started. Also exit with the exit status of the command, rather than
using base.Errorf so that we don't print an extra error message to the
terminal.
For #68106
Change-Id: If968e88b95031761432d13dc47c5febe3391945d
Reviewed-on: https://go-review.googlesource.com/c/go/+/675076
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Auto-Submit: Michael Matloob <matloob@google.com>
This is part of a series of CLs that aim to help allocations
in reflect and reduce how often interface arguments escape
for the print functions in fmt.
Before this change, the reflect.Value parameter for packEface leaks
immediately to the heap due to the various ODOTPTR operations on the
*emptyInterface. The -m=2 logs report:
parameter v leaks to <heap> for packEface with derefs=0:
flow: <heap> ← v:
from v.ptr (dot) at .\value.go:145:13
from e.word = v.ptr (assign) at .\value.go:145:10
After this change, the input leaks to the result, which is what
we want:
parameter v leaks to ~r0 with derefs=0:
flow: e = v:
from v.ptr (dot) at .\value.go:143:13
from e.Data = v.ptr (assign) at .\value.go:143:10
flow: ~r0 = e:
from &e (address-of) at .\value.go:147:32
from *(*any)(unsafe.Pointer(&e)) (indirection) at .\value.go:147:9
from return *(*any)(unsafe.Pointer(&e)) (return) at .\value.go:147:2
This change here is needed, but reflect.Value.Interface still leaks its
input to the heap for other reasons having to do with method values,
which we attempt to address in CL 530097, CL 530095, and CL 530096.
Updates #8618
Updates #71349
Change-Id: Ie77bc850ff261212eeafe190bd6f9a879676a51d
Reviewed-on: https://go-review.googlesource.com/c/go/+/528535
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: qiu laidongfeng2 <2645477756@qq.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Updates the skip reason for the following BoGo tests:
* TLS-ECH-Client-TLS12SessionID
* SupportTicketsWithSessionID
* ResumeTLS12SessionID-TLS13
The crypto/tls package does not support session ID based resumption at
this time, and so any tests that rely on this support need to be
skipped.
Updates #72006
Updates #25228
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5d0
Reviewed-on: https://go-review.googlesource.com/c/go/+/673737
TryBot-Bypass: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
The crypto/tls package produces the expected error for this test case,
and so it can be enabled.
Looking at the history of the relevant code it appears the TLS 1.3
implementation has always had the correct behaviour for HRR changing to
an unsupported group after the initial hello.
I think this test was skipped initially because at the time of
initial BoGo config commit we hadn't implemented the -curves argument
for the test shim yet, and this test relies on it. We later added
support for that flag alongside X25519Kyber768Draft00 KX and I think we
missed the chance to enable the test then.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/673756
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Bypass: Daniel McCarney <daniel@binaryparadox.net>
When a pre-TLS 1.3 server processes a client hello message that
indicates compression methods that don't include the null compression
method, send an illegal parameter alert.
Previously we did this for TLS 1.3 server handshakes only, and the
legacy TLS versions used alertHandshakeFailure for this circumstance. By
switching this to alertIllegalParameter we use a consistent alert across
all TLS versions, and can also enable the NoNullCompression-TLS12 BoGo
test we were skipping.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/673736
TryBot-Bypass: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: David Chase <drchase@google.com>
When a crypto/tls client using TLS < 1.3 sends supported elliptic_curves
in a client hello message the server must limit itself to choosing one
of the supported options from our message. If we process a server key
exchange message that chooses an unadvertised curve, abort the
handshake w/ an error.
Previously we would not note that the server chose a curve we didn't
include in the client hello message, and would proceed with the
handshake as long as the chosen curve was one that we've implemented.
However, RFC 8422 5.1 makes it clear this is a server acting
out-of-spec, as it says:
If a server does not understand the Supported Elliptic Curves
Extension, does not understand the Supported Point Formats Extension,
or is unable to complete the ECC handshake while restricting itself
to the enumerated curves and point formats, it MUST NOT negotiate the
use of an ECC cipher suite.
Changing our behaviour to enforce this also allows enabling the
UnsupportedCurve BoGo test.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/673735
TryBot-Bypass: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Our prior CL 649078 teaches the compiler to use a pointer to
runtime.zeroVal as the data pointer for an interface in cases it where
it can see that a zero value struct or array is being used in
an interface conversion.
This applies to some uses with reflect, such as:
s := S{}
v := reflect.ValueOf(s)
This CL builds on that to do a cheap pointer check in reflect.IsZero
to see if the Value points to runtime.zeroVal, which means it is a zero
value.
An alternative might be to do an initial pointer check in the typ.Equal
function for types where it makes sense to do but doesn't already.
This CL gives a performance boost of -51.71% geomean for
BenchmarkZero/IsZero, with most of the impact there on
arrays of structs. (The left column is CL 649078 and the right column
is this CL).
goos: linux
goarch: amd64
pkg: reflect
cpu: Intel(R) Xeon(R) CPU @ 2.80GHz
│ find-zeroVal │ check-zeroVal │
│ sec/op │ sec/op vs base │
Zero/IsZero/ByteArray/size=16-4 4.171n ± 0% 3.123n ± 0% -25.13% (p=0.000 n=20)
Zero/IsZero/ByteArray/size=64-4 3.864n ± 0% 3.129n ± 0% -19.02% (p=0.000 n=20)
Zero/IsZero/ByteArray/size=1024-4 3.878n ± 0% 3.126n ± 0% -19.39% (p=0.000 n=20)
Zero/IsZero/BigStruct/size=1024-4 5.061n ± 0% 3.273n ± 0% -35.34% (p=0.000 n=20)
Zero/IsZero/SmallStruct/size=16-4 4.191n ± 0% 3.275n ± 0% -21.87% (p=0.000 n=20)
Zero/IsZero/SmallStructArray/size=64-4 8.636n ± 0% 3.127n ± 0% -63.79% (p=0.000 n=20)
Zero/IsZero/SmallStructArray/size=1024-4 80.055n ± 0% 3.126n ± 0% -96.10% (p=0.000 n=20)
Zero/IsZero/Time/size=24-4 3.865n ± 0% 3.274n ± 0% -15.29% (p=0.000 n=20)
geomean 6.587n 3.181n -51.71%
Note these are of course micro benchmarks with easily predicted
branches. The extra branch we introduce in the CL might hurt if there
was for example a tight loop where 50% of the values used the
global zeroVal and 50% didn't in a way that is not well predicted,
although if the typ.Equal for many types already does an initial
pointer check, it might not matter much.
For the older BenchmarkIsZero in reflect, this change does not help.
(The compiler does not use the global zeroVal as the data word for the
interfaces in this benchmark because values are part of a larger value
that is too big to be used in the global zeroVal, and also a piece of
the larger value is mutated and is not zero).
│ find-zeroVal │ check-zeroVal │
│ sec/op │ sec/op vs base │
IsZero/ArrayComparable-4 14.58n ± 0% 14.59n ± 0% ~ (p=0.177 n=20)
IsZero/ArrayIncomparable-4 163.8n ± 0% 167.5n ± 0% +2.26% (p=0.000 n=20)
IsZero/StructComparable-4 6.847n ± 0% 6.847n ± 0% ~ (p=0.703 n=20)
IsZero/StructIncomparable-4 35.41n ± 0% 35.10n ± 0% -0.86% (p=0.000 n=20)
IsZero/ArrayInt_4-4 8.631n ± 0% 8.363n ± 0% -3.10% (p=0.000 n=20)
IsZero/ArrayInt_1024-4 265.5n ± 0% 265.4n ± 0% ~ (p=0.288 n=20)
IsZero/ArrayInt_1024_NoZero-4 135.8n ± 0% 136.2n ± 0% +0.33% (p=0.000 n=20)
IsZero/Struct4Int-4 8.451n ± 0% 8.386n ± 0% -0.77% (p=0.000 n=20)
IsZero/ArrayStruct4Int_1024-4 265.2n ± 0% 266.0n ± 0% +0.30% (p=0.000 n=20)
IsZero/ArrayChanInt_1024-4 265.5n ± 0% 265.4n ± 0% ~ (p=0.605 n=20)
IsZero/StructInt_512-4 135.8n ± 0% 135.8n ± 0% ~ (p=0.396 n=20)
geomean 55.22n 55.12n -0.18%
Updates #71323
Change-Id: Ie083853a5bff03856277a293d94532a681f4a8d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/654135
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
This is a small-ish adjustment to the change earlier in our
stack in CL 649555, which started creating read-only global storage
for a composite literal used in an interface conversion and setting
the interface data pointer to point to that global storage.
In some cases, there are execution-time performance benefits to point
to runtime.zeroVal in particular. In reflect, pointer checks against
the runtime.zeroVal memory address are used to side-step some work,
such as in reflect.Value.Set and reflect.Value.IsZero.
In this CL, we therefore dig up the zeroVal symbol, and we use the
machinery from earlier in our stack to use a pointer to zeroVal for
the interface data pointer if we see examples like:
sink = S{}
or:
s := S{}
sink = s
CL 649076 (also earlier in our stack) added most of the tests
along with debug diagnostics in convert.go to make it easier
to test this change.
We add a benchmark in reflect to show examples of performance benefit.
The left column is our immediately prior CL 649555, and the right is
this CL. (The arrays of structs here do not seem to benefit, which
we attempt to address in our next CL).
goos: linux
goarch: amd64
pkg: reflect
cpu: Intel(R) Xeon(R) CPU @ 2.80GHz
│ cl-649555 │ new │
│ sec/op │ sec/op vs base │
Zero/IsZero/ByteArray/size=16-4 4.176n ± 0% 4.171n ± 0% ~ (p=0.151 n=20)
Zero/IsZero/ByteArray/size=64-4 6.921n ± 0% 3.864n ± 0% -44.16% (p=0.000 n=20)
Zero/IsZero/ByteArray/size=1024-4 21.210n ± 0% 3.878n ± 0% -81.72% (p=0.000 n=20)
Zero/IsZero/BigStruct/size=1024-4 25.505n ± 0% 5.061n ± 0% -80.15% (p=0.000 n=20)
Zero/IsZero/SmallStruct/size=16-4 4.188n ± 0% 4.191n ± 0% ~ (p=0.106 n=20)
Zero/IsZero/SmallStructArray/size=64-4 8.639n ± 0% 8.636n ± 0% ~ (p=0.973 n=20)
Zero/IsZero/SmallStructArray/size=1024-4 79.99n ± 0% 80.06n ± 0% ~ (p=0.213 n=20)
Zero/IsZero/Time/size=24-4 7.232n ± 0% 3.865n ± 0% -46.56% (p=0.000 n=20)
Zero/SetZero/ByteArray/size=16-4 13.47n ± 0% 13.09n ± 0% -2.78% (p=0.000 n=20)
Zero/SetZero/ByteArray/size=64-4 14.14n ± 0% 13.70n ± 0% -3.15% (p=0.000 n=20)
Zero/SetZero/ByteArray/size=1024-4 24.22n ± 0% 20.18n ± 0% -16.68% (p=0.000 n=20)
Zero/SetZero/BigStruct/size=1024-4 24.24n ± 0% 20.18n ± 0% -16.73% (p=0.000 n=20)
Zero/SetZero/SmallStruct/size=16-4 13.45n ± 0% 13.10n ± 0% -2.60% (p=0.000 n=20)
Zero/SetZero/SmallStructArray/size=64-4 14.12n ± 0% 13.69n ± 0% -3.05% (p=0.000 n=20)
Zero/SetZero/SmallStructArray/size=1024-4 24.62n ± 0% 21.61n ± 0% -12.26% (p=0.000 n=20)
Zero/SetZero/Time/size=24-4 13.59n ± 0% 13.40n ± 0% -1.40% (p=0.000 n=20)
geomean 14.06n 10.19n -27.54%
Finally, here are results from the benchmark example from #71323.
Note however that almost all the benefit shown here is from our earlier
CL 649555, which is a more general purpose change and eliminates
the allocation using a different read-only global than this CL.
│ go1.24 │ new │
│ sec/op │ sec/op vs base │
InterfaceAny 112.6000n ± 5% 0.8078n ± 3% -99.28% (p=0.000 n=20)
ReflectValue 11.63n ± 2% 11.59n ± 0% ~ (p=0.330 n=20)
│ go1.24.out │ new.out │
│ B/op │ B/op vs base │
InterfaceAny 224.0 ± 0% 0.0 ± 0% -100.00% (p=0.000 n=20)
ReflectValue 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=20) ¹
│ go1.24.out │ new.out │
│ allocs/op │ allocs/op vs base │
InterfaceAny 1.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=20)
ReflectValue 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=20) ¹
Updates #71359
Updates #71323
Change-Id: I64d8cf1a7900f011d2ec59b948388aeda1150676
Reviewed-on: https://go-review.googlesource.com/c/go/+/649078
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: David Chase <drchase@google.com>
Today, this interface conversion causes the struct literal
to be heap allocated:
var sink any
func example1() {
sink = S{1, 1}
}
For basic literals like integers that are directly used in
an interface conversion that would otherwise allocate, the compiler
is able to use read-only global storage (see #18704).
This CL extends that to struct and array literals as well by creating
read-only global storage that is able to represent for example S{1, 1},
and then using a pointer to that storage in the interface
when the interface conversion happens.
A more challenging example is:
func example2() {
v := S{1, 1}
sink = v
}
In this case, the struct literal is not directly part of the
interface conversion, but is instead assigned to a local variable.
To still avoid heap allocation in cases like this, in walk we
construct a cache that maps from expressions used in interface
conversions to earlier expressions that can be used to represent the
same value (via ir.ReassignOracle.StaticValue). This is somewhat
analogous to how we avoided heap allocation for basic literals in
CL 649077 earlier in our stack, though here we also need to do a
little more work to create the read-only global.
CL 649076 (also earlier in our stack) added most of the tests
along with debug diagnostics in convert.go to make it easier
to test this change.
See the writeup in #71359 for details.
Fixes#71359Fixes#71323
Updates #62653
Updates #53465
Updates #8618
Change-Id: I8924f0c69ff738ea33439bd6af7b4066af493b90
Reviewed-on: https://go-review.googlesource.com/c/go/+/649555
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Previously the common Config.mutualVersion() code prioritized the
selected version based on the provided peerVersions being sent in peer
preference order.
Instead we would prefer to see TLS 1.3 used whenever it is
supported, even if the peer would prefer an older protocol version.
This commit updates mutualVersions() to implement this policy change.
Our new behaviour matches the behaviour of other TLS stacks, notably
BoringSSL, and so also allows enabling the IgnoreClientVersionOrder BoGo
test that we otherwise must skip.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/673236
Auto-Submit: Daniel McCarney <daniel@binaryparadox.net>
TryBot-Bypass: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Drop the entire pre-AVX2 assembly implementation of SHA-1, SHA-256, and
SHA-512. This also technically impacts the SHA-1 AVX2 implementation,
since it previously called the pre-AVX2 implementation for the last
block if the number of blocks wasn't a multiple of 2.
Instead of keeping the entire implementation just for that case, we
just call the generic implementation for the last block. This will be a
little slower, but still seems like a win.
Updates #69587
Change-Id: Id5234c42910d8c6ec6b8df700a721c0953dff02b
Reviewed-on: https://go-review.googlesource.com/c/go/+/657716
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Currently, the integer value in the following interface conversion gets
heap allocated:
v := 1000
fmt.Println(v)
In contrast, this conversion does not currently cause the integer value
to be heap allocated:
fmt.Println(1000)
The second example is able to avoid heap allocation because of an
optimization in walk (by Josh in #18704 and related issues) that
recognizes a literal is being used. In the first example, that
optimization is currently thwarted by the literal getting assigned
to a local variable prior to use in the interface conversion.
This CL propagates constants to interface conversions like
in the first example to avoid heap allocations, instead using
a read-only global. The net effect is roughly turning the first example
into the second.
One place this comes up in practice currently is with logging or
debug prints. For example, if we have something like:
func conditionalDebugf(format string, args ...interface{}) {
if debugEnabled {
fmt.Fprintf(io.Discard, format, args...)
}
}
Prior to this CL, this integer is heap allocated, even when the
debugEnabled flag is false, and even when the compiler
inlines conditionalDebugf:
v := 1000
conditionalDebugf("hello %d", v)
With this CL, the integer here is no longer heap allocated, even when
the debugEnabled flag is enabled, because the compiler can now see that
it can use a read-only global.
See the writeup in #71359 for more details.
CL 649076 (earlier in our stack) added most of the tests
along with debug diagnostics in convert.go to make it easier
to test this change.
Updates #71359
Updates #62653
Updates #53465
Updates #8618
Change-Id: I19a51e74b36576ebb0b9cf599267cbd2bd847ce4
Reviewed-on: https://go-review.googlesource.com/c/go/+/649079
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Using the new-ish ir.ReassignOracle is more efficient than calling
ir.StaticValue repeatedly.
This CL now uses an ir.ReassignOracle for the recent
make constant propagation introduced in CL 649035.
We also pull the main change from CL 649035 into a new function,
which we will update later in our stack. We will also use the
ReassignOracles introduced here later in our stack.
(We originally did most of this work in CL 649077, but we abandoned
that in favor of CL 649035).
We could also use an ir.ReassignOracle in the older processing of
ir.OCALLFUNC in (*escape).call, but for now, we just leave that
as a TODO.
Updates #71359
Change-Id: I6e02eeac269bde3a302622b4dfe0c8dc63ec9ffc
Reviewed-on: https://go-review.googlesource.com/c/go/+/673795
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Uses the new weak package to replace the existing custom intern cache
with a map of weak.Pointers instead. This simplifies the cache, and
means we don't need to store a slice of handles on the Conn anymore.
Change-Id: I5c2bf6ef35fac4255e140e184f4e48574b34174c
Reviewed-on: https://go-review.googlesource.com/c/go/+/644176
TryBot-Bypass: Roland Shoemaker <roland@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Roland Shoemaker <roland@golang.org>
This CL adds two related features enabled by default via compatibility
GODEBUGs containermaxprocs and updatemaxprocs.
On Linux, containermaxprocs makes the Go runtime consider cgroup CPU
bandwidth limits (quota/period) when setting GOMAXPROCS. If the cgroup
limit is lower than the number of logical CPUs available, then the
cgroup limit takes precedence.
On all OSes, updatemaxprocs makes the Go runtime periodically
recalculate the default GOMAXPROCS value and update GOMAXPROCS if it has
changed. If GOMAXPROCS is set manually, this update does not occur. This
is intended primarily to detect changes to cgroup limits, but it applies
on all OSes because the CPU affinity mask can change as well.
The runtime only considers the limit in the leaf cgroup (the one that
actually contains the process), caching the CPU limit file
descriptor(s), which are periodically reread for updates. This is a
small departure from the original proposed design. It will not consider
limits of parent cgroups (which may be lower than the leaf), and it will
not detection cgroup migration after process start.
We can consider changing this in the future, but the simpler approach is
less invasive; less risk to packages that have some awareness of runtime
internals. e.g., if the runtime periodically opens new files during
execution, file descriptor leak detection is difficult to implement in a
stable way.
For #73193.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Change-Id: I6a6a636c631c1ae577fb8254960377ba91c5dc98
Reviewed-on: https://go-review.googlesource.com/c/go/+/670497
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Add build tag gated Valgrind annotations to the runtime which let it
understand how the runtime manages memory. This allows for Go binaries
to be run under Valgrind without emitting spurious errors.
Instead of adding the Valgrind headers to the tree, and using cgo to
call the various Valgrind client request macros, we just add an assembly
function which emits the necessary instructions to trigger client
requests.
In particular we add instrumentation of the memory allocator, using a
two-level mempool structure (as described in the Valgrind manual [0]).
We also add annotations which allow Valgrind to track which memory we
use for stacks, which seems necessary to let it properly function.
We describe the memory model to Valgrind as follows: we treat heap
arenas as a "pool" created with VALGRIND_CREATE_MEMPOOL_EXT (so that we
can use VALGRIND_MEMPOOL_METAPOOL and VALGRIND_MEMPOOL_AUTO_FREE).
Within the pool we treat spans as "superblocks", annotated with
VALGRIND_MEMPOOL_ALLOC. We then allocate individual objects within spans
with VALGRIND_MALLOCLIKE_BLOCK.
It should be noted that running binaries under Valgrind can be _quite
slow_, and certain operations, such as running the GC, can be _very
slow_. It is recommended to run programs with GOGC=off. Additionally,
async preemption should be turned off, since it'll cause strange
behavior (GODEBUG=asyncpreemptoff=1).
Running Valgrind with --leak-check=yes will result in some errors
resulting from some things not being marked fully free'd. These likely
need more annotations to rectify, but for now it is recommended to run
with --leak-check=off.
Updates #73602
[0] https://valgrind.org/docs/manual/mc-manual.html#mc-manual.mempools
Change-Id: I71b26c47d7084de71ef1e03947ef6b1cc6d38301
Reviewed-on: https://go-review.googlesource.com/c/go/+/674077
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
go doc tries to find a package to display documentation for. In the case
that no package is provided, it uses "." just like go list does. So if
go doc -http is run without any arguments, it tries to show the
documentation for the package in the current directory. As a special
case, if no arguments are provided, allow no package to match the
current directory and just open the root pkgsite page.
For #68106
Change-Id: I6d65b160a838591db953fac630eced6b09106877
Reviewed-on: https://go-review.googlesource.com/c/go/+/675075
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently the checkfinalizers test (TestDetectCleanupOrFinalizerLeak)
only *tries* to ensure the tiny alloc with a cleanup attached shares a
block with other objects. However, what it does is insufficient, because
it could get unlucky and have the last object allocated be the first
object of a new block.
This change changes the test to guarantee that a tiny object is not at
the start of a fresh block by looking at the alignment of the object's
pointer. If the object's pointer is odd, then that's good enough to know
that it shares a block with something else, since the blocks themselves
are aligned to a much higher power of two.
This fixes a failure I've seen on the builders.
Fixes#73810.
Change-Id: Ieafdbb9cccb0d2dc3659a9a5d9d9233718461635
Reviewed-on: https://go-review.googlesource.com/c/go/+/674655
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
The types being hoisted are those which cannot be referenced; that is,
where Ref[T] is illegal. These are most clearly owned by pkgbits. The
types which follow are those which can be referenced.
Referenceable types are more hazy due to the reference mechanism of UIR
- sections. These are a detail of the UIR file format and are surfaced
directly to importers.
I suspect that pkgbits would benefit from a reference mechanism not
dependent on sections. This would permit us to push down many types
from the noder into pkgbits, reducing the interface surface without
giving up deduplication.
Change-Id: Ifaf5cd9de20c767ad0941413385b308d628aac6c
Reviewed-on: https://go-review.googlesource.com/c/go/+/674635
Auto-Submit: Mark Freeman <mark@golang.org>
TryBot-Bypass: Mark Freeman <mark@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
Add generator tests that verify the timestamps for the sync events
emitted in the go1.25 trace format and earlier versions.
Add the ability to configure the properties of the per-generation sync
batches in testgen. Also refactor testgen to produce more realistic
timestamps by keeping track of lastTs and using it for structural
batches that don't have their own timestamps. Otherwise they default to
zero which means the minTs of the generation can't be controlled.
For #69869
Change-Id: I92a49b8281bc4169b63e13c030c1de7720cd6f26
Reviewed-on: https://go-review.googlesource.com/c/go/+/653876
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Replace the per-generation EvEventBatch containing a lone EvFrequency
event with a per-generation EvEventBatch containing a EvSync header
followed by an EvFrequency and EvClockSnapshot event.
The new EvClockSnapshot event contains trace, mono and wall clock
snapshots taken in close time proximity. Ignoring minor resolution
differences, the trace and mono clock are the same on linux, but not on
windows (which still uses a TSC based trace clock).
Emit the new sync batch at the very beginning of every new generation
rather than the end to be in harmony with the internal/trace reader
which emits a sync event at the beginning of every generation as well
and guarantees monotonically increasing event timestamps.
Bump the version of the trace file format to 1.25 since this change is
not backwards compatible.
Update the internal/trace reader implementation to decode the new
events, but do not expose them to the public reader API yet. This is
done in the next CL.
For #69869
Change-Id: I5bfedccdd23dc0adaf2401ec0970cbcc32363393
Reviewed-on: https://go-review.googlesource.com/c/go/+/653575
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
The RISC-V integer vector multiply add instructions are not encoded
correctly; the first and second arguments are swapped. For example,
the instruction
VMACCVV V1, V2, V3
encodes to
b620a1d7 or vmacc.vv v3,v1,v2
and not
b61121d7 or vmacc.vv v3,v2,v1
as expected.
This is inconsistent with the argument ordering we use for 3
argument vector instructions, in which the argument order, as given
in the RISC-V specifications, is reversed, and also with the vector
FMA instructions which have the same argument ordering as the vector
integer multiply add instructions in the "The RISC-V Instruction Set
Manual Volume I". For example, in the ISA manual we have the
following instruction definitions
; Integer multiply-add, overwrite addend
vmacc.vv vd, vs1, vs2, vm # vd[i] = +(vs1[i] * vs2[i]) + vd[i]
; FP multiply-accumulate, overwrites addend
vfmacc.vv vd, vs1, vs2, vm # vd[i] = +(vs1[i] * vs2[i]) + vd[i]
It's reasonable to expect that the Go assembler would use the same
argument ordering for both of these instructions. It currently does
not.
We fix the issue by switching the argument ordering for the vector
integer multiply add instructions to match those of the vector FMA
instructions.
Change-Id: Ib98e9999617f991969e5c831734b3bb3324439f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/670335
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
For the package github.com/microsoft/typescript-go/internal/checker,
compilation currently spends most of its time in escape analysis.
Here, we re-order work to be more efficient when analyzing many
locations, and delay visiting some locations to prioritize locations
that might be more likely to reach a terminal point of reaching the
heap and possibly reduce the count of intermediate states for each location.
Action graph reported build times show roughly a 5x improvement for
compilation of the typescript-go/internal/checker package:
go1.24.0: 91.792s
cl-657179-ps1: 17.578s
with timing via:
go build -a -debug-actiongraph=/tmp/actiongraph-cl-657179-ps1 -v github.com/microsoft/typescript-go/internal/checker
There are some additional adjustments to make here, including we can
consider a follow-on CL I have that parallelizes the operations of the
core loop, but this seems to be a nice win as is, and my understanding
is the desire is to merge this as it stands.
Updates #72815
Change-Id: I1753c5354b495b059f68fb97f3103ee7834f9eee
Reviewed-on: https://go-review.googlesource.com/c/go/+/657179
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
All private keys need to go through a slow PCT in FIPS-140 mode.
ECDH and RSA keys have places to hide a precomputed value without
causing races, but Ed25519 and ECDSA keys might be constructed by the
application and then used with concurrent Sign calls.
For these, implement an equivalent to crypto/internal/boring/bcache
using weak.Pointer and runtime.AddCleanup.
fips140: latest
goos: linux
goarch: amd64
pkg: crypto/ed25519
cpu: AMD Ryzen 7 PRO 8700GE w/ Radeon 780M Graphics
│ 1a93e4a2cf │ 78a819ea78 │
│ sec/op │ sec/op vs base │
Signing-16 72.72µ ± 0% 16.93µ ± 1% -76.72% (p=0.002 n=6)
fips140: off
goos: linux
goarch: amd64
pkg: crypto/ed25519
cpu: AMD Ryzen 7 PRO 8700GE w/ Radeon 780M Graphics
│ 310bad31e5 │ 310bad31e5-dirty │
│ sec/op │ sec/op vs base │
Signing-16 17.18µ ± 1% 16.95µ ± 1% -1.36% (p=0.002 n=6)
fips140: latest
goos: linux
goarch: amd64
pkg: crypto/ecdsa
cpu: AMD Ryzen 7 PRO 8700GE w/ Radeon 780M Graphics
│ 1a93e4a2cf │ 78a819ea78 │
│ sec/op │ sec/op vs base │
Sign/P256-16 90.97µ ± 0% 21.04µ ± 0% -76.87% (p=0.002 n=6)
Sign/P384-16 701.6µ ± 1% 142.0µ ± 0% -79.75% (p=0.002 n=6)
Sign/P521-16 2943.5µ ± 1% 491.9µ ± 0% -83.29% (p=0.002 n=6)
fips140: off
goos: linux
goarch: amd64
pkg: crypto/ecdsa
cpu: AMD Ryzen 7 PRO 8700GE w/ Radeon 780M Graphics
│ 1a93e4a2cf │ 78a819ea78 │
│ sec/op │ sec/op vs base │
Sign/P256-16 21.27µ ± 0% 21.13µ ± 0% -0.65% (p=0.002 n=6)
Sign/P384-16 143.3µ ± 0% 142.4µ ± 0% -0.63% (p=0.009 n=6)
Sign/P521-16 525.3µ ± 0% 462.1µ ± 0% -12.04% (p=0.002 n=6)
This unavoidably introduces allocations in the very first use of Ed25519
private keys, but usually that's not in the hot path.
Change-Id: I6a6a465640a5dff64edd73ee5dda5f2ad1b476b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/654096
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Replace the hchan.synctest bool with an hchan.bubble reference
to the synctest bubble that created the chan. I originally used
a bool to avoid increasing the size of hchan, but we have space
in hchan's current size class for another pointer.
This lets us detect one bubble operating on a chan created
in a different bubble.
For #67434
Change-Id: If6cf9ffcb372fe7fb3f8f4ef27b664848578ba5c
Reviewed-on: https://go-review.googlesource.com/c/go/+/674515
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
Use a sync.OnceValue rather than a sync.WaitGroup to
coordinate access to encoderCache entries.
The OnceValue better expresses the intent of the code
(we want to initialize the cache entry only once).
However, the motivation for this change is to avoid
testing/synctest incorrectly reporting a deadlock
when multiple bubbles call Marshal at the same time.
Goroutines blocked on WaitGroup.Wait are "durably blocked",
causing confusion when a goroutine in one bubble Waits
for a goroutine in a different bubble. Goroutines blocked
on OnceValue are not durably blocked, avoiding the problem.
Fixes#73733
For #67434
Change-Id: I81cddda80af67cf5c280fd4327620bc37e7a6fe6
Reviewed-on: https://go-review.googlesource.com/c/go/+/673335
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This change switches the pkgsite command invoked to start a pkgsite
server from golang.org/x/pkgsite/cmd/pkgsite to
golang.org/x/pkgsite/cmd/internal/doc. The doc command is a simplified
version of cmd/pkgsite that changes some options to improve the user
experience. For example, it limits logging informational log messages,
doesn't always expect to find modules (for example if we're outside of a
module getting documentation for the standard library), and it takes the
address of the page to open in the browser (which simplifies waiting for
the server to start listening).
Fixes#68106
Change-Id: I667a49d03823242fa1aff333ecb1c0f198e92412
Reviewed-on: https://go-review.googlesource.com/c/go/+/674158
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: David Chase <drchase@google.com>
cmd/doc uses go/build to get information about the packages it's
documenting. In some cases, go/build can return a build.Package that it
couldn't determine an import path for, in which case it sets the import
path to ".". This can happen for relative package paths in in a module:
for relative package paths we don't use the go command to get
information about the module and just open the source files directly
instead, and will be missing the import path. This is usually okay
because go doc doesn't need to print the import path of the package it's
documenting, but for go doc -http, we want to know the import path so we
can open the right page in the browser.
For #68106
Change-Id: Ifba92862ad01d8d63f531c2451f18db2b0d7a3e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/674556
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>
This fixes a bug where we start pkgsite for every requested object,
rather than the one that we would have printed the documentation for.
To make things simple, we'll run the logic that prints the
documentation, but with an io.Discard writer. Then we can tell if the
documentation was found based on the return values of those functions.
For #68106
Change-Id: Ibf2ab1720f381d7214fc9239b9c2e915c91f7f7b
Reviewed-on: https://go-review.googlesource.com/c/go/+/674555
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
This implementation is zero-alloc when T is a concrete type,
allocates when val contains a method or when T is a interface
and Value was obtained for example through Elem(), in which case
it has to be allocated to avoid sharing the same memory.
goos: linux
goarch: amd64
pkg: reflect
cpu: AMD Ryzen 5 4600G with Radeon Graphics
│ /tmp/bench2 │
│ sec/op │
TypeAssert/TypeAssert[int](int)-12 2.725n ± 1%
TypeAssert/TypeAssert[uint8](int)-12 2.599n ± 1%
TypeAssert/TypeAssert[fmt.Stringer](reflect_test.testTypeWithMethod)-12 8.470n ± 0%
TypeAssert/TypeAssert[fmt.Stringer](*reflect_test.testTypeWithMethod)-12 8.460n ± 1%
TypeAssert/TypeAssert[interface_{}](int)-12 4.181n ± 1%
TypeAssert/TypeAssert[interface_{}](reflect_test.testTypeWithMethod)-12 4.178n ± 1%
TypeAssert/TypeAssert[time.Time](time.Time)-12 2.839n ± 0%
TypeAssert/TypeAssert[func()_string](func()_string)-12 151.1n ± 1%
geomean 6.645n
│ /tmp/bench2 │
│ B/op │
TypeAssert/TypeAssert[int](int)-12 0.000 ± 0%
TypeAssert/TypeAssert[uint8](int)-12 0.000 ± 0%
TypeAssert/TypeAssert[fmt.Stringer](reflect_test.testTypeWithMethod)-12 0.000 ± 0%
TypeAssert/TypeAssert[fmt.Stringer](*reflect_test.testTypeWithMethod)-12 0.000 ± 0%
TypeAssert/TypeAssert[interface_{}](int)-12 0.000 ± 0%
TypeAssert/TypeAssert[interface_{}](reflect_test.testTypeWithMethod)-12 0.000 ± 0%
TypeAssert/TypeAssert[time.Time](time.Time)-12 0.000 ± 0%
TypeAssert/TypeAssert[func()_string](func()_string)-12 72.00 ± 0%
geomean ¹
Fixes#62121
Change-Id: I0911c70c5966672c930d387438643f94a40441c4
GitHub-Last-Rev: ce89a53097
GitHub-Pull-Request: golang/go#71639
Reviewed-on: https://go-review.googlesource.com/c/go/+/648056
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
For goroutines in a synctest bubble, include whether the goroutine
is "durably blocked" or not in the goroutine status.
Synctest categorizes goroutines in certain states as "durably"
blocked, where the goroutine is not merely idle but can only
be awoken by another goroutine in its bubble. To make it easier
for users to understand why a bubble is or is not idle,
print the state of each bubbled goroutine.
For example:
goroutine 36 [chan receive, synctest bubble 34, not durably blocked]:
goroutine 37 [chan receive (synctest), synctest bubble 34, durably blocked]:
Goroutine 36 is receiving from a channel created outside its bubble.
Goroutine 36 is receiving from a channel created inside its bubble.
For #67434
Change-Id: I006b656a9ce7eeb75b2be21e748440a5dd57ceb0
Reviewed-on: https://go-review.googlesource.com/c/go/+/670976
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Before this change, when we tried to compute the set of packages in
'all', we'd add packages with invalid import paths to the set and try to
load them, which would fail. Instead, do not add them to the list of
packages to load in the second iteration of the loader. We'll still
return errors for invalid imports in the importing packages.
Change-Id: I682229011f555ed1d0c827f79100c1c43bf7f93a
Reviewed-on: https://go-review.googlesource.com/c/go/+/673655
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This new debug mode detects cleanup/finalizer leaks using checkmark
mode. It runs a partial GC using only specials as roots. If the GC can
find a path from one of these roots back to the object the special is
attached to, then the object might never be reclaimed. (The cycle could
be broken in the future, but it's almost certainly a bug.)
This debug mode is very barebones. It contains no type information and
no stack location for where the finalizer or cleanup was created.
For #72949.
Change-Id: Ibffd64c1380b51f281950e4cfe61f677385d42a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/634599
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
When an errorcheck test uses -m and instantiates an imported generic
function, the errors will include -m messages from the imported package
(since the new function has not previously been walked). These errors
cannot be matched since we can't write errors in files outside the test
input.
To fix this (and enable the other CLs in this stack), drop any unmatched
errors that occur in files outside those in the input set.
Change-Id: I2fcf0dd4693125d2e5823ea4437011730d8b1b1f
Reviewed-on: https://go-review.googlesource.com/c/go/+/672515
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
This adds additional logging for the work that walk does to reduce
how often an interface conversion results in an allocation.
Also, as part of #71359, we will be updating how escape analysis and
walk handle basic literals, composite literals, and zero values,
so add some tests that uses this new logging.
By the end of our CL stack, we address all of these tests.
Updates #71359
Change-Id: I43fde8343d9aacaec1e05360417908014a86c8bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/649076
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
CL 614257 refactored mallocgc but lost an optimization: if a span for a
large object is already backed by memory fresh from the OS (and thus
zeroed), we don't need to zero it. CL 614257 unconditionally zeroed
spans for large objects that contain pointers.
This change restores the optimization from before CL 614257, which seems
to matter in some real-world programs.
While we're here, let's also fix a hole with the garbage collector being
able to observe uninitialized memory of the large object is observed
by the conservative scanner before being published. The gory details are
in a comment in heapSetTypeLarge. In short, this change makes
span.largeType an atomic variable, such that the GC can only observe
initialized memory if span.largeType != nil.
Fixes#72991.
Change-Id: I2048aeb220ab363d252ffda7d980b8788e9674dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/659956
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
Currently, it's possible for asynchronous preemption to observe a
partially initialized object. The sequence of events goes like this:
- The GC is in the mark phase.
- Thread T1 is allocating object O1.
- Thread T1 zeroes the allocation, runs the publication barrier, and
updates freeIndexForScan. It has not yet updated the mark bit on O1.
- Thread T2 is conservatively scanning some stack frame.
That stack frame has a dead pointer with the same address as O1.
- T2 picks up the pointer, checks isFree (which checks
freeIndexForScan without an import barrier), and sees that O1 is
allocated. It marks and queues O1.
- T2 then goes to scan O1, and observes uninitialized memory.
Although a publication barrier was executed, T2 did not have an import
barrier. T2 may thus observe T1's writes to zero the object out-of-order
with the write to freeIndexForScan.
Normally this would be impossible if T2 got a pointer to O1 from
somewhere written by T1. The publication barrier guarantees that if the
read side is data-dependent on the write side then we'd necessarily
observe all writes to O1 before T1 published it. However, T2 got the
pointer 'out of thin air' by scanning a stack frame with a dead pointer
on it.
One fix to this problem would be to add the import barrier in the
conservative scanner. We would then also need to put freeIndexForScan
behind the publication barrier, or make the write to freeIndexForScan
exactly that barrier.
However, there's a simpler way. We don't actually care if conservative
scanning observes a stale freeIndexForScan during the mark phase.
Newly-allocated memory is always marked at the point of allocation (the
allocate-black policy part of the GC's design). So it doesn't actually
matter that if the garbage collector scans that memory or not.
This change modifies the allocator to only update freeIndexForScan
outside the mark phase. This means freeIndexForScan is essentially
a snapshot of freeindex at the point the mark phase started. Because
there's no more race between conservative scanning and newly-allocated
objects, the complicated scenario above is no longer a possibility.
One thing we do have to be careful of is other callers of isFree.
Previously freeIndexForScan would always track freeindex, now it no
longer does. This change thus introduces isFreeOrNewlyAllocated which is
used by the conservative scanner, and uses freeIndexForScan. Meanwhile
isFree goes back to using freeindex like it used to. This change also
documents the requirement on isFree that the caller must have obtained
the pointer not 'out of thin air' but after the object was published.
isFree is not currently used anywhere particularly sensitive (heap dump
and checkmark mode, where the world is stopped in both cases) so using
freeindex is both conceptually simple and also safe.
Change-Id: If66b8c536b775971203fb4358c17d711c2944723
Reviewed-on: https://go-review.googlesource.com/c/go/+/672340
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
CL 518776 deleted the cmd/go/internal/modconv package and dropped the
ability to import dependency requirements from ~nine or so legacy
pre-module dependency configuration files. Part of the rationale from
Russ in 2023 for dropping that support was that "by now no one is
running into those configs anymore during 'go mod init'".
For two of those legacy file formats, Godeps.json and vendor.json, the
ability to import their listed dependencies was dropped in CL 518776,
but what remained for those two formats was the ability to guess the
resulting module name in the absence of a name being supplied to 'go mod
init'.
This could be explained by the fact that this smaller functionality for
guessing a module name was separate, did not rely on the deleted modconv
package, and instead only relied on simple JSON parsing.
The name guessing was helpful as part of the transition when module
support was initially released, but it was never perfect, including the
various third-party dependency managers did not all have the same naming
rules that were enforced by modules.
In short, it is very unlikely anyone is relying on this now, so we
delete it.
This CL was spawned from discussion in two related documentation CLs
(CL 662675 and CL 662695).
Updates #71537
Change-Id: I9e087aa296580239562a0ecee58913c5edc533ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/664315
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Sam Thanawalla <samthanawalla@google.com>
In regalloc, we allocate some values to registers before loop entry,
so that they don't need to be loaded (from spill locations) during
the loop.
But it is pointless if we've already regalloc'd the loop body.
Whatever restores we needed for the body are already generated.
It's not clear if this code is ever useful. No tests fail if I just
remove it. But at least this change is worthwhile. It doesn't help,
and it actively inserts more restores than we really need (mostly
because the desired register list is approximate - I have seen cases
where the loads implicated here end up being dead because the restores
hit the wrong registers and the edge shuffle pass knows it needs
the restores in different registers).
While we are here, might as well have layoutRegallocOrder return
the standard layout order instead of recomputing it.
Change-Id: Ia624d5121de59b6123492603695de50b272b277f
Reviewed-on: https://go-review.googlesource.com/c/go/+/672735
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: David Chase <drchase@google.com>
When appending, if the backing store doesn't escape and a
constant-sized backing store is big enough, use a constant-sized
stack-allocated backing store instead of allocating it from the heap.
cmd/go is <0.1% bigger.
As an example of how this helps, if you edit strings/strings.go:FieldsFunc
to replace
spans := make([]span, 0, 32)
with
var spans []span
then this CL removes the first 2 allocations that are part of the growth sequence:
│ base │ exp │
│ allocs/op │ allocs/op vs base │
FieldsFunc/ASCII/16-24 3.000 ± ∞ ¹ 2.000 ± ∞ ¹ -33.33% (p=0.008 n=5)
FieldsFunc/ASCII/256-24 7.000 ± ∞ ¹ 5.000 ± ∞ ¹ -28.57% (p=0.008 n=5)
FieldsFunc/ASCII/4096-24 11.000 ± ∞ ¹ 9.000 ± ∞ ¹ -18.18% (p=0.008 n=5)
FieldsFunc/ASCII/65536-24 18.00 ± ∞ ¹ 16.00 ± ∞ ¹ -11.11% (p=0.008 n=5)
FieldsFunc/ASCII/1048576-24 30.00 ± ∞ ¹ 28.00 ± ∞ ¹ -6.67% (p=0.008 n=5)
FieldsFunc/Mixed/16-24 2.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~ (p=1.000 n=5)
FieldsFunc/Mixed/256-24 7.000 ± ∞ ¹ 5.000 ± ∞ ¹ -28.57% (p=0.008 n=5)
FieldsFunc/Mixed/4096-24 11.000 ± ∞ ¹ 9.000 ± ∞ ¹ -18.18% (p=0.008 n=5)
FieldsFunc/Mixed/65536-24 18.00 ± ∞ ¹ 16.00 ± ∞ ¹ -11.11% (p=0.008 n=5)
FieldsFunc/Mixed/1048576-24 30.00 ± ∞ ¹ 28.00 ± ∞ ¹ -6.67% (p=0.008 n=5)
(Of course, people have spotted and fixed a bunch of allocation sites
like this, but now we're ~automatically doing it everywhere going forward.)
No significant increases in frame sizes in cmd/go.
Change-Id: I301c4d9676667eacdae0058960321041d173751a
Reviewed-on: https://go-review.googlesource.com/c/go/+/664299
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Keith Randall <khr@golang.org>
Sync is used in the definition of primitives and documented by pkgbits.
It's not much help to also document it here.
Change-Id: I18bd0c7816f8249483550a1f0af7c76b9cfe09fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/674156
Auto-Submit: Mark Freeman <mark@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Bypass: Mark Freeman <mark@golang.org>
ncpu is the total logical CPU count at startup. It is never updated. For
#73193, we will start using updated CPU counts for updated GOMAXPROCS,
making the ncpu name a bit ambiguous. Change to a less ambiguous name.
While we're at it, give the OS specific lookup functions a common name,
so it can be used outside of osinit later.
For #73193.
Change-Id: I6a6a636cf21cc60de36b211f3c374080849fc667
Reviewed-on: https://go-review.googlesource.com/c/go/+/672277
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Positions mostly borrow their representation from package syntax. Of
note, constants (such as the zero value for positions) are not encoded
directly. Rather, a flag typically signals such values.
Change-Id: I6b4bafc6e96bb21902dd2d6e164031e7dd5aabdd
Reviewed-on: https://go-review.googlesource.com/c/go/+/673535
TryBot-Bypass: Mark Freeman <mark@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Mark Freeman <mark@golang.org>
asancall and msancall are reachable from the signal handler, where we
are running on gsignal. Currently, these calls will use the g0 stack in
this case, but if the interrupted code was running on g0 this will
corrupt the stack and likely cause a crash.
As far as I know, racecall is not reachable from the signal handler, but
I have updated it as well for consistency.
This is the most straightforward fix, though it would be nice to
eventually migrate these wrappers to asmcgocall, which already handled
this case.
Fixes#71395.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-asan-clang15,gotip-linux-amd64-msan-clang15,gotip-linux-amd64-race
Change-Id: I6a6a636ccba826dd53e31c0e85b5d42fb1e98d12
Reviewed-on: https://go-review.googlesource.com/c/go/+/643875
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The tests using testprog / testprogcgo are currently not covered on the
asan/msan/race builders because they don't build testprog with the
sanitizer flag.
Explicitly pass the flag if the test itself is built with the sanitizer.
There were a few tests that explicitly passed -race (even on non-race
builders). These tests will now only run on race builders.
For #71395.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-asan-clang15,gotip-linux-amd64-msan-clang15,gotip-linux-amd64-race
Change-Id: I6a6a636ce8271246316a80d426c0e4e2f6ab99c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/643897
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Previously, distpack filtered out tools from the packaged distribution
using a list of tools to remove. Instead follow mpratt's suggestion on
CL 666755 and instead filter out tools that are not on a list of tools
to keep. This will make it easier to tell which tools are actually in
the distribution.
For #71867
Change-Id: I8336465703ac820028c3381a0a743c457997e78a
Reviewed-on: https://go-review.googlesource.com/c/go/+/673696
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>
This CL enables intrinsic support to emit the following prefetch
instructions for loong64 platform:
1.Prefetch - prefetches data from memory address to cache;
2.PrefetchStreamed - prefetches data from memory address, with a
hint that this data is being streamed.
Benchmarks picked from go/test/bench/garbage
Parameters tested with:
GOMAXPROCS=8
tree2 -heapsize=1000000000 -cpus=8
tree -n=18
parser
peano
Benchmarks Loongson-3A6000-HV @ 2500.00MHz:
| bench.old | bench.new |
| sec/op | sec/op vs base |
Tree2-8 1238.2µ ± 24% 999.9µ ± 453% ~ (p=0.089 n=10)
Tree-8 277.4m ± 1% 275.5m ± 1% ~ (p=0.063 n=10)
Parser-8 3.564 ± 0% 3.509 ± 1% -1.56% (p=0.000 n=10)
Peano-8 39.12m ± 2% 38.85m ± 2% ~ (p=0.353 n=10)
geomean 83.19m 78.28m -5.90%
Change-Id: I59e9aa4f609a106d4f70706e6d6d1fe6738ab72a
Reviewed-on: https://go-review.googlesource.com/c/go/+/671876
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Meidan Li <limeidan@loongson.cn>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: sophie zhao <zhaoxiaolin@loongson.cn>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Deeply nested parenthesized expressions could cause a stack
overflow during parsing. This change introduces a depth limit
(maxStackDepth) tracked in Tree.stackDepth to prevent this.
Additionally, this commit clarifies the security model in
the package documentation, noting that template authors
are trusted as text/template does not auto-escape.
Fixes#71201
Change-Id: Iab2c2ea6c193ceb44bb2bc7554f3fccf99a9542f
GitHub-Last-Rev: f4ebd1719f
GitHub-Pull-Request: golang/go#73670
Reviewed-on: https://go-review.googlesource.com/c/go/+/671755
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Sean Liao <sean@liao.dev>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Rob Pike <r@golang.org>
The "doc", "fix", and "covdata" tools invoked by the go command are not
needed for builds. Instead of invoking them directly using the installed
binary in the tool directory, use "go tool" to run them, building them
if needed. We can then stop distributing those tools in the
distribution.
covdata is used in tests and can form part of a cached test result, but
test results don't have the same requirements as build outputs to be
completely determined by the action id. We already don't include a
toolid for the covdata tool in the action id for a test run. The more
principled way to do things would be to load the covdata package,
create the actions to build it, and then depend on the output of
that action from the the test action and use that as the covdata tool.
For now, it's probably not worth the effort, but, in the future, if we
wanted to build a tool like cgo as needed, it would be best to build it
in the same action graph. That would introduce a whole bunch of complexity
because we'd need to build the tool in the host configuration, and all
the configuration parameters are global.
For #71867
Change-Id: Id9bbbb5c169296f66c072949f9da552424ecfa2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/673119
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>
This change removes some tools that are not used for builds, or
otherwise invoked by the go command (other than through "go tool"
itself) from the packaged distributions produced by distpack. When these
tools are missing, "go tool" will build and run them as needed.
Also update a case where we print a buildid commandline to specify
invoking buildid using "go tool" rather than the binary at it's install
location, because it may not exist there in packaged distributions
anymore.
The tools in this CL are the lowest hanging fruit. There are a few more
tools that aren't used by builds, but we'd have to get the go command to
run them using "go tool" rather than finding them in the tool install
directory.
For #71867
Change-Id: I217683bd549962a1add87405bf3fb1225e2333c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/666755
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>
The purpose of this change is to enable go get to be used when working
on a module that is usually built from a workspace and has unreleased
dependencies in that workspacae that have no requirements satisfying
them. These modules can't build in single module mode, and the
expectation is that a workspace will provide the unreleased
requirements.
Before this change, if go get was run in a module, and any of the
module's imports, that were not already satisfied using requirements,
could not be resolved from that module in single module mode, go get
would report an error. This could happen if, for example, the dependency
was unreleased, even privately, and couldn't be fetched using version
control or a proxy. go get would also do a check using
cmd/go/internal/modget.(*resolver).checkPackageProblems that, among
other things, any package patterns provided to go get (the pkgPattern
argument to checkPackageProblems) could properly load. When checking in
single-module mode, this would cause an error because imports in the
non-required workspace dependencies could not be resolved.
This change makes a couple of changes to address each of those problems.
First, while "go get" still uses the single module's module graph to
load packages and determine which imports are not satisfied by a module
in the build list (the set of modules the build is done with), it will
"cheat" and look up the set of modules that would be loaded in workspace
mode. It will not try to fetch modules to satisfy imports of packages in
those modules. (Alternatively, it could have tried to fetch modules to
satisfy the requirements, and allowed an error if it could not be found,
but we took the route of not doing the fetch to preserve the invariant
that the behavior wouldn't change if the network was down). The second,
and by far more complex, change is that the load that's done in
checkPackageProblems will be done in workspace mode rather than module
mode. While it is important that the requirements added by "go get" are
not determined using the workspace (with the necessary exception of the
skipped fetches) it is okay to use the workspace to load the modules,
as, if a go.work file is present, the go command would by default run
builds in workspace mode rather than single module mode. This more
relaxed check will allow get to succeed if a go list would succeed in
the workspace, even if it wouldn't from the single module.--
To avoid trying to satisfy imports that are in the other workspace
modules, we add a workspace field to the resolver that can be used to
check if an import is in the workspace. It reads the go.work file and
determines each of the modules' modroots. The hasPackage function will
call into modload logic using the new PkgIsInLocalModule function that
in turn calls into dirInModule to determine if the directory that would
contain the package sources exists in the module. We do that check in
cmd/go/internal/modget.(*resolver).loadPackages, which is used to
resolve modules to satisfy imports in the package graph supplied on the
command line. (Note that we do not skip resolving modules in the
functions that query for a package at a specific module version (such as
in "go get golang.org/x/tools/go/packages@latest), which are done in
(*resolver).queryPath. In that case, the user is explicitly requesting
to add a requirement on that package's module.)
The next step, checking for issues in the workspace mode, is more
complex because of two reasons. First, can't do all of
checkPackageProblems's work in a workspace, and second, that the module
loading state in the go command is global, so we have to manage the
global state in switching to workspace mode and back.
On the work that checkPackageProblems does: it broadly does three things:
first, a load of the packages specified to "go get", to make sure that
imports are satisfied and not ambiguous. Second, using that loaded
information, reporting any retracted or deprecated modules that are
relevant to the user and that they may want to take action on. And
third, checking that all the modules in the build list (the set of
module versions used in a load or build) have sums, and adding those
sums to the in-memory representation of the go.sum file that will be
written out at the end. When there's a workspace, the first two checks
need to be done in workspace mode so that we properly point out issues
in the build, but the sums need to be updated in module mode so that the
module's go.sum file is updated to reflect changes in go.mod.
To do the first two steps in workspace mode, we add a new
modload.EnterWorkspace function that will reset the global modload state
and load from the workspace, using the updated requirements that have
been calculated for the module. It returns a cleanup function that will
exit the workspace and reset to the previous global state. (We need the
previous global state because it holds the updated in memory
representations of go.mod and go.sum that will be written out by go get
if there are no errors.) We switch to workspace mode if there's a
relevant go.work file that would trigger a workspace load _and_ the
module go get is being run from belongs to that workspace (it wouldn't
make sense to use a workspace that the module itself didn't belong to).
We then switch back to module mode after the first two steps are
complete using the cleanup function. We have to be careful to finish
all the tasks checking for deprecations and retractions before to start
looking at the sums because they retraction and deprecation checking
tasks will depend on the global workspace state.
It's a bit unfortunate that much of the modload and modfetch state is
global. It's pretty gross doing the switch. It would be a lot of
work, but if we need to do a switch in a third instance other than for
go work sync and go get it might be worth doing the work to refactor
modload so that the state isn't global.
The EnterWorkspace function that does the switch to workspace mode (in
cmd/go/internal/modload/init.go) first saves the in memory
representation of the go.mod file (calculated using UpdateGoModFromReqs)
so that it can be applied in the workspace. It then uses the new
setState function to save the old state and reset to a clean state,
loads in workspace mode (using InitWorkfile to so that the go.work file
is used by LoadModFile), and then replaces the go.mod file
representation for the previous main module with the contents saved
earlier and reloads the requirements (holding the roots of the module
graph) to represent the updates. In workspace mode, the roots field of the
requirements is the same, but the reload is used to update the set of
direct requirements. rawGoModSummary is also update to use the in-
memory representation of the previous main module's go.mod file rather
than always reading it from disk so that it can take those updated
contents into account. (It previously didn't need to do this because
rawGoModSummary is primarily used to get module information for
dependency modules. The exception is in workspace mode but the same
logic worked because we didn't update workspace module's go.mod files in
a go command running in workspace mode). Finally, EnterWorkspace returns
a function that calls setState with the old state, to revert to the
previous state.
When we save the state of the modload package, we also need to save the
state of the modfetch package because it holds the state of the go.sum
file and the operation of looking up a value in the lookupCache or
downloadCache affects whether it appears in the go.sum file. lookupCache
and downloadCache are turned into pointers so that they can be saved in
the modfetchState.
Fixes#73654
Change-Id: I65cf835ec2293d4e3f66b91d3e77d3bb8d2f26d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/669635
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
This test expects to be able to drain a Pool using only Get. This isn't
actually possible in the general case, since a pooled value could get
stuck in some P's private slot. However, if GOMAXPROCS=1, there's only 1
P we could be running on, so getting stuck becomes impossible.
This test isn't checking any concurrent properties of Pool, so this is
fine. Just set GOMAXPROCS=1 for this one particular test.
Fixes#73728.
Change-Id: I9053e28118060650f2cd7d0d58f5a86d630b36f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/673375
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
This CL adds the ignore directive which enables users to tell the Go
Command to skip traversing into a given directory.
This behaves similar to how '_' or 'testdata' are currently treated.
This mainly has benefits for go list and go mod tidy.
This does not affect what is packed into a module.
Fixes: #42965
Change-Id: I232e27c1a065bb6eb2d210dbddad0208426a1fdd
Reviewed-on: https://go-review.googlesource.com/c/go/+/643355
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@google.com>
Currently, there's a window of time where each cleanup goroutine has
committed to going to sleep (immediately after full.pop() == nil) but
hasn't yet marked itself as asleep (state.sleep()). If new work arrives
in this window, it might get missed. This is what we see in #73642, and
I can reproduce it with stress2.
Side-note: even if the work gets missed by the existing sleeping
goroutines, needg is incremented. So in theory a new goroutine will
handle the work. Right now that doesn't happen in tests like the one
running in #73642, where there might never be another call to AddCleanup
to create the additional goroutine. Also, if we've hit the maximum on
cleanup goroutines and all of them are in this window simultaneously, we
can still end up missing work, it's just more rare. So this is still a
problem even if we choose to just be more aggressive about creating new
cleanup goroutines.
This change fixes the problem and also aims to make the cleanup
wake/sleep code clearer. The way this change fixes this problem is to
have cleanup goroutines re-check the work list before going to sleep,
but after having already marked themselves as sleeping. This way, if new
work comes in before the cleanup goroutine marks itself as going to
sleep, we can rely on the re-check to pick up that work. If new work
comes after the goroutine marks itself as going to sleep and after the
re-check, we can rely on the scheduler noticing that the goroutine is
asleep and waking it up. If work comes in between a goroutine marking
itself as sleeping and the re-check, then the re-check will catch that
piece of work. However, the scheduler might now get a false signal that
the goroutine is asleep and try to wake it up. This is OK. The sleeping
signal is now mutated and double-checked under the queue lock, so the
scheduler will grab the lock, may notice there are no sleeping
goroutines, and go on its way. This may cause spurious lock acquisitions
but it should be very rare. The window between a cleanup goroutine
marking itself as going to sleep and re-checking the work list is a
handful of instructions at most.
This seems subtle but overall it's a simplification of the code. We
rely more on the lock, which is easier to reason about, and we track two
separate atomic variables instead of the merged cleanupSleepState: the
length of the full list, and the number of cleanup goroutines that are
asleep. The former is now the primary way to acquire work. Cleanup
goroutines must decrement the length successfully to obtain an item off
the full list. The number of cleanup goroutines asleep, meanwhile, is
now only updated with the queue lock held. It can be checked without the
lock held, and the invariant to make that safe is simple: it must always
be an overestimate of the number of sleeping cleanup goroutines.
The changes here do change some other behaviors.
First, since we're tracking the length of the full list instead of the
abstract concept of a wake-up, the waker can't consume wake-ups anymore.
This means that cleanup goroutines may be created more aggressively. If
two threads in the scheduler see that there are goroutines that are
asleep, only one will win the race, but the other will observe zero
asleep goroutines but potentially many work units available. This will
cause it to signal many goroutines to be created. This is OK since we
have a cap on the number of cleanup goroutines, and the race should be
relatively rare.
Second, because cleanup goroutines can now fail to go to sleep if any
units of work come in, they might spend more time contended on the lock.
For example, if we have N cleanup goroutines and work comes in at *just*
the wrong rate, in the worst case we'll have each of G goroutines loop
N times for N blocks, resulting in O(G*N) thread time to handle each
block in the worst case. To paint a picture, imagine each goroutine
trying to go to sleep, fail because a new block of work came in, and
only one goroutine will get that block. Then once that goroutine is
done, we all try again, fail because a new block of work came in, and so
on and so forth. This case is unlikely, though, and probably not worth
worrying about until it actually becomes a problem. (A similar problem
exists with parking (and exists before this change, too) but at least in
that case each goroutine parks, so it doesn't block the thread.)
Fixes#73642.
Change-Id: I6bbe1b789e7eb7e8168e56da425a6450fbad9625
Reviewed-on: https://go-review.googlesource.com/c/go/+/671676
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
FILE_FLAG_BACKUP_SEMANTICS is necessary to open directories on Windows,
and to enable backup applications do extended operations on files if
they hold the SE_BACKUP_NAME and SE_RESTORE_NAME privileges.
os.OpenFile currently sets FILE_FLAG_BACKUP_SEMANTICS for all supported
cases except when the file is opened with O_WRONLY | O_RDWR (that is,
access mode 3). This access mode doesn't correspond to any of the
standard POSIX access modes, but some OSes special case it to mean
different things. For example, on Linux, O_WRONLY | O_RDWR means check
for read and write permission on the file and return a file descriptor
that can't be used for reading or writing.
On Windows, os.OpenFile has historically mapped O_WRONLY | O_RDWR to a
0 access mode, which Windows internally interprets as
FILE_READ_ATTRIBUTES. Additionally, it doesn't prepare the file for I/O,
given that the read attributes permission doesn't allow reading or
writing (not that this is similar to what happens on Linux). This
makes opening the file around 50% faster, and one can still use the
handle to stat it, so some projects have been using this behavior
to open files without I/O access.
This CL updates os.OpenFile so that directories can also be opened
without I/O access. This effectively closes#23312, as all the remaining
cases where we don't set FILE_FLAG_BACKUP_SEMANTICS imply opening
with O_WRONLY or O_RDWR, and that's not allowed by Unix's open.
Closes#23312.
Change-Id: I77c4f55e1ca377789aef75bd8a9bce2b7499f91d
Reviewed-on: https://go-review.googlesource.com/c/go/+/673035
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The BoGo IgnoreClientVersionOrder test checks that a client that sends
a supported_versions extension with the list [TLS 1.2, TLS 1.3] ends up
negotiating TLS 1.3.
However, the crypto/tls module treats this list as being in client
preference order, and so negotiates TLS 1.2, failing the test.
Our behaviour appears to be the correct handling based on RFC 8446
§4.2.1 where it says:
The extension contains a list of supported versions in preference
order, with the most preferred version first.
This commit updates the reason we skip this test to cite the RFC instead
of saying it's something to be fixed.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/671415
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Daniel McCarney <daniel@binaryparadox.net>
x += *p
We want to do this with a single load+add operation on amd64.
The tricky part is that we don't want to combine if there are
other uses of x after this instruction.
Implement a simple detector that seems to capture a common situation -
x += *p is in a loop, and the other use of x is after loop exit.
In that case, it does not hurt to do the load+add combo.
Change-Id: I466174cce212e78bde83f908cc1f2752b560c49c
Reviewed-on: https://go-review.googlesource.com/c/go/+/672957
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TestMutexBlockFullAggregation aggregates stacks by function, file, and
line number. But there can be multiple function calls on the same line,
giving us different sequences of PCs. This causes the test to spuriously
fail in some cases. Include PCs in the stacks for this test.
Also pick up a small "range over int" modernize suggestion while we're
looking at the test.
Fixes#73641
Change-Id: I50489e19fcf920e27b9eebd9d4b35feb89981cbc
Reviewed-on: https://go-review.googlesource.com/c/go/+/673115
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
os.Stat and os.Lstat first try stating the file without opening it. If
that fails, then they open the file and try again, operations that tends
to be slow. There is no point in trying the slow path if the file
doesn't exist, we should just return an error immediately.
This CL makes stating a non-existent file on Windows 50% faster:
goos: windows
goarch: amd64
pkg: os
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
│ old.txt │ new.txt │
│ sec/op │ sec/op vs base │
StatNotExist-12 43.65µ ± 15% 20.02µ ± 10% -54.14% (p=0.000 n=10+7)
│ old.txt │ new.txt │
│ B/op │ B/op vs base │
StatNotExist-12 224.0 ± 0% 224.0 ± 0% ~ (p=1.000 n=10+7) ¹
¹ all samples are equal
│ old.txt │ new.txt │
│ allocs/op │ allocs/op vs base │
StatNotExist-12 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10+7) ¹
Updates #72992.
Change-Id: Iaeb9596d0d18e5a5a1bd1970e296a3480501af78
Reviewed-on: https://go-review.googlesource.com/c/go/+/671458
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Jake Bailey <jacob.b.bailey@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
The WSASocket documentation states that the returned socket must be
closed by calling closesocket instead of CloseHandle. The different
File methods on the net package return an os.File that is not aware
that it should use closesocket. Ideally, os.NewFile should detect that
the passed handle is a socket and use the appropriate close function,
but there is no reliable way to detect that a handle is a socket on
Windows (see CL 671455).
To work around this, we add a hidden function to the os package that
can be used to return an os.File that uses closesocket. This approach
is the same as used on Unix, which also uses a hidden function for other
purposes.
While here, fix a potential issue with FileConn, which was using File.Fd
rather than File.SyscallConn to get the handle. This could result in the
File being closed and garbage collected before the syscall was made.
Fixes#73683.
Change-Id: I179405f34c63cbbd555d8119e0f77157c670eb3e
Reviewed-on: https://go-review.googlesource.com/c/go/+/672195
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
On every arch except amd64, it is faster to do x&(x-1) than x^(1<<n).
Most archs need 3 instructions for the latter: MOV $1, R; SLL n, R;
ANDN R, x. Maybe 4 if there's no ANDN.
Most archs need only 2 instructions to do x&(x-1). It takes 3 on
x86/amd64 because NEG only works in place.
Only amd64 can do x^(1<<n) in a single instruction.
(We could on 386 also, but that's currently not implemented.)
Change-Id: I3b74b7a466ab972b20a25dbb21b572baf95c3467
Reviewed-on: https://go-review.googlesource.com/c/go/+/672956
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
To understand this change, we begin with a short description of the UIR
file format.
Every file is a header followed by a series of sections. Each section
has a kind, which determines the type of elements it contains. An
element is just a collection of one or more primitives, as defined by
package pkgbits.
Strings have their own section. Elements in the string section contain
only string primitives. To use a string, elements in other sections
encode a reference to the string section.
To illustrate, consider a simple file which exports nothing at all.
package p
In the meta section, there is an element representing a package stub.
In that package stub, a string ("p") represents both the path and name
of the package. Again, these are encoded as references.
To manage references, every element begins with a reference table.
Instead of writing the bytes for "p" directly, the package stub encodes
an index in this reference table. At that index, a pair of numbers is
stored, indicating:
1. which section
2. which element index within the section
Effectively, elements always use *2* layers of indirection; first to the
reference table, then to the bytes themselves.
With some minor hand-waving, an encoding for the above package is given
below, with (S)ections, (E)lements and (P)rimitives denoted.
+ Header
| + Section Ends // each section has 1 element
| | + 1 // String is elements [0, 1)
| | + 2 // Meta is elements [1, 2)
| + Element Ends
| | + 1 // "p" is bytes [0, 1)
| | + 6 // stub is bytes [1, 6)
+ Payload
| + (S) String
| | + (E) String
| | | + (P) String { byte } 0x70 // "p"
| + (S) Meta
| | + (E) Package Stub
| | | + Reference Table
| | | | + (P) Entry Count uvarint 1 // there is a single entry
| | | | + (P) 0th Section uvarint 0 // to String, 0th section
| | | | + (P) 0th Index uvarint 0 // to 0th element in String
| | | + Internals
| | | | + (P) Path uvarint 0 // 0th entry in table
| | | | + (P) Name uvarint 0 // 0th entry in table
Note that string elements do not have reference tables like other
elements. They behave more like a primitive.
As this is a bit complicated and getting into details of the UIR file
format, we omit some details in the documentation here. The structure
will become clearer as we continue documenting.
Change-Id: I12a5ce9a34251c5358a20f2f2c4d0f9bd497f4d0
Reviewed-on: https://go-review.googlesource.com/c/go/+/671997
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Mark Freeman <mark@golang.org>
TryBot-Bypass: Mark Freeman <mark@golang.org>
Git's supported SHA 256 object hashes since 2.29[1] in 2021, and Gitlab
now has experimental support for sha256 repos.
Take rsc@'s suggestion of checking the of the length of the hashes from
git ls-remote to determine whether a git repo is using sha256 hashes and
decide whether to pass --object-format=sha256 to git init.
Unfortunately, just passing --object-format=sha256 wasn't quite enough,
though. We also need to decide whether the hash-length is 64 hex bytes
or 40 hex bytes when resolving refs to decide whether we've been passed
a full commit-hash. To that end, we use
git config extensions.objectformat to decide whether the (now guaranteed
local) repo is using sha256 hashes and hence 64-hex-byte strings.
[1]: lost experimental status in 2.42 from Aug 2023
(https://lore.kernel.org/git/xmqqr0nwp8mv.fsf@gitster.g/)
For: #68359
Change-Id: I47f480ab8334128c5d17570fe76722367d0d8ed8
Reviewed-on: https://go-review.googlesource.com/c/go/+/636475
Auto-Submit: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@google.com>
Reviewed-by: David Finkel <david.finkel@gmail.com>
Specified in RISC-V ELF psABI[1], mapping symbols are symbols starting
with "$d" or "$x" with STT_NOTYPE, STB_LOCAL and zero sizes, indicating
boundaries between code and data in the same section.
Let's simply ignore them as they're only markers instead of real symbols.
This fixes linking errors like
sym#63 ("$d"): ignoring symbol in section 4 (".riscv.attributes") (type 0)
when using CGO together with Clang and internal linker, which are caused
by unnecessary (but technically correct) mapping symbols created by LLVM
for various sections.
[1]: 87aecf6017/riscv-elf.adoc (L1448)Fixes#73516
Change-Id: I02ca90c100ba8a38733fe3b8b8403836b44a3dd1
GitHub-Last-Rev: d7842ceafb
GitHub-Pull-Request: golang/go#73592
Reviewed-on: https://go-review.googlesource.com/c/go/+/669675
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
This complements the grammar being developed in package noder. It
is unclear how to discuss references in their current state, as
they require knowledge of sections, elements, etc.
Perhaps the references here should refer to indices on the byte
array. This would allow a stronger separation of pkgbits and noder.
Change-Id: Ic0e5ac9c07f0a0b92d6ffd4d4e26dbe5dcf89e57
Reviewed-on: https://go-review.googlesource.com/c/go/+/671440
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Bypass: Mark Freeman <mark@golang.org>
Auto-Submit: Mark Freeman <mark@golang.org>
NewFile was recently updated (in CL 668195) to detect whether the
handle is a socket or not. This special case is not really necessary,
given that socket handles can be used as if they were normal file
handles on all functions supported by os.File (see https://learn.microsoft.com/en-us/windows/win32/winsock/socket-handles-2).
Not only is not necessary, but is can also be problematic, as there is
no way to reliably detect whether a handle is a socket or not. For
example, the test failure reported in #73630 is caused by a named pipe
wrongly detected as a socket.
This aligns with the Unix NewFile behavior of returning an os.File that
identifies itself as a file handle even if it is a socket. This makes
os.File.Close to always return os.ErrClosed in case of multiple calls
rather than sometimes returning "use of closed network connection".
Updates #10350.
Fixes#73630.
Change-Id: Ia8329783d5c8ef6dac34ef69ed1ce9d2a9862e11
Reviewed-on: https://go-review.googlesource.com/c/go/+/671455
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Windows prefers 64-bit binaries to be loaded at an address above 4GB.
Having a preferred base address below this boundary triggers a
compatibility mode in Address Space Layout Randomization (ASLR) on
recent versions of Windows that reduces the number of locations to which
ASLR may relocate the binary.
The Go internal linker was using a smaller base address due to an issue
with how dynamic cgo symbols were relocated, which has been fixed in
this CL.
Fixes#73561.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest
Change-Id: Ia8cb35d57d921d9be706a8975fa085af7996f124
Reviewed-on: https://go-review.googlesource.com/c/go/+/671515
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
The first loop leaves the lengths of the two arguments unchanged.
Take advantage of this invariant in the loop's condition. Here are some
benchmark results (no change to allocations):
goos: darwin
goarch: amd64
pkg: strings
cpu: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
│ old │ new │
│ sec/op │ sec/op vs base │
EqualFold/Tests-8 240.0n ± 4% 245.1n ± 5% ~ (p=0.516 n=20)
EqualFold/ASCII-8 11.50n ± 1% 11.04n ± 0% -3.96% (p=0.000 n=20)
EqualFold/UnicodePrefix-8 102.1n ± 0% 102.2n ± 0% ~ (p=0.455 n=20)
EqualFold/UnicodeSuffix-8 90.14n ± 0% 89.80n ± 1% ~ (p=0.113 n=20)
geomean 71.00n 70.60n -0.56%
Change-Id: I1f6d1df8a0398f9493692f59d7369c3f0fbba436
GitHub-Last-Rev: 9508ee26ad
GitHub-Pull-Request: golang/go#73672
Reviewed-on: https://go-review.googlesource.com/c/go/+/671756
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
If the client hello legacy version is >= TLS 1.3, and no
supported_versions extension is sent, negotiate TLS 1.2 or lower when
supported.
On the topic of supported version negotiation RFC 8446 4.2.1 indicates
TLS 1.3 implementations MUST send a supported_versions extension with
a list of their supported protocol versions. The crypto/tls package
enforces this when the client hello legacy version indicates TLS 1.3
(0x0304), aborting the handshake with an alertMissingExtension alert if
no supported_versions were received.
However, section 4.2.1 indicates different behaviour should be used when
the extension is not present and TLS 1.2 or prior are supported:
If this extension is not present, servers which are compliant with
this specification and which also support TLS 1.2 MUST negotiate
TLS 1.2 or prior as specified in [RFC5246], even if
ClientHello.legacy_version is 0x0304 or later.
This commit updates the client hello processing logic to allow this
behaviour. If no supported_versions extension was received we ignore the
legacy version being >= TLS 1.3 and instead negotiate a lower supported
version if the server configuration allows.
This fix in turn allows enabling the BoGo ClientHelloVersionTooHigh,
MinorVersionTolerance, and MajorVersionTolerance tests.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/671235
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Previously for protocol versions older than TLS 1.3 our server handshake
implementation sent an alertBadCertificate alert in the case where the
server TLS config indicates a client cert is required and none was
received.
This commit updates the relevant logic to instead send
alertHandshakeFailure in these circumstances.
For TLS 1.2, RFC 5246 §7.4.6 unambiguously describes this as the correct
alert:
If the client does not send any certificates, the
server MAY at its discretion either continue the handshake without
client authentication, or respond with a fatal handshake_failure
alert.
The TLS 1.1 and 1.0 specs also describe using this alert (RFC 4346 §7.4.6
and RFC 2246 §7.4.6) both say:
If client authentication is required by the server for the handshake
to continue, it may respond with a fatal handshake failure alert.
Making this correction also allows enabling the
RequireAnyClientCertificate-TLS1* bogo tests.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/671195
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Previously a handful of large record tests were in the bogo config
ignore list. The ignored tests were failing because they used
insecure ciphersuites that aren't enabled by default.
This commit adds the non-default insecure ciphersuites to the bogo
TLS configuration and re-enables the tests. Doing this uncovered
a handful of unrelated tests that needed to be fixed, each handled
before this commit.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/669158
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
These two bogo tests mutate the version number used for the premaster
secret calculation for a client RSA key exchange, with the expectation
the server rejects the handshake.
Per the comment in the end of rsaKeyAgreement.processClientKeyExchange
we explicitly choose *not* to verify the version number.
This commit adds the two version number tests to the ignore list. They
coincidentally happen to produced the expected failure because they use
a non-default ciphersuite. When we add this ciphersuite to the client
config for the bogo test they will start to fail unless ignored.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/669175
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Unlike in earlier TLS versions, in TLS 1.3 when processing a server
hello the legacy_compression_method MUST have the value 0. It is no
longer a parameter that offers a choice of compression method.
With this in mind, it seems more appropriate to return a decode error
when we encounter a non-zero compression method in a server hello
message. We haven't found a parameter value we reject, we've found
a message that doesn't decode according to its specification.
Making this change also aligns with BoringSSL and allows enabling the
TLS13-HRR-InvalidCompressionMethod bogo test.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/669156
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Previously if the clientHandshakeState for the TLS 1.2 client code
encountered a server helo message that contained a compression method
other than compressionNone, we would emit an unexpected message alert.
Instead, it seems more appropriate to return an illegal parameter alert.
The server hello message _was_ expected, it just contained a bad
parameter option.
Making this change also allows enabling the InvalidCompressionMethod
bogo test.
Updates #72006
Change-Id: I27a2cd231e4b8762b0d9e2dbd3d8ddd5b87fd5c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/669155
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
If a tool in cmd is not installed in $GOROOT/pkg/tool/${GOOS}_${GOARCH},
go tool will build (if it's not cached) and run it in a similar way
(with some changes) to how tools declared with tool directives are built
and run.
The main change in how builtin tools are run as compared to mod tools is
that they are built "in host mode" using the running go command's GOOS
and GOARCH. The "-exec" flag is also ignored and we don't add GOROOT/bin
to the PATH.
A ForceHost function has been added to the cfg package to force the
configuration to runtime.GOOS/runtime.GOARCH. It has to recompute the
BuildContext because it's normally determined at init time but we're
changing it after we realize we're running a builtin tool. (Detecting
that we're running a builtin tool at init time would mean replicating
the cmd line parsing logic so recomputing BuildContext sounds like the
smaller change.)
For #71867
Change-Id: I3b2edf2cb985c1dcf5f845fbf39b7dc11dea4df7
Reviewed-on: https://go-review.googlesource.com/c/go/+/666476
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Matloob <matloob@google.com>
Meta is the most fundamental section. To flesh this out, we discuss references. Primitives are briefly mentioned by pointing to pkgbits,
where they will be defined using a similar grammar.
Change-Id: I7abd899f38fad4cc5caf87ebfc7aa1b1985b17d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/671176
Auto-Submit: Mark Freeman <mark@golang.org>
TryBot-Bypass: Mark Freeman <mark@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
I noticed a failure of this test on a linux/amd64 builder and reproduced
it locally. I can only really reproduce it in a stress test when I
overload my system (`stress2 ./tls.test -test.run=TestCertCache`) but
this points to the root of the problem: it's possible for a timer to get
delayed and the timeout fires before we ever get the chance to check.
After copious debugging printlns, this is essentially what I'd observed.
There would only be one failed check of the reference count from before
it was updated.
Change the test to be a busy-loop again, but call runtime.Gosched. This
is also what we do for the os.Root tests, and in hindsight should've
been my go-to. This has a much higher likelihood of executing promptly.
We may want to go back and understand why the 1 ms timer would fire so
hilariously late the second time. This might be a real bug. For now,
this change makes the test more stable. It no longer fails when it's
hammered under `stress2`.
Fixes#73637.
Change-Id: I316bd9e30946f4c055e61d179c4efc5fe029c608
Reviewed-on: https://go-review.googlesource.com/c/go/+/671175
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The UIR export data format can be reasonably expressed using EBNF.
The noder owns the definition of the export data format, so this
seems like a reasonable place to put this.
Change-Id: I0205ab29a3c5e57d670d7fd3164a8bd604ab8e59
Reviewed-on: https://go-review.googlesource.com/c/go/+/670616
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Bypass: Mark Freeman <mark@golang.org>
Auto-Submit: Mark Freeman <mark@golang.org>
This change splits the finalizer and cleanup queues and implements a new
lock-free blocking queue for cleanups. The basic design is as follows:
The cleanup queue is organized in fixed-sized blocks. Individual cleanup
functions are queued, but only whole blocks are dequeued.
Enqueuing cleanups places them in P-local cleanup blocks. These are
flushed to the full list as they get full. Cleanups can only be enqueued
by an active sweeper.
Dequeuing cleanups always dequeues entire blocks from the full list.
Cleanup blocks can be dequeued and executed at any time.
The very last active sweeper in the sweep phase is responsible for
flushing all local cleanup blocks to the full list. It can do this
without any synchronization because the next GC can't start yet, so we
can be very certain that nobody else will be accessing the local blocks.
Cleanup blocks are stored off-heap because the need to be allocated by
the sweeper, which is called from heap allocation paths. As a result,
the GC treats cleanup blocks as roots, just like finalizer blocks.
Flushes to the full list signal to the scheduler that cleanup goroutines
should be awoken. Every time the scheduler goes to wake up a cleanup
goroutine and there were more signals than goroutines to wake, it then
forwards this signal to runtime.AddCleanup, so that it creates another
goroutine the next time it is called, up to gomaxprocs goroutines.
The signals here are a little convoluted, but exist because the sweeper
and the scheduler cannot safely create new goroutines.
For #71772.
For #71825.
Change-Id: Ie839fde2b67e1b79ac1426be0ea29a8d923a62cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/650697
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Currently TestCertCache will busy loop waiting for a cleanup (in the
runtime.AddCleanup sense) to execute. If we ever get into this busy
loop, then on single-threaded platforms like js/wasm, we'll end up
_always_ timing out.
This doesn't happen right now because we're getting lucky. The finalizer
goroutine is scheduled into the runnext slot with 'ready' and is thus
scheduled immediately after the GC call. In a follow-up CL, scheduling
cleanup goroutines becomes less aggressive, and thus this test fails.
Although perhaps that CL should schedule cleanup goroutines more
aggressively, the test is still technically buggy, because it expects
busy loops like this to call into the scheduler, but that won't happen
on certain platforms.
Change-Id: I8efe5975be97f4314aec1c8c6e9e22f396be9c94
Reviewed-on: https://go-review.googlesource.com/c/go/+/670755
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The stated goal for pkgbits is to implement encoding / decoding of
primitives. However, pkgbits has knowledge of high-level details like
elements, sections, and file layout.
This change starts to clarify pkgbits by paring back documentation to
only those concepts which pkgbits owns. Further CLs are needed to shift
away logic that pkgbits should not own.
Change-Id: Id93003d080f58ffbd6327e2db1a4878500511619
Reviewed-on: https://go-review.googlesource.com/c/go/+/670176
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Mark Freeman <mark@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Mark Freeman <mark@golang.org>
It's the job of the last sweeper to emit the GC pacer trace. The last
sweeper can identify themselves by reducing the count of sweepers, and
also seeing that there's no more sweep work.
Currently this identification is broken, however, because the last
sweeper doesn't check the state they just transitioned sweeping into,
but rather the state they transitioned from (one sweeper, no sweep work
left). By design, it's impossible to transition *out* of this state,
except for another GC to start, but that doesn't take this codepath.
This means lines like
pacer: sweep done at heap size ...
were missing from the gcpacertrace output for a long time.
This change fixes this problem by having the last sweeper check the
state they just transitioned sweeping to, instead of the state they
transitioned from.
Change-Id: I44bcd32fe2c8ae6ac6c21ba6feb2e7b9e17f60cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/670735
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
The RISC-V Instruction Set Manual Volume states that "for vadc and
vsbc, the instruction encoding is reserved if the destination vector
register is v0". The assembler currently allows instructions like
VADCVVM V1, V2, V0, V0
to be assembled. It's not clear what the behaviour of such
instructions will be on target hardware so it's best to disallow
them.
For reference, binutils (2.44-3.fc42) allows the instruction
vadc.vvm v0, v4, v8, v0
to be assembled and the instruction actually executes on a Banana PI
F3 without crashing. However, clang (20.1.2) refuses to assemble the
instruction, producing the following error.
error: the destination vector register group cannot be V0
vadc.vvm v0, v4, v8, v0
^
Change-Id: Ia913cbd864ae8dbcf9227f69b963c93a99481cff
Reviewed-on: https://go-review.googlesource.com/c/go/+/669315
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
The encodings for the riscv64 special operands SPOP_MF2 and SPOP_MF8
are incorrect, i.e., their values are swapped. This leads to
incorrect encodings for the VSETVLI and VSETIVLI instructions. The
assembler currently encodes
VSETVLI X10, E32, MF8, TA, MA, X12
as
VSETVLI X10, E32, MF2, TA, MA, X12
We update the encodings for SPOP_MF2 and SPOP_MF8 so that they match
the LMUL table in section "31.3.4. Vector type register, vtype" of
the "RISC-V Instruction Set Manual Volume 1".
Change-Id: Ic73355533d7c2a901ee060b35c2f7af6d58453e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/670016
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
Have the test use the same clock (cputicks) as the profiler, and use the
test's own measurements as hard bounds on the magnitude to expect in the
profile.
Compare the depiction of two users of the same lock: one where the
critical section is fast, one where it is slow. Confirm that the profile
shows the slow critical section as a large source of delay (with #66999
fixed), rather than showing the fast critical section as a large
recipient of delay.
Previously reviewed as https://go.dev/cl/586237.
For #66999
Change-Id: Ic2d78cc29153d5322577d84abdc448e95ed8f594
Reviewed-on: https://go-review.googlesource.com/c/go/+/667616
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com>
Correct how the mutex contention profile reports on runtime-internal
mutex values, to match sync.Mutex's semantics.
Decide at the start of unlock2 whether we'd like to collect a contention
sample. If so: Opt in to a slightly slower unlock path which avoids
accidentally accepting blame for delay caused by other Ms. Release the
lock before doing an O(N) traversal of the stack of waiting Ms, to
calculate the total delay to those Ms that our critical section caused.
Report that, with the current callstack, in the mutex profile.
Fixes#66999
Change-Id: I561ed8dc120669bd045d514cb0d1c6c99c2add04
Reviewed-on: https://go-review.googlesource.com/c/go/+/667615
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
This change moves the unique package away from using a concurrent map
and instead toward a bespoke concurrent canonicalization map. The map
holds all its keys weakly, though keys may be looked up by value. The
result is the strong pointer for the canonical value. Entries in the map
are automatically cleaned up once the canonical reference no longer
exists.
Why do this? There's a problem with the current implementation when it
comes to chains of unique.Handle: because the unique map will have a
unique.Handle stored in its keys, each nested handle must be cleaned up
1 GC at a time. It takes N GC cycles, at minimum, to clean up a nested
chain of N handles. This implementation, where the *only* value in the
set is weakly-held, does not have this problem. The entire chain is
dropped at once.
The canon map implementation is a stripped-down version of HashTrieMap.
The weak set implementation also has lower memory overheads by virtue of
the fact that keys are all stored weakly. Whereas the previous map had
both a T and a weak.Pointer[T], this *only* has a weak.Pointer[T].
The canonicalization map is a better abstraction overall and
dramatically simplifies the unique.Make code.
While we're here, delete the background goroutine and switch to
runtime.AddCleanup. This is a step toward fixing #71772. We still need
some kind of back-pressure mechanism, which will be implemented in a
follow-up CL.
For #71772.
Fixes#71846.
Change-Id: I5b2ee04ebfc7f6dd24c2c4a959dd0f6a8af24ca4
Reviewed-on: https://go-review.googlesource.com/c/go/+/650256
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
The doInRoot function operates on a path split into components.
The final path component retained any trailing path separator
characters, to permit operations in a Root to retain the
trailing-separator behavior of non-Root operations. However,
doInRoot failed to take trailing separators into account
when checking for .. path components.
This could permit opening the parent directory of the Root
with a path ending in "../".
Change the split path to never include path separators in
components, and handle trailing separators independently
of the split path.
Thanks to Dan Sebastian Thrane of SDU eScience Center for
reporting this issue.
Fixes#73555
Fixes CVE-2025-22873
Change-Id: I9a33a145c22f5eb1dd4e4cafae5fcc61a8d4f0d4
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/2160
Reviewed-by: Neal Patel <nealpatel@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/670036
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
The runtime poller and os.NewFile recently gained support for
disassociating the handle from the runtime poller IOCP (see CL 664455).
This was the main blocker for allowing the conversion between *os.File
and net.Conn.
Implementing the conversion is now trivial. The only remaining work,
implemented in this CL, is improving os.NewFile to also support
socket handles and updating some build tags so that Windows can share
almost the same net's File implementation as Unix.
There is one important limitation, though: the duplicated socket handle
returned by the various File methods in the net package is not
usable on other process. If someone needs to pass a socket handle to
another process, they should manually call the WSADuplicateSocket
Windows API passing the process ID of the target process.
Fixes#9503.
Fixes#10350.
Updates #19098.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-race,gotip-windows-amd64-longtest,gotip-windows-arm64
Change-Id: Ic43cadaac2662b925d57a9d362ddc7ae21d1b56e
Reviewed-on: https://go-review.googlesource.com/c/go/+/668195
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
On arm64, systemstack restores the frame pointer from g0.sched to R29
prior to calling the callback. That doesn't really make any sense. The
frame pointer value in g0.sched is some arbitrary BP from a prior
context save, but that is not the caller of systemstack.
amd64 does not do this. In fact, it leaves BP completely unmodified so
frame pointer unwinders like gdb can walk through the systemstack frame
and continue traceback on the caller's stack. Unlike mcall, systemstack
always returns to the original goroutine, so that is safe.
We should do the same on arm64.
For #63630.
Cq-Include-Trybots: luci.golang.try:gotip-linux-arm64-longtest
Change-Id: I6a6a636c35d321dd5d7dc1c4d09e29b55b1ab621
Reviewed-on: https://go-review.googlesource.com/c/go/+/669236
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Nick Ripley <nick.ripley@datadoghq.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
On amd64, mcall leaves BP untouched, so the callback will push BP,
connecting the g0 stack to the calling g stack. This seems OK (frame
pointer unwinders like Linux perf can see what user code called into the
scheduler), but the "scheduler" part is problematic.
mcall is used when calling into the scheduler to deschedule the current
goroutine (e.g., in goyield). Once the goroutine is descheduled, it may
be picked up by another M and continue execution. The other thread is
mutating the goroutine stack, but our M still has a frame pointer
pointing to the goroutine stack.
A frame pointer unwinder like Linux perf could get bogus values off of
the mutating stack. Note that though the execution tracer uses
framepointer unwinding, it never unwinds a g0, so it isn't affected.
Clear the frame pointer in mcall so that unwinding always stops at
mcall.
On arm64, mcall stores the frame pointer from g0.sched.bp. This doesn't
really make any sense. mcall wasn't called by whatever used g0 last, so
at best unwinding will get misleading results (e.g., it might look like
cgocallback calls mcall?).
Also clear the frame pointer on arm64.
Other architectures don't use frame pointers.
For #63630.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-linux-arm64-longtest
Change-Id: I6a6a636cb6404f3c95ecabdb969c9b8184615cee
Reviewed-on: https://go-review.googlesource.com/c/go/+/669615
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Nick Ripley <nick.ripley@datadoghq.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
The existing implementation lacks the Status function for retrieving VCS build
information for Subversion. As a consequence, binaries aren't stamped with the
Revision, CommitTime and Uncommitted information from SVN repositories.
This change provides the svnStatus function and retrieves the information by
running svn info and svn status commands.
Fixes#73444
Change-Id: Ie6d95ffbb3a3c580cc42128ad1f8d82a869c91f2
GitHub-Last-Rev: 3472222865
GitHub-Pull-Request: golang/go#73446
Reviewed-on: https://go-review.googlesource.com/c/go/+/666875
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Sean Liao <sean@liao.dev>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Our current parallel mark algorithm suffers from frequent stalls on
memory since its access pattern is essentially random. Small objects
are the worst offenders, since each one forces pulling in at least one
full cache line to access even when the amount to be scanned is far
smaller than that. Each object also requires an independent access to
per-object metadata.
The purpose of this change is to improve garbage collector performance
by scanning small objects in batches to obtain better cache locality
than our current approach. The core idea behind this change is to defer
marking and scanning small objects, and then scan them in batches
localized to a span.
This change adds scanned bits to each small object (<=512 bytes) span in
addition to mark bits. The scanned bits indicate that the object has
been scanned. (One way to think of them is "grey" bits and "black" bits
in the tri-color mark-sweep abstraction.) Each of these spans is always
8 KiB and if they contain pointers, the pointer/scalar data is already
packed together at the end of the span, allowing us to further optimize
the mark algorithm for this specific case.
When the GC encounters a pointer, it first checks if it points into a
small object span. If so, it is first marked in the mark bits, and then
the object is queued on a work-stealing P-local queue. This object
represents the whole span, and we ensure that a span can only appear at
most once in any queue by maintaining an atomic ownership bit for each
span. Later, when the pointer is dequeued, we scan every object with a
set mark that doesn't have a corresponding scanned bit. If it turns out
that was the only object in the mark bits since the last time we scanned
the span, we scan just that object directly, essentially falling back to
the existing algorithm. noscan objects have no scan work, so they are
never queued.
Each span's mark and scanned bits are co-located together at the end of
the span. Since the span is always 8 KiB in size, it can be found with
simple pointer arithmetic. Next to the marks and scans we also store the
size class, eliminating the need to access the span's mspan altogether.
The work-stealing P-local queue is a new source of GC work. If this
queue gets full, half of it is dumped to a global linked list of spans
to scan. The regular scan queues are always prioritized over this queue
to allow time for darts to accumulate. Stealing work from other Ps is a
last resort.
This change also adds a new debug mode under GODEBUG=gctrace=2 that
dumps whole-span scanning statistics by size class on every GC cycle.
A future extension to this CL is to use SIMD-accelerated scanning
kernels for scanning spans with high mark bit density.
For #19112. (Deadlock averted in GOEXPERIMENT.)
For #73581.
Change-Id: I4bbb4e36f376950a53e61aaaae157ce842c341bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/658036
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
For riscv64/rva22u64 and above, we can intrinsify math/bits.OnesCount
using the CPOP/CPOPW machine instructions. Since the native Go
implementation of OnesCount is relatively expensive, it is also
worth emitting a check for Zbb support when compiled for rva20u64.
On a Banana Pi F3, with GORISCV64=rva22u64:
│ oc.1 │ oc.2 │
│ sec/op │ sec/op vs base │
OnesCount-8 16.930n ± 0% 4.389n ± 0% -74.08% (p=0.000 n=10)
OnesCount8-8 5.642n ± 0% 5.016n ± 0% -11.10% (p=0.000 n=10)
OnesCount16-8 9.404n ± 0% 5.015n ± 0% -46.67% (p=0.000 n=10)
OnesCount32-8 13.165n ± 0% 4.388n ± 0% -66.67% (p=0.000 n=10)
OnesCount64-8 16.300n ± 0% 4.388n ± 0% -73.08% (p=0.000 n=10)
geomean 11.40n 4.629n -59.40%
On a Banana Pi F3, compiled with GORISCV64=rva20u64 and with Zbb
detection enabled:
│ oc.3 │ oc.4 │
│ sec/op │ sec/op vs base │
OnesCount-8 16.930n ± 0% 5.643n ± 0% -66.67% (p=0.000 n=10)
OnesCount8-8 5.642n ± 0% 5.642n ± 0% ~ (p=0.447 n=10)
OnesCount16-8 10.030n ± 0% 6.896n ± 0% -31.25% (p=0.000 n=10)
OnesCount32-8 13.170n ± 0% 5.642n ± 0% -57.16% (p=0.000 n=10)
OnesCount64-8 16.300n ± 0% 5.642n ± 0% -65.39% (p=0.000 n=10)
geomean 11.55n 5.873n -49.16%
On a Banana Pi F3, compiled with GORISCV64=rva20u64 but with Zbb
detection disabled:
│ oc.3 │ oc.5 │
│ sec/op │ sec/op vs base │
OnesCount-8 16.93n ± 0% 29.47n ± 0% +74.07% (p=0.000 n=10)
OnesCount8-8 5.642n ± 0% 5.643n ± 0% ~ (p=0.191 n=10)
OnesCount16-8 10.03n ± 0% 15.05n ± 0% +50.05% (p=0.000 n=10)
OnesCount32-8 13.17n ± 0% 18.18n ± 0% +38.04% (p=0.000 n=10)
OnesCount64-8 16.30n ± 0% 21.94n ± 0% +34.60% (p=0.000 n=10)
geomean 11.55n 15.84n +37.16%
For hardware without Zbb, this adds ~5ns overhead, while for hardware
with Zbb we achieve a performance gain up of up to 11ns. It is worth
noting that OnesCount8 is cheap enough that it is preferable to stick
with the generic version in this case.
Change-Id: Id657e40e0dd1b1ab8cc0fe0f8a68df4c9f2d7da5
Reviewed-on: https://go-review.googlesource.com/c/go/+/660856
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
Reviewed-by: Mark Ryan <markdryan@rivosinc.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
It's not currently possible to build cgo programs that are partially
compiled with gcc-15 on riscv64 using the internal linker. There are
two reasons for this.
1. When gcc-15 compiles _cgo_export.c, which contains no actual code,
for a riscv64 target, it emits a label in the .text section called
.Letext0. This label is referred to by another section, .debug_line,
and an entry is generated in the symbol table for it. The Go linker
panics when processing the .Letext0 symbol in _cgo_export.o, as it
occurs in an empty section.
2. GCC-15 is generating additional debug symbols with the .LVUS
prefix, e.g., .LVUS33, that need to be ignored.
We fix the issue by removing the check in
cmd/link/internal/loader/loader.go that panics if we encounter a
symbol in an empty section (the comments preceding this check suggest
it's safe to remove it) and by adding .LVUS to the list of symbol
prefixes to ignore.
Fixes#72840
Change-Id: I00658b6bdd01606dde1581b5bc2f42edfc37de82
Reviewed-on: https://go-review.googlesource.com/c/go/+/668276
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
These were the remaining instances in the main Go repo I found where a
Go version like "devel go1.25-9ce47e66e8 Wed Mar 26 03:48:50 2025 -0700"
is considered to be a development version rather than a release version,
but the version "go1.25-devel_9ce47e66e8 Wed Mar 26 03:48:50 2025 -0700"
is not.
Update this in preparation of the move of "devel" from front to middle.
For #73372.
For #73369.
Change-Id: If5442ecb0751c08b3a1b4d1148193e501700b956
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/668355
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
NewFile recently added support for overlapped I/O on Windows,
which allows us to set deadlines on them, but the test coverage for
this new feature is not exhaustive.
Modify the existing pipe deadline tests to also exercise named
overlapped pipes.
Updates #19098.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-race,gotip-windows-amd64-longtest,gotip-windows-arm64
Change-Id: I86d284d9fb054c24959045a922cf84feeda5b5f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/668095
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Windows is unlike the other OSs and depends on a syscall for most
errors. This can be costly; cache the returned string for later reuse.
This helps test caching, since errors are written out as string to the
test ID, which are often PathErrors wrapping Errnos.
For now, only cache ERROR_FILE_NOT_FOUND and ERROR_PATH_NOT_FOUND.
goos: windows
goarch: amd64
pkg: syscall
cpu: Intel(R) Core(TM) i9-10900K CPU @ 3.70GHz
│ old.txt │ new.txt │
│ sec/op │ sec/op vs base │
ErrnoString-20 1788.00n ± 1% 11.08n ± 1% -99.38% (p=0.000 n=10)
│ old.txt │ new.txt │
│ B/op │ B/op vs base │
ErrnoString-20 48.00 ± 0% 0.00 ± 0% -100.00% (p=0.000 n=10)
│ old.txt │ new.txt │
│ allocs/op │ allocs/op vs base │
ErrnoString-20 1.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10)
For #72992
Change-Id: I9a0910fa6538772ffc64ef7670b44059a2c7d18c
Reviewed-on: https://go-review.googlesource.com/c/go/+/667495
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Carlos Amedee <carlos@golang.org>
In CL 656755, the readRandom function was modified
to read an integer from /dev/random.
However, on Plan 9, /dev/random can only return
a few hundred bits a second.
The issue is that readRandom is called by randinit,
which is called at the creation of Go processes.
Consequently, it lead the Go programs to be very
slow on Plan 9.
This change reverts the change done in CL 656755
to make the readRandom function always returning 0
on Plan 9.
Change-Id: Ibe1bf7e4c8cbc82998e4f5e1331f5e29a047c4fc
Cq-Include-Trybots: luci.golang.try:gotip-plan9-arm
Reviewed-on: https://go-review.googlesource.com/c/go/+/663195
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Richard Miller <millerresearch@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Non-release versions that are built from source without a VERSION file
specifying any particular version end up with a development version like
"devel go1.25-67e0681aef Thu Apr 24 12:17:27 2025 -0700". Right now
those versions are correctly determined to be non-release because they
don't have a "go" prefix, instead they have a "devel " prefix.
In preparation of being able to move the "devel" substring, add a check
that said substring isn't present anywhere, since it is certain not to
be included in any released Go version we publish at https://go.dev/dl/.
For #73372.
Change-Id: Ia3e0d03b5723d4034d6270c3a2224f8dfae380e9
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/667955
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
We currently make some parts of the preamble unpreemptible because
it confuses morestack. See comments in the code.
Instead, have morestack handle those weird cases so we can
remove unpreemptible marks from most places.
This CL makes user functions preemptible everywhere if they have no
write barriers (at least, on x86). In cmd/go the fraction of functions
that need preemptible markings drops from 82% to 36%. Makes the cmd/go
binary 0.3% smaller.
Update #35470
Change-Id: Ic83d5eabfd0f6d239a92e65684bcce7e67ff30bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/648518
Auto-Submit: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Go 1.25 will gain support for overlapped IO on handles passed to
os.NewFile thanks to CL 662236. It was previously not possible to add
an overlapped handle to the Go runtime's IO completion port (IOCP),
and now happens on the first call the an IO method.
This means that there is code that relies on the fact that File.Fd
returns a handle that can always be associated with a custom IOCP.
That wouldn't be the case anymore, as a handle can only be associated
with one IOCP at a time and it must be explicitly disassociated.
To fix this breaking change, File.Fd will disassociate the handle
from the Go runtime IOCP before returning it. It is then not necessary
to defer the association until the first IO method is called, which
was recently added in CL 661955 to support this same use case, but
in a more complex and unreliable way.
Updates #19098.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-race,gotip-windows-amd64-longtest,gotip-windows-arm64
Change-Id: Id8a7e04d35057047c61d1733bad5bf45494b2c28
Reviewed-on: https://go-review.googlesource.com/c/go/+/664455
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
We only want to call into the race detector for Go global variables.
By rounding up the region bounds, we can include some C globals.
Even worse, we can include only *part* of a C global, leading to
race{read,write}range calls which straddle the end of shadow memory.
That causes the race detector to barf.
Fix some off-by-one errors in the assembly comparisons. We want to
skip calling the race detector when addr == racedataend.
Fixes#73483
Change-Id: I436b0f588d6165b61f30cb7653016ba9b7cbf585
Reviewed-on: https://go-review.googlesource.com/c/go/+/667655
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Currently we assume alignment to 8 bytes, so we can steal the low 3 bits.
This CL assumes alignment to 512 bytes, so we can steal the low 9 bits.
That's 6 extra bits!
Aligning to 512 bytes wastes a bit of space but it is not egregious.
Most of the objects that we make tagged pointers to are pretty big.
Update #49405
Change-Id: I66fc7784ac1be5f12f285de1d7851d5a6871fb75
Reviewed-on: https://go-review.googlesource.com/c/go/+/665815
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
If the thing we're ranging over is an array or ptr to array, and
it doesn't have a function call or channel receive in it, then we
shouldn't evaluate it.
Typecheck the ranged-over value as a constant in that case.
That makes the unified exporter replace the range expression
with a constant int.
Change-Id: I0d4ea081de70d20cf6d1fa8d25ef6cb021975554
Reviewed-on: https://go-review.googlesource.com/c/go/+/659317
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Broadly speaking, escape analysis has two main phases. First, it
traverses the AST while building a data-flow graph of locations and
edges. Second, during "solve", it repeatedly walks the data-flow graph
while carefully propagating information about each location, including
whether a location's address reaches the heap.
Once escape analysis is in the solve phase and repeatedly walking the
data-flow graph, almost all the information it needs is within the
location graph, with a notable exception being the ir.Class of an
ir.Name, which currently must be checked by following a pointer from
the location to its ir.Node.
For typical graphs, that does not matter much, but if the graph becomes
large enough, cache misses in the inner solve loop start to matter more,
and the class is checked many times in the inner loop.
We therefore store the class information on the location in the graph
to reduce how much memory we need to load in the inner loop.
The package github.com/microsoft/typescript-go/internal/checker
has many locations, and compilation currently spends most of its time
in escape analysis.
This CL gives roughly a 30% speedup for wall clock compilation time
for the checker package:
go1.24.0: 91.79s
this CL: 64.98s
Linux perf shows a healthy reduction for example in l2_request.miss and
dTLB-load-misses on an amd64 test VM.
We could tweak things a bit more, though initial review feedback
has suggested it would be good to get this in as it stands.
Subsequent CLs in this stack give larger improvements.
Updates #72815
Change-Id: I3117430dff684c99e6da1e0d7763869873379238
Reviewed-on: https://go-review.googlesource.com/c/go/+/657295
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Jake Bailey <jacob.b.bailey@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
The arith assembly is big enough, and the details that you have to keep
in mind are complex enough and varied enough, that it is worth using
a Go program to generate the assembly. That way, all the architectures
can use the same algorithms, and porting to new architectures will be
easier.
This is the first of a sequence of CLs to introduce a new mini-compiler
for generating the arith assembly, in math/big/internal/asmgen.
This CL has the basics of the compiler as well as a couple simple
architectures and the generator for addVV/subVV. It does not check
in the generated assembly yet. That will happen in a followup CL after
the other architectures and generators have been added.
Change-Id: Ib704c60fd972fc5690ac04d8fae3712ee2c1a80a
Reviewed-on: https://go-review.googlesource.com/c/go/+/664935
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
The Unicode specification defines aliases for some of the general
category names. For example the category "L" has alias "Letter".
The regexp package supports \p{L} but not \p{Letter}, because there
was nothing in the Unicode tables that lets regexp know about Letter.
Now that package unicode provides CategoryAliases (see #70780),
we can use it to provide \p{Letter} as well.
This is the only feature missing from making package regexp suitable
for use in a JSON-API Schema implementation. (The official test suite
includes usage of aliases like \p{Letter} instead of \p{L}.)
For better conformity with Unicode TR18, also accept case-insensitive
matches for names and ignore underscores, hyphens, and spaces;
and add Any, ASCII, and Assigned.
Fixes#70781.
Change-Id: I50ff024d99255338fa8d92663881acb47f1e92a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/641377
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
This test fails on GOEXPERIMENT=noswissmap as it is testing behavior
specific to swissmaps. Move it to map_swiss_test.go to skip it on
noswissmap.
We could also switch the test to use NewTestMap, which provides a
swissmap even in GOEXPERIMENT=noswissmap, but that is tedious to use and
noswissmap is going away soon anyway.
For #70886.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest-noswissmap
Change-Id: I6a6a636c5ec72217d936cd01e9da36ae127ea2c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/666437
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This imports the proposed new v2 JSON API implemented in
github.com/go-json-experiment/json as of commit
d3c622f1b874954c355e60c8e6b6baa5f60d2fed.
When GOEXPERIMENT=jsonv2 is set, the encoding/json/v2 and
encoding/jsontext packages are visible, the encoding/json
package is implemented in terms of encoding/json/v2, and
the encoding/json package include various additional APIs.
(See #71497 for details.)
When GOEXPERIMENT=jsonv2 is not set, the new API is not
present and the encoding/json package is unchanged.
The experimental API is not bound by the Go compatibility
promise and is expected to evolve as updates are made to
the json/v2 proposal.
The contents of encoding/json/internal/jsontest/testdata
are compressed with zstd v1.5.7 with the -19 option.
Fixes#71845
For #71497
Change-Id: Ib8c94e5f0586b6aaa22833190b41cf6ef59f4f01
Reviewed-on: https://go-review.googlesource.com/c/go/+/665796
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
CL 653856 enabled stack allocation of variable-sized makeslice results.
This CL adds debug hashing of that change, plus a debug flag
to control the byte threshold used.
The debug hashing machinery means we also now have a way to disable just
the CL 653856 optimization by doing -gcflags='all=-d=variablemakehash=n'
or similar, though the stderr output will then typically have many
lines of debug hash output.
Using this CL plus the bisect command, I was able to retroactively
find one of the lines of code responsible for #73199:
$ bisect -compile=variablemake go test -skip TestListWireGuardDrivers
[...]
bisect: FOUND failing change set
--- change set #1 (enabling changes causes failure)
./security_windows.go:1321:38 (variablemake)
./security_windows.go:1321:38 (variablemake)
---
Previously, I had tracked down those lines by diffing '-gcflags=-m=1'
output and brief code inspection, but seeing the bisect was very nice.
This CL also adds a compiler debug flag to control the threshold for
stack allocation of variably sized make results. This can help
us identify more code that is relying on certain stack allocations.
This might be a temporary flag that we delete prior to Go 1.25
(given we would not want people to rely on it), or maybe it
might make sense to keep it for some period of time beyond the release
of Go 1.25 to help the ecosystem shake out other bugs.
Using these two flags together (and picking a threshold of 64 rather
than the default of 32), it looks for example like this
x/sys/windows code might be relying on stack allocation of
a byte slice:
$ bisect -compile=variablemake go test -gcflags=-d=variablemakethreshold=64 -skip TestListWireGuardDrivers
[...]
bisect: FOUND failing change set
--- change set #1 (enabling changes causes failure)
./syscall_windows_test.go:1178:16 (variablemake)
Updates #73199Fixes#73253
Change-Id: I160179a0e3c148c3ea86be5c9b6cea8a52c3e5b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/663795
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TestScan loads encoding/json and verifies that various imports
match expectations. The new v2 encoding/json violates these
expectations. Since this test is testing the ScanDir function,
not encoding/json, change it to use a test package with defined
imports instead.
Change-Id: I68a0813ccf37daadbd6ea52872a8ac132141e82a
Reviewed-on: https://go-review.googlesource.com/c/go/+/665795
Reviewed-by: Alan Donovan <adonovan@google.com>
TryBot-Bypass: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
CL 661575 inadvertently caused os.RemoveDir on Windows to
fail when given a path with a trailing / or \, due to the
splitPath function not correctly stripping trailing
separators.
Fixes#73317
Change-Id: I21977b94bb08ff1e563de6f5f16a4bdf5024a15e
Reviewed-on: https://go-review.googlesource.com/c/go/+/664715
Auto-Submit: Damien Neil <dneil@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
Before CL, all instances of gQueue and gList stored the size of
structures in a separate variable. The size changed manually and passed
as a separate argument to different functions. This CL added an
additional field to gQueue and gList structures to store the size. Also,
the calculation of size was moved into the implementation of API for
these structures. This allows to reduce possible errors by eliminating
manual calculation of the size and simplifying functions' signatures.
Change-Id: I087da2dfaec4925e4254ad40fce5ccb4c175ec41
Reviewed-on: https://go-review.googlesource.com/c/go/+/664777
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@golang.org>
The implementation of iter.Pull and iter.Pull2 functions is based on
closures and sharing local state, which results in one heap allocation
for each captured variable.
The number of heap allocations can be reduced by grouping the state
shared between closures in a struct, allowing the compiler to allocate
all local variables in a single heap region instead of creating
individual heap objects for each variable.
This approach can sometimes have downsides when it couples unrelated
objects in a single memory region, preventing the garbage collector from
reclaiming unused memory. While technically only a subset of the local
state is shared between the next and stop functions, it seems unlikely
that retaining the rest of the state until stop is reclaimed would be
problematic in practice, since the two closures would often have very
similar lifetimes.
The change also reduces the total memory footprint due to alignment
rules, the two booleans can be packed in memory and sometimes can even
exist within the padding space of the v value. There is also less
metadata needed for the garbage collector to track each individual heap
allocation.
goos: darwin
goarch: arm64
pkg: iter
cpu: Apple M2 Pro
│ /tmp/bench.old │ /tmp/bench.new │
│ sec/op │ sec/op vs base │
Pull-12 218.6n ± 7% 146.1n ± 0% -33.19% (p=0.000 n=10)
Pull2-12 239.8n ± 5% 155.0n ± 5% -35.36% (p=0.000 n=10)
geomean 229.0n 150.5n -34.28%
│ /tmp/bench.old │ /tmp/bench.new │
│ B/op │ B/op vs base │
Pull-12 288.0 ± 0% 176.0 ± 0% -38.89% (p=0.000 n=10)
Pull2-12 312.0 ± 0% 176.0 ± 0% -43.59% (p=0.000 n=10)
geomean 299.8 176.0 -41.29%
│ /tmp/bench.old │ /tmp/bench.new │
│ allocs/op │ allocs/op vs base │
Pull-12 11.000 ± 0% 5.000 ± 0% -54.55% (p=0.000 n=10)
Pull2-12 12.000 ± 0% 5.000 ± 0% -58.33% (p=0.000 n=10)
geomean 11.49 5.000 -56.48%
Change-Id: Iccbe233e8ae11066087ffa4781b66489d0d410a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/552375
Reviewed-by: Sean Liao <sean@liao.dev>
Auto-Submit: Sean Liao <sean@liao.dev>
Reviewed-by: qiu laidongfeng2 <2645477756@qq.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This is a case where CL 653856 saves an allocation.
│ old │ new │
│ sec/op │ sec/op vs base │
Join-24 73.57n ± 1% 60.27n ± 1% -18.07% (p=0.000 n=10)
│ old │ new │
│ B/op │ B/op vs base │
Join-24 48.00 ± 0% 24.00 ± 0% -50.00% (p=0.000 n=10)
│ old │ new │
│ allocs/op │ allocs/op vs base │
Join-24 2.000 ± 0% 1.000 ± 0% -50.00% (p=0.000 n=10)
Change-Id: I56308262ca73a7ab9698b54fd8681f5b44626995
Reviewed-on: https://go-review.googlesource.com/c/go/+/665075
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
When the compiler builds a Go package with DWARF 5 generation enabled,
it emits relocations into various generated DWARF symbols (ex:
SDWARFFCN) that use the R_DWTXTADDR_* flavor of relocations. The
specific size of this relocation is selected based on the total number
of functions in the package -- if the package is tiny (just a couple
funcs) we can use R_DWTXTADDR_U1 relocs (which target just a byte); if
the package is larger we might need to use the 2-byte or 3-byte flavor
of this reloc.
Prior to this patch, the strategy used to pick the right relocation
size was flawed in that it didn't take into account packages with
assembly code. For example, if you have a package P with 200 funcs
written in Go source and 200 funcs written in assembly, you can't use
the R_DWTXTADDR_U1 reloc flavor for indirect text references since the
real function count for the package (asm + go) exceeds 255.
The new strategy (with this patch) is to have the compiler look at the
"symabis" file to determine the count of assembly functions. For the
assembler, rather than create additional plumbing to pass in the Go
source func count we just use an dummy (artificially high) function
count so as to select a relocation that will be large enough.
Fixes#72810.
Updates #26379.
Change-Id: I98d04f3c6aacca1dafe1f1610c99c77db290d1d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/663235
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
It is annoying that non-x86 implementations of shlVU and shrVU
have to go out of their way to handle the trivial case shift==0
with their own copy loops. Instead, arrange to never call them
with shift==0, so that the code can be removed.
Unfortunately, there are linknames of shlVU, so we cannot
change that function. But we can rename the functions and
then leave behind a shlVU wrapper, so do that.
Since the big.Int API calls the operations Lsh and Rsh, rename
shlVU/shrVU to lshVU/rshVU. Also rename various other shl/shr
methods and functions to lsh/rsh.
Change-Id: Ieaf54e0110a298730aa3e4566ce5be57ba7fc121
Reviewed-on: https://go-review.googlesource.com/c/go/+/664896
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
addMulVVW is an unnecessarily special case.
All other assembly routines taking []Word (V as in vector) arguments
take separate source and destination. For example:
addVV: z = x+y
mulAddVWW: z = x*m+a
addMulVVW uses the z parameter as both destination and source:
addMulVVW: z = z+x*m
Even looking at the signatures is confusing: all the VV routines take
two input vectors x and y, but addMulVVW takes only x: where is y?
(The answer is that the two inputs are z and x.)
It would be nice to fix this, both for understandability and regularity,
and to simplify a future assembly generator.
We cannot remove or redefine addMulVVW, because it has been used
in linknames. Instead, the CL adds a new final addend argument ‘a’
like in mulAddVWW, making the natural name addMulVVWW
(two input vectors, two input words):
addMulVVWW: z = x+y*m+a
This CL updates all the assembly implementations to rename the
inputs z, x, y -> x, y, m, and then introduces a separate destination z.
Change-Id: Ib76c80b53f6d1f4a901f663566e9c4764bb20488
Reviewed-on: https://go-review.googlesource.com/c/go/+/664895
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
The sendfile implementation for platforms supporting it is now in
net/sendfile.go, rather than being duplicated in separate files for
each platform.
The only difference between the implementations was the poll.SendFile
parameters, which have been harmonized, and also linux strictly
asserting for os.File, which now have been relaxed to allow any
type implementing syscall.Conn.
Change-Id: Ia1a2d5ee7380710a36fc555dbf681f7e996ea2ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/664075
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Quim Muntal <quimmuntal@gmail.com>
Windows sendfile optimization is skipped since CL 472475, which started
passing an os.fileWithoutWriteTo instead of an os.File to sendfile,
and that function was only implemented for os.File.
This CL fixes the issue by asserting against an interface rather than
a concrete type.
Some tests have been reenabled, triggering bugs in poll.SendFile which
have been fixed in this CL.
Fixes#67042.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest
Change-Id: Id6f7a0e1e0f34a72216fa9d00c5bf36f5a994219
Reviewed-on: https://go-review.googlesource.com/c/go/+/664055
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
The following sequence in the scheduler may potentially lead to
deadlock:
- globrunqget() -> runqput() -> runqputslow() -> globrunqputbatch()
However, according to the current logic of the scheduler it is not
possible to face the deadlock.
The patch explicitly excludes the deadlock, even though it is impossible
situation at the moment.
Additionally, the "runq" and "globrunq" APIs were partially refactored,
which allowed to minimize the usage of these APIs by each other.
This will prevent situations described in the CL.
Change-Id: I7318f935d285b95522998e0903eaa6193af2ba48
Reviewed-on: https://go-review.googlesource.com/c/go/+/662216
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
CL 518776 dropped the ability of 'go mod init' to convert
legacy pre-module dependency configuration files, such as automatically
transforming a Gopkg.lock to a go.mod file with similar requirements,
but some of the documentation remained.
In this CL, we remove it from the cmd/go documentation.
(CL 662675 is a companion change that removes it from the Modules
Reference page).
Updates #71537
Change-Id: Ieccc64c811c4c25a657c00e42f7362a32b5fd661
Reviewed-on: https://go-review.googlesource.com/c/go/+/662695
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Sean Liao <sean@liao.dev>
Auto-Submit: Sean Liao <sean@liao.dev>
Previously, Hijack allocated a new write buffer and the existing
connection write buffer used an extra 4KiB of memory until the handler
finished and the "conn" was garbage collected. Now, hijack re-uses the
existing write buffer and re-attaches it to the raw connection to avoid
referencing the net/http "conn" after returning.
After a handler that hijacked exited, the "conn" reference in
"connReader" will now be unset. This allows all of the "conn",
"response" and "Request" to get garbage collected.
Overall, this is reducing the memory usage by 43% or 6.7KiB per hijacked
connection (see BenchmarkServerHijackMemoryUsage in an earlier revision
of the CL).
CloseNotify will continue to work _before_ the handler has exited
(i.e. while the "conn" is still referenced in "connReader"). This aligns
with the documentation of CloseNotifier:
> After the Handler has returned, there is no guarantee that the channel
> receives a value.
goos: linux
goarch: amd64
pkg: net/http
cpu: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
│ before │ after │
│ sec/op │ sec/op vs base │
ServerHijack-8 42.59µ ± 8% 39.47µ ± 16% ~ (p=0.481 n=10)
│ before │ after │
│ B/op │ B/op vs base │
ServerHijack-8 16.12Ki ± 0% 12.06Ki ± 0% -25.16% (p=0.000 n=10)
│ before │ after │
│ allocs/op │ allocs/op vs base │
ServerHijack-8 51.00 ± 0% 49.00 ± 0% -3.92% (p=0.000 n=10)
Change-Id: I20a37ee314ed0d47463a4657d712154e78e48138
GitHub-Last-Rev: 80f09dfa27
GitHub-Pull-Request: golang/go#70756
Reviewed-on: https://go-review.googlesource.com/c/go/+/634855
Reviewed-by: Sean Liao <sean@liao.dev>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Sean Liao <sean@liao.dev>
There is a potential race between a concurrent call to FD.initIO, which
calls FD.pd.init, and a call to FD.Close, which calls FD.pd.evict.
This is solved by calling FD.initIO in FD.Close, as that will block
until the concurrent FD.initIO has completed. Note that FD.initIO is
no-op if first called from here.
The race window is so small that it is not possible to write a test
that triggers it.
Change-Id: Ie2f2818e746b9d626fe3b9eb6b8ff967c81ef863
Reviewed-on: https://go-review.googlesource.com/c/go/+/663815
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
This CL removes some unnecessary code and duplicated NewFile tests
cases.
It also simplifies TestPipeCanceled by removing the need for using
SetReadDeadline. Using CancelIoEx instead of CancelIo makes the cancel
operations to finish almost instantly. The latter could take more than
20s to finish if called from a thread different from the one that
called ReadFile.
Change-Id: I9033cbcad277666bc2aec89b3e5a3ef529da2cd8
Reviewed-on: https://go-review.googlesource.com/c/go/+/663755
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Windows' _PROC_THREAD_ATTRIBUTE_LIST can contain pointers to memory
owned by Go, but the GC is not aware of this. This can lead to the
memory being freed while the _PROC_THREAD_ATTRIBUTE_LIST is still in
use.
This CL uses the same approach as in x/sys/windows to ensure that the
attributes are not collected by the GC.
Fixes#73170.
Updates #73199.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest
Change-Id: I7dca8d386aed4c02fdcd4a631d0fa4dc5747a96f
Reviewed-on: https://go-review.googlesource.com/c/go/+/663715
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
We were not passing the module path to newCodeRepo which caused it to
incorrectly parse the major version. This allowed v0 and v1 modules to
work because an empty major version is allowed in that case.
Additionally we need to pass the root module path to derive the correct tag
for subdirectories.
Fixes: #72877Fixes: #71738
Change-Id: Id792923f426858513972e713623270edbc76c545
Reviewed-on: https://go-review.googlesource.com/c/go/+/661875
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
The runtime/poll package has just gained support for overlapped IO,
see CL 660595 and CL 661955. The only remaining piece was making it
visible to user code via os.NewFile.
Some of the poll.FD.Init responsibility has been moved to os.NewFile
to avoid unnecessary syscalls for the common case of using os.Open,
os.Create, os.OpenFile, and os.Pipe, where we know that the file
is not opened for overlapped IO.
Some internal/poll tests have been moved to the os package to exercise
public APIs rather than internal ones.
The os.NewFile function definition has been moved into an OS-agnostic
file to avoid having duplicated documentation and ensure that the
caller is aware of its behavior across all platforms.
Closes#19098.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-amd64-race,gotip-windows-arm64
Change-Id: If043f8b34d588cd4b481777203107ed92d660fd9
Reviewed-on: https://go-review.googlesource.com/c/go/+/662236
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Damien Neil <dneil@google.com>
Instead of always allocating variable-sized "make" calls on the heap,
allocate a small, constant-sized array on the stack and use that array
as the backing store if it is big enough.
Requires the result of the "make" doesn't escape.
if cap <= K {
var arr [K]E
slice = arr[:len:cap]
} else {
slice = makeslice(E, len, cap)
}
Pretty conservatively for now, K = 32/sizeof(E). The slice header is
already 24 bytes, so wasting 32 bytes of stack if the requested size
is too big isn't that bad. Larger would waste more stack space but
maybe avoid more allocations.
This CL also requires the element type be pointer-free. Maybe we
could relax that at some point, but it is hard. If the element type
has pointers we can get heap->stack pointers (in the case where the
requested size is too big and the slice is heap allocated).
Note that this only handles the case of makeslice called directly from
compiler-generated code. It does not handle slices built in the
runtime on behalf of the program (e.g. in growslice). Some of those
are currently handled by passing in a tmpBuf (e.g. concatstrings),
but we could probably do more.
Change-Id: I8378efad527cd00d25948a80b82a68d88fbd93a1
Reviewed-on: https://go-review.googlesource.com/c/go/+/653856
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Improve the compiler's store-to-load forwarding optimization by relaxing the
type comparison condition. Instead of requiring exact type equality (CMPeq),
we now use copyCompatibleType which allows forwarding between compatible
types where safe.
Fix several size comparison bugs in the nested store patterns. Previously,
we were comparing the size of the outer store with the load type,
rather than comparing with the size of the actual store being forwarded
from.
Skip OpConvert in dead store elimination to help get rid of dead stores such
as zeroing slices. OpConvert, like OpInlMark, doesn't really use the memory.
This optimization is particularly beneficial for code that creates slices with
computed pointers, such as the runtime's heapBitsSlice function, where
intermediate calculations were previously causing the compiler to miss
store-to-load forwarding opportunities.
Local sweet run result on an x86_64 laptop:
│ Orig.res │ Hopt.res │
│ sec/op │ sec/op vs base │
BiogoIgor-8 5.303 ± 1% 5.322 ± 1% ~ (p=0.190 n=10)
BiogoKrishna-8 7.894 ± 1% 7.828 ± 2% ~ (p=0.190 n=10)
BleveIndexBatch100-8 2.257 ± 1% 2.248 ± 2% ~ (p=0.529 n=10)
EtcdPut-8 30.12m ± 1% 30.03m ± 1% ~ (p=0.796 n=10)
EtcdSTM-8 127.1m ± 1% 126.2m ± 0% -0.74% (p=0.023 n=10)
GoBuildKubelet-8 52.21 ± 0% 52.05 ± 1% ~ (p=0.063 n=10)
GoBuildKubeletLink-8 4.342 ± 1% 4.305 ± 0% -0.85% (p=0.000 n=10)
GoBuildIstioctl-8 43.33 ± 0% 43.24 ± 0% -0.22% (p=0.015 n=10)
GoBuildIstioctlLink-8 4.604 ± 1% 4.598 ± 0% ~ (p=0.063 n=10)
GoBuildFrontend-8 15.33 ± 0% 15.29 ± 0% ~ (p=0.143 n=10)
GoBuildFrontendLink-8 740.0m ± 1% 737.7m ± 1% ~ (p=0.912 n=10)
GopherLuaKNucleotide-8 9.590 ± 1% 9.656 ± 1% ~ (p=0.165 n=10)
MarkdownRenderXHTML-8 96.97m ± 1% 97.26m ± 2% ~ (p=0.105 n=10)
Tile38QueryLoad-8 335.9µ ± 1% 335.6µ ± 1% ~ (p=0.481 n=10)
geomean 1.336 1.333 -0.22%
Change-Id: I031552623e6d5a3b1b5be8325e6314706e45534f
Reviewed-on: https://go-review.googlesource.com/c/go/+/662075
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Add a regression test similar to the reproducer from #73141 to try to
help catch future issues with vgetrandom and thread exit. Though the
test isn't very precise, it just hammers thread exit.
When the test reproduces #73141, it simply crashes with a SIGSEGV and no
output or stack trace, which would be very unfortunate on a builder.
https://go.dev/issue/49165 tracks collecting core dumps from builders,
which would make this more tractable to debug.
For #73141.
Change-Id: I6a6a636c7d7b41e2729ff6ceb30fd7f979aa9978
Reviewed-on: https://go-review.googlesource.com/c/go/+/662636
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
The unsafe.Pointer -> uintptr conversion must happen when calling
syscall.Syscall, not when calling the auto-generated wrapper function,
else the Go compiler doesn't know that it has to keep the pointer alive.
This can cause undefined behavior and stack corruption.
Fixes#73135.
Fixes#73112 (potentially).
Fixes#73128 (potentially).
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-race
Change-Id: Ib3ad8b99618d8997bfd0742c0e44aeda696856c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/662575
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Due to a flaw in the %GOROOT_BOOTSTRAP% detection logic, the last Go
executable found by `where go` was taking precedence over the first one.
In batch scripts, environment variable expansion happens when each line
of the script is read, not when it is executed. Thus, the check in the
loop for GOROOT_BOOTSTRAP being unset would always be true, even when
the variable had been set in a previous loop iteration.
See SET /? for more information.
Change-Id: I15ddcbe771a902acb47a1f07ba7f4cb8a311e0dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/653535
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
When an M is destroyed, we put its vgetrandom state back on the shared
list for another M to reuse. This list is simply a slice, so appending
to the slice may allocate. Currently this operation is performed in
mdestroy, after the P is released, meaning allocation is not allowed.
More the cleanup earlier in mdestroy when allocation is still OK.
Also add //go:nowritebarrierrec to mdestroy since it runs without a P,
which would have caught this bug.
Fixes#73141.
Change-Id: I6a6a636c3fbf5c6eec09d07a260e39dbb4d2db12
Reviewed-on: https://go-review.googlesource.com/c/go/+/662455
Reviewed-by: Jason Donenfeld <Jason@zx2c4.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
When both a direct call and an interface call appear on the same line,
PGO devirtualization may make a suboptimal decision. In some cases,
the directly called function becomes a candidate for devirtualization
if no other relevant outgoing edges with non-zero weight exist for the
caller's IRNode in the WeightedCG. The edge to this candidate is
considered the hottest. Despite having zero weight, this edge still
causes the interface call to be devirtualized.
This CL prevents devirtualization when the weight of the hottest edge
is 0.
Fixes#72092
Change-Id: I06c0c5e080398d86f832e09244aceaa4aeb98721
Reviewed-on: https://go-review.googlesource.com/c/go/+/655475
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Defer the association of the IOCP to the handle until the first
I/O operation is performed.
A handle can only be associated with one IOCP at a time, so this allows
external code to associate the handle with their own IOCP and still be
able to use a FD (through os.NewFile) to pass the handle around
(e.g. to a child process standard input, output, and error) without
having to worry about the IOCP association.
This CL doesn't change any user-visible behavior, as os.NewFile still
initializes the FD as non-pollable.
For #19098.
Change-Id: Id22a49846d4fda3a66ffcc0bc1b48eb39b395dc5
Reviewed-on: https://go-review.googlesource.com/c/go/+/661955
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Open /dev/bintime at process start on Plan 9,
marked close-on-exec, hold it open for the duration of the
process, and use it for obtaining time.
The change to using /dev/bintime also sets up for an upcoming
Plan 9 change to add monotonic time to that file. If the monotonic
field is available, then nanotime1 and time.now use that field.
Otherwise they fall back to using Unix nanoseconds as "monotonic",
as they always have.
Before this CL, monotonic time went backward any time
aux/timesync decided to adjust the system's time-of-day backward.
Also use /dev/random for randomness (once at startup).
Before this CL, there was no real randomness in the runtime
on Plan 9 (the crypto/rand package still had some). Now there will be.
Change-Id: I0c20ae79d3d96eff1a5f839a56cec5c4bc517e61
Reviewed-on: https://go-review.googlesource.com/c/go/+/656755
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Bypass: Russ Cox <rsc@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Calling syscall.ReadFile and syscall.WriteFile on overlapped handles
always need to be passed a valid *syscall.Overlapped structure, even if
the handle is not added to a IOCP (like the Go runtime poller). Else,
the syscall will fail with ERROR_INVALID_PARAMETER.
We also need to handle ERROR_IO_PENDING errors when the overlapped
handle is not added to the poller, in which case we need to block until
the operation completes.
Previous CLs already added support for overlapped handles to the poller,
mostly to keep track of the file offset independently of the file
pointer (which is not supported for overlapped handles).
Fixed#15388.
Updates #19098.
Change-Id: I2103ab892a37d0e326752ae8c2771a43c13ba42e
Reviewed-on: https://go-review.googlesource.com/c/go/+/661795
Auto-Submit: Quim Muntal <quimmuntal@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
On master, lookups on small Swiss Table maps (<= 8 elements) for
non-specialized key types are seemingly a performance regression
compared to the Go 1.23 map implementation (reported in #70849).
Currently, a linear scan is used for gets in these cases.
This CL changes (*Map).getWithKeySmall to instead use the SIMD or SWAR
match on the control bytes to then jump to candidate matching slots,
with sample results below for a 16-byte key. This especially helps the
hit case when the key is unpredictable, which previously had to scan an
unpredictable number of control bytes to find a candidate slot when the
key is unpredictable.
Separately, other CLs in this stack modify the main Swiss Table
benchmarks to randomize lookup key order (vs. previously most of the
benchmarks had a repeating lookup key ordering, which likely is
predictable until the map is too big). We have sample results for the
randomized key order benchmarks followed by results from the older
benchmarks.
The first table below is with randomized key order. For hits, the older
results get slower as there are more elements. With this CL, we see hits
for unpredictable key ordering (sizes 2-8) get a ~1.7x speedup from
~25ns to ~14ns, with a now consistent lookup time for the different
sizes. (The 1 element size map has a predictable key ordering because
there is only one key, and that reports a modest ~0.5ns or ~3%
performance penalty). Misses for unpredictable key order get a ~1.3x
speedup, from ~13ns to ~10ns, with similar results for the 1 element
size.
│ no-fix-new-bmarks │ fix-with-new-bmarks │
│ sec/op │ sec/op vs base │
MapSmallAccessHit/Key=smallType/Elem=int32/len=1-4 13.26n ± 0% 13.64n ± 0% +2.90% (p=0.000 n=20)
MapSmallAccessHit/Key=smallType/Elem=int32/len=2-4 19.47n ± 0% 13.62n ± 0% -30.05% (p=0.000 n=20)
MapSmallAccessHit/Key=smallType/Elem=int32/len=3-4 22.23n ± 0% 13.64n ± 0% -38.68% (p=0.000 n=20)
MapSmallAccessHit/Key=smallType/Elem=int32/len=4-4 23.98n ± 0% 13.64n ± 0% -43.11% (p=0.000 n=20)
MapSmallAccessHit/Key=smallType/Elem=int32/len=5-4 25.02n ± 0% 13.67n ± 0% -45.35% (p=0.000 n=20)
MapSmallAccessHit/Key=smallType/Elem=int32/len=6-4 25.77n ± 1% 13.68n ± 2% -46.89% (p=0.000 n=20)
MapSmallAccessHit/Key=smallType/Elem=int32/len=7-4 26.38n ± 0% 13.64n ± 0% -48.28% (p=0.000 n=20)
MapSmallAccessHit/Key=smallType/Elem=int32/len=8-4 26.31n ± 0% 13.71n ± 21% -47.90% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=1-4 13.055n ± 0% 9.815n ± 0% -24.82% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=2-4 13.070n ± 0% 9.813n ± 0% -24.92% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=3-4 13.060n ± 0% 9.819n ± 0% -24.82% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=4-4 13.075n ± 0% 9.816n ± 0% -24.92% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=5-4 13.060n ± 0% 9.826n ± 0% -24.76% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=6-4 13.095n ± 19% 9.834n ± 31% -24.90% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=7-4 13.075n ± 19% 9.822n ± 27% -24.88% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=8-4 13.11n ± 16% 12.14n ± 19% -7.43% (p=0.000 n=20)
The next table uses the original benchmarks from just before this CL
stack (i.e., without shuffling lookup keys).
With this CL, we see improvement that is directionally similar to the
above results but not as large, presumably because the branches in the
linear scan are fairly predictable with predictable keys. (The numbers
here also include the time from a mod in the benchmark code, which
seemed to take around ~1/3 of CPU time based on spot checking a couple
of examples, vs. the modified benchmarks shown above have removed that
mod).
│ master-8c3e391573 │ just-fix-with-old-bmarks │
│ sec/op │ sec/op vs base │
MapSmallAccessHit/Key=smallType/Elem=int32/len=1-4 20.85n ± 0% 21.69n ± 0% +4.03% (p=0.000 n=20)
MapSmallAccessHit/Key=smallType/Elem=int32/len=2-4 21.22n ± 0% 21.70n ± 0% +2.24% (p=0.000 n=20)
MapSmallAccessHit/Key=smallType/Elem=int32/len=3-4 21.73n ± 0% 21.71n ± 0% ~ (p=0.158 n=20)
MapSmallAccessHit/Key=smallType/Elem=int32/len=4-4 22.06n ± 0% 21.71n ± 0% -1.56% (p=0.000 n=20)
MapSmallAccessHit/Key=smallType/Elem=int32/len=5-4 22.41n ± 0% 21.73n ± 0% -3.01% (p=0.000 n=20)
MapSmallAccessHit/Key=smallType/Elem=int32/len=6-4 22.71n ± 0% 21.72n ± 0% -4.38% (p=0.000 n=20)
MapSmallAccessHit/Key=smallType/Elem=int32/len=7-4 22.98n ± 0% 21.71n ± 0% -5.53% (p=0.000 n=20)
MapSmallAccessHit/Key=smallType/Elem=int32/len=8-4 23.20n ± 0% 21.72n ± 0% -6.36% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=1-4 19.95n ± 0% 17.30n ± 0% -13.28% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=2-4 19.96n ± 0% 17.31n ± 0% -13.28% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=3-4 19.95n ± 0% 17.29n ± 0% -13.33% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=4-4 19.95n ± 0% 17.30n ± 0% -13.29% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=5-4 19.96n ± 25% 17.32n ± 0% -13.22% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=6-4 19.99n ± 24% 17.29n ± 0% -13.51% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=7-4 19.97n ± 20% 17.34n ± 16% -13.14% (p=0.000 n=20)
MapSmallAccessMiss/Key=smallType/Elem=int32/len=8-4 20.02n ± 11% 17.33n ± 14% -13.44% (p=0.000 n=20)
geomean 21.02n 19.39n -7.78%
See #70849 for additional benchmark results, including results for arm64
(which also means without SIMD support).
Updates #54766
Updates #70700Fixes#70849
Change-Id: Ic2361bb6fc15b4436d1d1d5be7e4712e547f611b
Reviewed-on: https://go-review.googlesource.com/c/go/+/634396
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
With recent LLVM toolchain, on macOS/AMD64, the race detector syso
file built from it contains X86_64_RELOC_SUBTRACTOR relocations,
which the Go linker currently doesn't handle in internal linking
mode. To ensure internal linking mode continue to work with the
race detector syso, this CL adds support of X86_64_RELOC_SUBTRACTOR
relocations.
X86_64_RELOC_SUBTRACTOR is actually a pair of relocations that
resolves to the difference between two symbol addresses (each
relocation specifies a symbol). For the cases we care (the race
syso), the symbol being subtracted out is always in the current
section, so we can just convert it to a PC-relative relocation,
with the addend adjusted. If later we need the more general form,
we can introduce a new mechanism (say, objabi.R_DIFF) that works
as a pair of relocations like the Mach-O one.
As we expect the pair of relocations be consecutive, don't reorder
(sort) relocation records when loading Mach-O objects.
Change-Id: I757456b07270fb4b2a41fd0fef67a2b39dd6b238
Reviewed-on: https://go-review.googlesource.com/c/go/+/660715
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
FD.Read converts a syscall.ERROR_OPERATION_ABORTED error to
ErrFileClosing. It does that in case the pipe operation was aborted by
a CancelIoEx call in FD.Close.
It doesn't take into account that the operation might have been
aborted by a CancelIoEx call in external code. In that case, the
operation should return the error as is.
Change-Id: I75dcf0edaace8b57dc47b398ea591ca9f116112b
Reviewed-on: https://go-review.googlesource.com/c/go/+/661555
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
The docs currently are imprecise about comparisons. This could lead
users to believe that objects of the same type, allocated at the same
address, could produce weak pointers that are equal to
previously-created weak pointers. This is not the case. Weak pointers
map to objects, not addresses.
Update the documentation to state precisely that if two pointers do not
compare equal, then two weak pointers created from those two pointers
are guaranteed not to compare equal. Since a future pointer pointing to
the same address is not comparable with a pointer produced *before* an
object at that address has been reclaimed, this is sufficient to explain
that weak pointers map 1:1 with object offsets, not addresses.
(An object slot cannot be reused unless that slot is unreachable, so
by construction, there's never an opportunity to compare an "old" and
"new" pointer unless one uses unsafe tricks that violate the
unsafe.Pointer rules.)
Fixes#71381.
Change-Id: I5509fd433cde013926d725694d480c697a8bc911
Reviewed-on: https://go-review.googlesource.com/c/go/+/643935
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
When two packages declare a variable with the same name (with
linkname at least on one side), the linker will choose one as the
actual definition of the symbol if one has content (i.e. a DATA
symbol) and the other does not (i.e. a BSS symbol). When both have
content, it is redefinition error. When neither has content,
currently the choice is sort of arbitrary (depending on symbol
loading order, etc. which are subject to change).
One use case for that is that one wants to reference a symbol
defined in another package, and the reference side just wants to
see some of the fields, so it may be declared with a smaller type.
In this case, we want to choose the one with the larger size as
the true definition. Otherwise the code accessing the larger
sized one may read/write out of bounds, corrupting the next
variable. This CL makes the linker do so.
Fixes#72032.
Change-Id: I160aa9e0234702066cb8f141c186eaa89d0fcfed
Reviewed-on: https://go-review.googlesource.com/c/go/+/660696
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Than McIntosh <thanm@golang.org>
Empty writes might be important for some protocols. Let Windows decide
what do with them rather than skipping them on our side. This is inline
with the behavior of other platforms.
While here, refactor the Read/Write/Pwrite methods to reduce one
indentation level and make the code easier to read.
Fixes#73084.
Change-Id: Ic5393358e237d53b8be6097cd7359ac0ff205309
Reviewed-on: https://go-review.googlesource.com/c/go/+/661435
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Optimise more branches with zero on riscv64. In particular, BLTU with
zero occurs with IsInBounds checks for index zero. This currently results
in two instructions and requires an additional register:
li t2, 0
bltu t2, t1, 0x174b4
This is equivalent to checking if the bounds is not equal to zero. With
this change:
bnez t1, 0x174c0
This removes more than 500 instructions from the Go binary on riscv64.
Change-Id: I6cd861d853e3ef270bd46dacecdfaa205b1c4644
Reviewed-on: https://go-review.googlesource.com/c/go/+/606715
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
This CL adds support for async file operations on Windows. The affected
functions are Read, Write, Pread, and Pwrite.
The code has been slightly refactored to avoid duplication. Both the
async and sync variants follow the same code path, with the exception of
the async variant passes an overlapped structure to the syscalls
and supports the use of a completion port.
This doesn't change any user-facing behavior, as the os package still
sets the pollable parameter to false when calling FD.Init.
For #19098.
Change-Id: Iead6e51fa8f57e83456eb5ccdce28c2ea3846cc2
Reviewed-on: https://go-review.googlesource.com/c/go/+/660595
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Before this change, in several cases where HasModRoot() returned false,
we'd return ErrNoModRoot. ErrNoModRoot would say that there was no
go.mod file but would not mention workspaces. With this change,
ErrNoModRoot will return error text that's different if we're in a
workspace, saying that there are no modules in the workspace.
Fixes#54419
Change-Id: I77c94d0011947bf8e33c066416ab3762502fd2e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/660675
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Setting the SIO_UDP_CONNRESET option in internal/poll.FD.Init
adds unnecessary complexity to the FD.Init signature and
implementation. Better to set it in the net package when initializing
the UDP connection, which is where conceptually it belongs.
While here, update an outdated comment in FD.Init that said the runtime
poller doesn't support I/O operations initialized by the user
outside the internal/poll package. It does support those operations
since CL 561895.
For #19098.
Updates #21172.
Change-Id: I9a70b0deafdb4619830abe2147e2d366b4c2b890
Reviewed-on: https://go-review.googlesource.com/c/go/+/660496
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Currently, if the user stops the timer in a B.Loop benchmark loop, the
benchmark will run until it hits the timeout and fails.
Fix this by detecting that the timer is stopped and failing the
benchmark right away. We avoid making the fast path more expensive for
this check by "poisoning" the B.Loop iteration counter when the timer
is stopped so that it falls back to the slow path, which can check the
timer.
This causes b to escape from B.Loop, which is totally harmless because
it was already definitely heap-allocated. But it causes the
test/inline_testingbloop.go errorcheck test to fail. I don't think the
escape messages actually mattered to that test, they just had to be
matched. To fix this, we drop the debug level to -m=1, since -m=2
prints a lot of extra information for escaping parameters that we
don't want to deal with, and change one error check to allow b to
escape.
Fixes#72971.
Change-Id: I7d4abbb1ec1e096685514536f91ba0d581cca6b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/659657
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently, if a benchmark function returns prior to B.Loop() returning
false, we'll report a bogus result. While there was no way to detect
this with b.N-style benchmarks, one way b.Loop()-style benchmarks are
more robust is that we *can* detect it.
This CL adds a flag to B that tracks if B.Loop() has finished and
checks it after the benchmark completes. If there was an early exit
(not caused by another error), it reports a B.Error.
Fixes#72933.
Updates #72971.
Change-Id: I731c1350e6df938c0ffa08fcedc11dc147e78854
Reviewed-on: https://go-review.googlesource.com/c/go/+/659656
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
This CL changes the toolchain selection behavior for go install pkg@v
and go run pkg@v to also take into account the go and toolchain version
lines in the containing go.mod and go.work file.
Before this change, the go command would detect that go install
pkg@version or go run pkg@version was being run and skip the standard
behavior that would select the toolchain based on the go version in the
go.mod or go.work file. It would instead check the go line of the module
being downloaded and switch to that version if necessary.
With this change, the go command does not skip the standard behavior. It
proceeds to determine if an upgrade is required based on the containing
go.mod or go.work file's go and toolchain lines. Then, it checks the
module being installed to see if it would require a higher version than
the determined upgrade (or the local version if no upgrade was
determined). If it does require a higher version, then a switch happens
to that version, and if not the upgrade logic proceeds as usual doing
the upgrade if one was determined.
Fixes#66518
Change-Id: I00d96170e8713c451cc0fd2203be521585418842
Reviewed-on: https://go-review.googlesource.com/c/go/+/660035
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently, b.Loop uses b.N as the iteration count target. However,
since it updates the target as it goes, the behavior is quite
different from a b.N-style benchmark. To avoid user confusion, this CL
gives b.Loop a separate, unexported iteration count target. It ensures
b.N is 0 within the b.Loop loop to help catch misuses, and commits the
final iteration count to b.N only once the loop is done (as the
documentation states "After Loop returns false, b.N contains the total
number of iterations that ran, so the benchmark may use b.N to compute
other average metrics.")
Since there are now two variables used by b.Loop, we put them in an
unnamed struct. Also, we rename b.loopN to b.loop.i because this
variable tracks the current iteration index (conventionally "i"), not
the target (conventionally "n").
Unfortunately, a simple renaming causes B.Loop to be too large for the
inliner. Thus, we make one simplification to B.Loop to keep it under
the threshold. We're about to lean into that simplification anyway in
a follow-up CL, so this is just temporary.
Prep for #72933 and #72971.
Change-Id: Ide1c4f1b9ca37f300f3beb0e60ba6202331b183e
Reviewed-on: https://go-review.googlesource.com/c/go/+/659655
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Austin Clements <austin@google.com>
Injecting a call to a thread context is complex enough to warrant
a dedicated function so that we don't repeat the same code in multiple
places. Note that the unix sigctxt struct also follows the
same approach.
The behavior is unchanged, but the implementation semantics are now
clearer by using goarch.StackAlign instead of a mix of goarch.PtrSize,
goarch.StackAlign and hardcoded values.
While here, fix#68552.
Cq-Include-Trybots: luci.golang.try:gotip-windows-arm64
Change-Id: Ic29cd2bf322b520127fecccafd61577076945758
Reviewed-on: https://go-review.googlesource.com/c/go/+/657815
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
When an AfterFunc executes in a synctest bubble, there is a series of
happens-before relationships:
1. The AfterFunc is created.
2. The AfterFunc goroutine executes.
3. The AfterFunc goroutine returns.
4. A subsequent synctest.Wait call returns.
We were failing to correctly establish the happens-before relationship
between the AfterFunc goroutine and the AfterFunc itself being created.
When an AfterFunc executes, the G running the timer temporarily switches
to the timer heap's racectx. It then calls time.goFunc, which starts a
new goroutine to execute the timer. time.goFunc relies on the new goroutine
inheriting the racectx of the G running the timer.
Normal, non-synctest timers, execute with m.curg == nil, which causes
new goroutines to inherit the g0 racectx. We were running synctest
timers with m.curg set (to the G executing synctest.Run), so the new
AfterFunc goroutine was created using m.curg's racectx. This resulted
in us not properly establishing the happens-before relationship between
AfterFunc being called and the AfterFunc goroutine starting.
Fix this by setting m.curg to nil while executing timers.
As one additional fix, when waking a blocked bubble, wake the root
goroutine rather than a goroutine blocked in Wait if there is a
timer that can fire.
Fixes#72750
Change-Id: I2b2d6b0f17f64649409adc93c2603f720494af89
Reviewed-on: https://go-review.googlesource.com/c/go/+/658595
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Currently, if computing of the go cache directory fails it does not expose the error. Commands like go clean, exec, modindex that use go cache directory continue execution producing incorrect or no result. This patch adds an error to the return values such that it can be validated on call sites. It also introduces such validation in go clean -cache command to fail execution in case when error occurred.
Fixes#69997
Change-Id: I53fd1ec67f0a6bd8a367e785dcb145a673c084dc
GitHub-Last-Rev: e2063d10db
GitHub-Pull-Request: golang/go#70392
Reviewed-on: https://go-review.googlesource.com/c/go/+/628596
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Matloob <matloob@golang.org>
At first glance i have thought that we do not handle such case properly,
because parseBlockStmt and parseStmtList do not call call the
incNestLev. Fortunately parseStmt does, so it is detected properly.
As we don't have a test case directly for blockstmts only, i think it is
worth adding one.
Change-Id: If149b86fd90a7ee4a33c861070d1bafdd40e98ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/659455
Reviewed-by: Roland Shoemaker <roland@golang.org>
Commit-Queue: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
TestRace runs a collection of tests, some of which are expected
to fail with data races. Make TestRace more robust at detecting
when the test run is cut short, such as when a test causes
an unhandled panic.
Skip TestRaceRangeFuncIterator, which contains an unhandled panic.
This test was causing all subsequent tests to not run.
Skip TestNoRaceRangeFuncIterator, which contains an unexpected data race.
This test was not running due to the above failure.
For #72925
Change-Id: Id662375cc498ea25ae308619709768588bf6a2f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/658875
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
CL 637939 changed ReverseProxy to report errors encountered when
copying data on an hijacked connection. This is generally not useful,
and when using the default error handler results in WriteHeader
being called on a hijacked connection.
While this is harmless with standard net/http ResponseWriter
implementations, it can confuse middleware layers.
Fixes#72954
Change-Id: I21f3d3d515e114dc5c298d7dbc3796c505d3c82f
Reviewed-on: https://go-review.googlesource.com/c/go/+/659255
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Syntactically incorrect branches, such as
BEQ X5, X6, $1
BEQ X5, X6, 31(X10)
cause the assembler to panic, which they shouldn't really do. It's
better for the user to see a normal error, as reported for other
syntax errors in riscv64 assembly. The panics also prevent us
from writing negative tests for these sorts of errors.
Here we fix the issue by ensuring we generate a normal error instead
of panicking when the user provides an invalid branch target. We
also add a couple of negative tests.
Change-Id: I1da568999a75097484b61a01d418f5d4be3e04fa
Reviewed-on: https://go-review.googlesource.com/c/go/+/637316
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
The riscv64 Go assembler can output certain errors, ones produced by
instructionsForProg, multiple times. These errors are guaranteed to
be output at least twice and can appear three or more times if a
rescan is needed to recompute branch addresses. For example, the
syntactically incorrect instruction
MOV (X10), $1
will generate at least two identical errors
asm: 86076 (asm.s:21524) MOV (X10), $1: unsupported MOV
asm: 86076 (asm.s:21524) MOV (X10), $1: unsupported MOV
asm: assembly failed
In addition to confusing the user, these duplicate errors make it
difficult to write negative tests for certain types of instructions,
e.g., branches, whose duplicate errors are not always identical,
and so not ignored by endtoend_test.go.
We fix the issue by returning from preprocess if any errors have been
generated by the time we reach the end of the rescan loop. One
implication of this change is that validation errors will no longer
be reported if an error is generated earlier in the preprocess stage.
Negative test cases for validation errors are therefore moved to
their own file as the existing riscv64error.s file contains errors
generated by instructionsForProg that will now suppress the
validation errors.
Change-Id: Iffacdbefce28f44970dd5dda44990b822b8a23d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/637315
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
This change adds extra logging in the case where there's an error
removing all the files in the gomodcache using modfetch.RemoveAll.
It logs the names of the files found in GOMODCACHE as well as their
modes. The modes are included because they should all be writable by the
time we call robustio.RemoveAll.
For #68087
Change-Id: Id9ae68bf6a3392baf88ec002d08fed1faf525927
Reviewed-on: https://go-review.googlesource.com/c/go/+/658816
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Matloob <matloob@golang.org>
Don't include a monotonic time in time.Times created inside
a bubble, to avoid the confusion of different Times using
different monotonic clock epochs.
For #67434
goos: darwin
goarch: arm64
pkg: time
cpu: Apple M1 Pro
│ /tmp/bench.0 │ /tmp/bench.1 │
│ sec/op │ sec/op vs base │
Since-10 18.42n ± 2% 18.68n ± 1% ~ (p=0.101 n=10)
Until-10 18.28n ± 2% 18.46n ± 2% +0.98% (p=0.009 n=10)
geomean 18.35n 18.57n +1.20%
Change-Id: Iaf1b80d0a4df52139c5b80d4bde4410ef8a49f2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/657415
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Previously, the code only checked supportedVersions[0] for TLS 1.3
However, Chromium-based
browsers may list TLS 1.3 at different positions, causing ECH failures.
This fix:
Iterates through supportedVersions to accept connections as long as TLS 1.3 is present.
Improves ECH compatibility, ensuring Chrome, Edge, and other browsers work properly.
Fixes#71642
Change-Id: I32f4219fb6654d5cc22c7f33497c6142c0acb4f2
Reviewed-on: https://go-review.googlesource.com/c/go/+/648015
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Go 1.24 included the spinbitmutex GOEXPERIMENT for several popular
architectures, based on their native support an atomic primitive (8-bit
exchange) that aided its efficient implementation.
Move towards making the new mutex implementation permanent, so it fully
replaces the two previous (sema- and futex-based "tristate")
implementations.
For #68578
Change-Id: I888a73959df42eb0ec53875309c446675af8f09d
Reviewed-on: https://go-review.googlesource.com/c/go/+/658455
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
On riscv64, subtraction from a constant is typically implemented as an
ADDI with the negative constant, followed by a negation. However this can
lead to multiple NEG/ADDI/NEG sequences that can be optimised out.
For example, runtime.(*_panic).nextDefer currently contains:
lbu t0, 0(t0)
addi t0, t0, -8
neg t0, t0
addi t0, t0, -7
neg t0, t0
Which is now optimised to:
lbu t0, 0(t0)
addi t0, t0, -1
Change-Id: Idf5815e6db2e3705cc4a4811ca9130a064ae3d80
Reviewed-on: https://go-review.googlesource.com/c/go/+/652318
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Mark Ryan <markdryan@rivosinc.com>
Reviewed-by: David Chase <drchase@google.com>
mp.isExtraInC is intended to indicate that this M has no Go frames at
all; it is entirely executing in C.
If there was a cgocallback to Go and then a cgocall to C, such that the
leaf frames are C, that is fine. e.g., traceback can handle this fine
with SetCgoTraceback (or by simply skipping the C frames).
However, we currently mismanage isExtraInC, unconditionally setting it
on return from cgocallback. This means that if there are two levels of
cgocallback, we end up running Go code with isExtraInC set.
1. C-created thread calls into Go function 1 (via cgocallback).
2. Go function 1 calls into C function 1 (via cgocall).
3. C function 1 calls into Go function 2 (via cgocallback).
4. Go function 2 returns back to C function 1 (returning via the remainder of cgocallback).
5. C function 1 returns back to Go function 1 (returning via the remainder of cgocall).
6. Go function 1 is now running with mp.isExtraInC == true.
The fix is simple; only set isExtraInC on return from cgocallback if
there are no more Go frames. There can't be more Go frames unless there
is an active cgocall out of the Go frames.
Fixes#72870.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Change-Id: I6a6a636c4e7ba75a29639d7036c5af3738033467
Reviewed-on: https://go-review.googlesource.com/c/go/+/658035
Reviewed-by: Cherry Mui <cherryyz@google.com>
Commit-Queue: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
For various cursed reasons we need to support the BMPString and
T61String ASN.1 string encodings. These types use the defunct UCS-2 and
T.61 character encodings respectively.
This change rejects some characters when decoding BMPStrings which are
not valid in UCS-2, and properly parses T61Strings instead of treating
them as plain UTF-8.
While still not perfect, this matches the behavior of most other
implementations, particularly BoringSSL. Ideally we'd just remove
support for these ASN.1 types (particularly in crypto/x509, where we
don't actually expose any API), but doing so is likely to break some
deploy certificates which unfortunately still use these types in DNs,
despite them being deprecated since 1999/2002.
Fixes#71862
Change-Id: Ib8f392656a35171e48eaf71a200be6d7605b2f02
Reviewed-on: https://go-review.googlesource.com/c/go/+/651275
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
In CL 475375 the Go command started to generate the "preferlinkext"
token file for "strange/dangerous" compiler flags. This serves as a hint
to the Go linker whether to call the external linker or not.
Permit compiler flag used by the hermetic_cc_toolchain bzlmod.
As a side effect, it also allows these flags to appear
in #cgo directives in source code. We don't know of any cases
where that is actually useful, but it appears to be harmless
and simplifies the implementation of the internal linking change.
Fixes#72842
Change-Id: Ic6de29b535a4e2c0720f383567ea6b3c7ca4f541
Reviewed-on: https://go-review.googlesource.com/c/go/+/657575
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
This required adding a new field to SessionState for TLS 1.0–1.2, since
the key exchange is not repeated on resumption. The additional field is
unfortunately not backwards compatible because current Go versions check
that the encoding has no extra data at the end, but will cause
cross-version tickets to be ignored. Relaxed that so we can add fields
in a backwards compatible way the next time.
For the cipher suite, we check that the session's is still acceptable
per the Config. That would arguably make sense here, too: if a Config
for example requires PQ, we should reject resumptions of connections
that didn't use PQ. However, that only applies to pre-TLS 1.3
connections, since in TLS 1.3 we always do a fresh key exchange on
resumption. Since PQ is the only main differentiator between key
exchanges (aside from off-by-default non-PFS RSA, which are controlled
by the cipher suite in TLS 1.0–1.2) and it's PQ-only, we can skip that
check.
Fixes#67516
Change-Id: I6a6a465681a6292edf66c7b8df8f4aba4171a76b
Reviewed-on: https://go-review.googlesource.com/c/go/+/653315
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
This patch extends the change in CL 657175 to apply the same abbrev
selection strategy to single-range lexical scopes that we're now using
for inlined routine bodies, when DWARF 5 is in effect. Ranges are more
compact and use fewer relocation than explicit hi/lo PC values, so we
might as well always use them.
Updates #26379.
Change-Id: Ieeaddf50e82acc4866010e29af32bcd1fb3b4f02
Reviewed-on: https://go-review.googlesource.com/c/go/+/657177
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Currently we check the size difference between non-PIE and PIE binaries
without specifying a linkmode (and that is presumed to be internal).
However, on some platforms (like openbsd/arm64), the use of
-buildmode=pie results in external linking. Ensure that we only test
internally linked non-PIE against internally linked PIE and externally
linked non-PIE against externally linked PIE, avoiding unexpected
differences.
Fixes#72818
Change-Id: I7e1da0976a4b5de387a59d0d6c04f58498a8eca0
Reviewed-on: https://go-review.googlesource.com/c/go/+/657035
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@golang.org>
This PR adds an helper FileContentDisposition that builds multipart
Content-Disposition header contents with field name and file name,
escaping quotes and escape characters.
The function is then called in the related helper CreateFormFile.
The new function allows users to add other custom MIMEHeaders,
without having to rewrite the char escaping logic of field name and
file name, which is provided by the new helper.
Fixes#46771
Change-Id: Ifc82a79583feb6dd609ca1e6024e612fb58c05ce
GitHub-Last-Rev: 969f846fa9
GitHub-Pull-Request: golang/go#63324
Reviewed-on: https://go-review.googlesource.com/c/go/+/531995
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
The Go command had a behavior of writing its own toolchain name when
updating the go line in a go.mod (for example when a user runs go get
go@version). This behavior was often undesirable and the toolchain line
was often removed by users before checking in go.mod files (including in
the x/ repos). It also led to user confusion.
This change removes that behavior. A toolchain line will not be added if
one wasn't present before. The toolchain line can still be removed
though: the toolchain line must be at least the go version, so if the go
version is increased above the toolchain version, the toolchain version
will be bumped up to that go version. Then the toolchain line will then
be dropped because go <version> implies toolchain <version>.
Making this change slightly hurts reproducability because future go
commands run on the go.mod file may be run with a different toolchain
than the one that used it, but that doesn't seem to be worth the
confusion the behavior resulted in.
We expect this change will not have negative consequences, but it could
be possible, and we would like to hear from any users that depended on
the previous behavior in case we need to roll it back before the
release.
Fixes#65847
Change-Id: Id795b7f762e4f90ba0fa8c7935d03f32dfc8590e
Reviewed-on: https://go-review.googlesource.com/c/go/+/656835
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This patch changes the strategy we use in the compiler for handling
range information for inlined subroutine bodies, fixing a bug in how
this was handled for DWARF 5. The high and lo PC values being emitted
for DW_TAG_inlined_subroutine DIEs were incorrect, pointing to the
start of functions instead of the proper location. The fix in this
patch is to move to unconditionally using DW_AT_ranges for inlined
subroutines, even those with only a single range.
Background: prior to this point, if a given inlined function body had
a single contiguous range, we'd pick an abbrev entry for it with
explicit DW_AT_low_pc and DW_AT_high_pc attributes. If the extent of
the code for the inlined body was not contiguous (which can happen),
we'd select an abbrev that used a DW_AT_ranges attribute instead. This
strategy (preferring explicit hi/lo PC attrs for a single-range func)
made sense for DWARF 4, since in DWARF 4 the representation used in
the .debug_ranges section was especially heavyweight (lots of space,
lots of relocations), so having explicit hi/lo PC attrs was less
expensive.
With DWARF 5 range info is written to the .debug_rnglists section, and
the representation here is much more compact. Specifically, a single
hi/lo range can be represented using a base address in addrx format
(max of 4 bytes, but more likely 2 or 3) followed by start and
endpoints of the range in ULEB128 format. This combination is more
compact spacewise than the explicit hi/lo values, and has fewer
relocations (0 as opposed to 2).
Note: we should at some point consider applying this same strategy to
lexical scopes, since we can probably reap some of the same benefits
there as well.
Updates #26379.
Fixes#72821.
Change-Id: Ifb65ecc6221601bad2ca3939f9b69964c1fafc7c
Reviewed-on: https://go-review.googlesource.com/c/go/+/657175
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alessandro Arzilli <alessandro.arzilli@gmail.com>
Bump the required version of GDB up to 10 from 7.7 in the runtime GDB
tests, so as to ensure that we have something that can handle DWARF 5
when running tests. In theory there is some DWARF 5 support on the
version 9 release branch, but we get "Dwarf Error: DW_FORM_addrx"
errors for some archs on builders where GDB 9.2 is installed.
Updates #26379.
Change-Id: I1b7b45f8e4dd1fafccf22f2dda0124458ecf7cba
Reviewed-on: https://go-review.googlesource.com/c/go/+/656836
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
I've originally used |= and &= to setup assumptions exploitable by the
operation under test but theses have multiple issues making it poor
for this usecase:
- &= does not pass the minimum value as-is, rather always set it to 0
- |= rounds up the max value to a number of the same length with all ones set
- I've never implemented them to work with negative signed numbers
Change-Id: Ie43c576fb10393e69d6f989b048823daa02b1df8
Reviewed-on: https://go-review.googlesource.com/c/go/+/656160
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Jorropo <jorropo.pgm@gmail.com>
This patch enables the DWARF version 5 experiment by default for most
platforms that support DWARF. Note that MacOS is kept at version 4,
due to problems with CGO builds; the "dsymutil" tool from older
versions of Xcode (prior to V16) can't handle DWARF5. Similar we keep
DWARF 4 for GOOS=aix, where XCOFF doesn't appear to support the new
section subtypes in DWARF 5.
Updates #26379.
Change-Id: I5edd600c611f03ce8e11be3ca18c1e6686ac74ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/637895
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
test/decoratemappingszero.go is intended to test that
//go:debug decoratemappings=0 disables annonations.
Unfortunately, //go:debug processing is handled by cmd/go, but
cmd/internal/testdir (which runs tests from test/) generally invokes the
compiler directly, thus it does not set default GODEBUGs.
Move this test to the cmd/go script tests, alongside the similar test
for language version.
Fixes#72772.
Cq-Include-Trybots: luci.golang.try:gotip-linux-ppc64le_power10
Change-Id: I6a6a636c9d380ef984f760be5689fdc7f5cb2aeb
Reviewed-on: https://go-review.googlesource.com/c/go/+/656795
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
We previously disallowed all non-regular files being embedded. This CL
relaxes the restriction a little: if the GODEBUG embedfollowsymlinks=1
is set, we allow the leaf files being embedded (not the directories
containing them) to be symlinks. The files pointed to by the symlinks
must still be regular files.
This will be used when a Bazel build action executing the Go command is
running in a symlink-based sandbox. It's not something we want to enable
in general for now, so it's behind a GODEBUG.
Fixes#59924
Change-Id: I895be14c12de55b7d1b663d81bdda1df37d54804
Reviewed-on: https://go-review.googlesource.com/c/go/+/643215
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Rather than relying on coreString, use the new commonUnder function
to determine the argument slice element types.
Factor out this functionality, which is shared for append and copy,
into a new helper function sliceElem (similar to chanElem).
Use sliceElem for both the append and copy implementation.
As a result, the error messages for invalid copy calls are
now more detailed.
While at it, handle the special cases for append and copy first
because they don't need the slice element computation.
Finally, share the same type recording code for the special and
general cases.
As an aside, in commonUnder, be clearer in the code that the
result is either a nil type and an error, or a non-nil type
and a nil error. This matches in style what we do in sliceElem.
Change-Id: I318bafc0d2d31df04f33b1b464ad50d581918671
Reviewed-on: https://go-review.googlesource.com/c/go/+/655675
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
If we weren't resuming an existing session, and we constructed a TLS 1.3
compatible client hello, ensure the server doesn't echo back the
made up compatibility session ID if we end up handshaking for TLS 1.2.
As part of an effort to make the initial stages of a TLS 1.3 handshake
compatible with TLS 1.2 middleboxes, TLS 1.3 requires that the client
hello contain a non-empty legacy_session_id value. For anti-ossification
purposes it's recommended this ID be randomly generated. This is the
strategy the crypto/tls package takes.
When we follow this approach, but then end up negotiating TLS 1.2, the
server should not have echoed back that random ID to us. It's impossible
for the server to have had a session with a matching ID and so it is
misbehaving and it's prudent for our side to abort the handshake.
See RFC 8446 Section 4.1.2 for more detail:
https://www.rfc-editor.org/rfc/rfc8446#section-4.1.2
Adopting this behaviour allows un-ignoring the BoGo
EchoTLS13CompatibilitySessionID testcase.
Updates #72006
Change-Id: I1e52075177a13a7aa103b45498eae38d8a4c34b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/652997
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
For malformed client/server certificates in a TLS handshake send
a decode_error alert, matching BoringSSL behaviour.
Previously crypto/tls used a bad_certificate alert for this purpose.
The TLS specification is imprecise enough to allow this to be considered
a spec. justified choice, but since all other places in the protocol
encourage using decode_error for structurally malformed messages we may
as well do the same here and get some extra cross-impl consistency for
free.
This also allows un-ignoring the BoGo
GarbageCertificate-[Client|Server]-[TLS12|TLS13] tests.
Updates #72006
Change-Id: Ide45ba1602816e71c3289a60e77587266c3b9036
Reviewed-on: https://go-review.googlesource.com/c/go/+/652995
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Previously this test was skipped without a comment clarifying why. In
practice it's because crypto/tls doesn't generate GREASE extensions at
this time, and the test expects to find one in the NewSessionTicket
message extensions produced by a server.
We're already skipping some other GREASE related test as
not-yet-implemented without explicit bogo_config.json exclusion by way
of the -enable-grease flag not being implemented, however for TLS
1.3 servers the BoGo expectation is that they _always_ send GREASE, and
so the -enable-grease flag isn't provided and an explicit skip must be
used.
We should revisit this alongside implementing GREASE ext production in
general for both clients and servers.
Updates #72006
Change-Id: I8af4b555ac8c32cad42215fbf26aa0feae90fa21
Reviewed-on: https://go-review.googlesource.com/c/go/+/650717
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
When this command line flag is provided to the BoGo runner it will:
* Disable some timeouts
* Limit concurrency to 1 worker at a time
* Pass the -wait-for-debugger flag to the shim process
* Print the PID of the shim process to status output
On the shim-side, we need to react to -wait-for-debugger by sending
ourselves a SIGSTOP signal. When a debugger attaches to the shim the
process will be resumed.
This makes it possible to debug both the runner side and the shim side
of a BoGo interaction without resorting to print style debugging.
Since SIGSTOP is not a signal we can use on Windows this functionality
is limited to unix builds.
Updates #72006
Change-Id: Iafa08cf71830cdfde3e6ee4826914236e3cd7e57
Reviewed-on: https://go-review.googlesource.com/c/go/+/650737
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
When encountering alertUserCanceled in a TLS 1.3 handshake, ignore the
alert and retry reading a record. This matches existing logic for how
TLS 1.2 alertLevelWarning alerts are handled.
For broader context, TLS 1.3 removed warning-level alerts except for
alertUserCanceled (RFC 8446, § 6.1). Since at least one major
implementation (https://bugs.openjdk.org/browse/JDK-8323517)
misuses this alert, many TLS stacks now ignore it outright when seen in
a TLS 1.3 handshake (e.g. BoringSSL, NSS, Rustls).
With the crypto/tls behaviour changed to match peer implementations we
can now enable the "SendUserCanceledAlerts-TLS13" BoGo test.
"SendUserCanceledAlerts-TooMany-TLS13" remains ignored, because like
"SendWarningAlerts*" fixing the test requires some general spam
protocol message enhancements be done first.
Updates #72006
Change-Id: I570c1fa674b5a4760836c514d35ee17f746fe28d
Reviewed-on: https://go-review.googlesource.com/c/go/+/650716
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
This commit removes SkipNewSessionTicket from the bogo_config.json
excluded tests list.
Previously this test was being skipped with a TODO that there might be
a bug here. In practice it seems like there's no bug and the test is
handled correctly by crypto/tls.
When activated, a TLS 1.2 client connecting to the bogo dispatcher goes
through the normal handshake process with the exception that the server
skips sending the NewSessionTicket msg expected by the client in
response to the client's final flight of handshake msgs.
The crypto/tls TLS 1.2 client_handshake.go logic correctly rejects the
unexpected message that follows (ChangeCipherSpec) when trying to read
the bytes necessary to unmarshal the expected NewSessionTicket message
that was omitted.
Updates #72006
Change-Id: I9faea4d18589d10b163211aa17b2d0da8af1187e
Reviewed-on: https://go-review.googlesource.com/c/go/+/650736
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
When clients use debug/dwarf to examine DWARF 5 binaries, we can run
into problems when the Seek() method is used to skip ahead from a DIE
in one compilation unit to a DIE in another unit. The problem here is
that it is common for DWARF 5 comp units to have attributes (ex:
DW_AT_addr_base) whose value must be applied as an offset when reading
certain forms (ex: DW_FORM_addrx) within that unit. The existing
implementation didn't have a good way to recover these attrs following
the Seek call, and had to essentially punt in this case, resulting in
incorrect attr values.
This patch adds new support for reading and caching the key comp unit
DIE attributes (DW_AT_addr_base, DW_AT_loclists_base, etc) prior to
visiting any of the DIE entries in a unit, storing the cache values of
these attrs the main table of units. This base attribute
reading/caching behavior also happens (where needed) after Seek calls.
Should resolve delve issue 3861.
Supercedes Go pull request 70400.
Updates #26379.
Fixes#57046.
Change-Id: I536a57e2ba4fc55132d91c7f36f67a91ac408dc3
Reviewed-on: https://go-review.googlesource.com/c/go/+/655976
Reviewed-by: Alessandro Arzilli <alessandro.arzilli@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This adds a new godebug to control whether the runtime applies the
anonymous memory mapping annotations added in https://go.dev/cl/646095.
It is enabled by default.
This has several effects:
* The feature is only enabled by default when the main go.mod has go >=
1.25.
* This feature can be disabled with GODEBUG=decoratemappings=0, or the
equivalents in go.mod or package main. See https://go.dev/doc/godebug.
* As an opaque setting, this option will not appear in runtime/metrics.
* This setting is non-atomic, so it cannot be changed after startup.
I am not 100% sure about my decision for the last two points.
I've made this an opaque setting because it affects every memory mapping
the runtime performs. Thus every mapping would report "non-default
behavior", which doesn't seem useful.
This setting could trivially be atomic and allow changes at run time,
but those changes would only affect future mappings. That seems
confusing and not helpful. On the other hand, going back to annotate or
unannotate every previous mapping when the setting changes is
unwarranted complexity.
For #71546.
Change-Id: I6a6a636c5ad551d76691cba2a6f668d5cff0e352
Reviewed-on: https://go-review.googlesource.com/c/go/+/655895
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
CL 637939 changed ReverseProxy's handling of hijacked connections:
After copying all data in one direction, it half-closes the outbound
connection rather than fully closing both.
Revert to the old behavior when the outbound connection does not support
CloseWrite, avoiding a case where one side of the proxied connection closes
but the other remains open.
Fixes#72140
Change-Id: Ic0cacaa6323290f89ba48fd6cae737e86045a435
Reviewed-on: https://go-review.googlesource.com/c/go/+/655595
Reviewed-by: Jonathan Amsterdam <jba@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
Fix incorrect godoc links related to the use of the name "Reader" for
different things in the various compress/* packages:
- in compress/flate Reader is the interface describing the underlying reader,
not the decompressor as in other packages, so "returned reader" must
not be linked to Reader.
- in compress/lzw and compress/gzip Reader is the decompressor, not the
interface of the underlying reader, so "underlying reader" must not
be linked to Reader.
With this patch the formatting of "underlying reader" and "returned
reader" is consistent accross compress/* packages.
Change-Id: Iea315fd5ee5b6c177855693d68841f3709a382cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/655335
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Russ Cox noticed that reset was clearing limbs up to the *previous* Nat
size, not up to the new size, because clear(x.limbs) was happening
before the x.limbs[:n] reslice.
That's potentially a severe issue, because it may leave garbage in
x.limbs[len(x.limbs):n] if n < cap(x.limbs).
We were saved by an accidental invariant caused by the bug itself,
though: x.limbs[len(x.limbs):cap(x.limbs)] are always zero.
reset was always clearing all exposed (and hence potentially non-zero)
limbs before shrinking the Nat, and the only other function that could
shrink the Nat was trim, which only trims zero limbs.
Near miss.
Preserve the accidental invariant in the fix, because memclr is cheap
and it just proved it can save us from potential mistakes.
Change-Id: I6a6a4656a77735d8e8d520c699c4d85dd33ce497
Reviewed-on: https://go-review.googlesource.com/c/go/+/655056
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
If the error cause is not further specified (empty string),
avoid allocating a new errorCause. This makes using errorCauses
as boolean signals efficient.
While at it, fix an error message for incomparable arrays:
report the array type rather than its underlying type.
Change-Id: I844b18a76695330ca726932ee760aa89635f6a38
Reviewed-on: https://go-review.googlesource.com/c/go/+/654575
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Combine commonUnder and commonUnderOrChan:
- Provide an optional cond(ition) function argument to commonUnder
to establish additional type set conditions.
- Instead of a *Checker and *string argument for error reporting,
return an error cause that is only allocated in the presence of
an error.
- Streamline some error messages.
Replace all calls to coreType with calls to commonUnder.
Change-Id: I81ac86d0d532cddc09164309acced61d90718b44
Reviewed-on: https://go-review.googlesource.com/c/go/+/654455
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
This CL stores coverage profile data in the GOCACHE under the
'coverprofile' subkey alongside tests. This makes tests which use
coverage profiles cacheable. The values of the -coverprofile and
-outputdir flags are not included in the cache key to allow cached
profile data to be written to any output file.
Note: This is a rebase and squash from the original PRs below that
was created/closed/abandoned by @jproberts and @macnibblet that I
plan to maintain.
- https://github.com/golang/go/pull/50483
- https://github.com/golang/go/pull/65657
I made improvements to the change based on feedback from @bcmills in Gerrit
https://go-review.googlesource.com/c/go/+/563138.
From @macnibblet:
I don't know if anyone has considered the environmental impact
(Yes, of course, dev experience too), but on a team with 3 backend
developers, when I replaced our CI Golang version with this build,
it reduced the build time by 50%, which would have
equated to about 5000 hours of CI reduced in the past year.
Fixes#23565
Change-Id: I59a20af5ea156f990a17544cf06dc667ae7f8aa3
GitHub-Last-Rev: a5a1d1b9c8
GitHub-Pull-Request: golang/go#69339
Reviewed-on: https://go-review.googlesource.com/c/go/+/610564
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@golang.org>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
The 'used' field on Var and PkgName is fundamentally an aspect of the
type checking pass: it records when objects are used, for the purposes
of reporting errors for unused variables or package names. While
expedient and performant, recording this information in the types.Object
instances themselves increases the memory footprint of type-checked
packages, and (as we saw in golang/go#71817) can lead to data races when
Objects are reused in follow-up type checking, such as is done with the
CheckExpr and Eval APIs.
Fix this by externalizing the 'used' information into two maps (one for
variables and one for packages) on the types.Checker, so that they are
garbage-collected after type checking, and cannot be a source of data
races.
Benchmarks showed essentially no change in performance.
Fixesgolang/go#71817
Change-Id: I40daeabe4ecaca3bcb494e2f1c62a04232098e49
Reviewed-on: https://go-review.googlesource.com/c/go/+/650796
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
cmd/go tests that run builds are generally skipped in short mode. This
change will adds skips for some tests that were running builds.
I found these by sorting tests by elapsed time and removing the top
tests that invoked go build. It's our practice to skip tests that run go
build without the -n flag (which prints but doesn't execute commands).
On my work laptop this reduces test run time from about 20 seconds to
about 16 seconds. On my linux workstation it reduces test run time from
about 10 seconds to about 5 seconds.
Change-Id: I18ffcc231df013cb6ac5f5eb3544bed28dadeda8
Reviewed-on: https://go-review.googlesource.com/c/go/+/653775
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL adds an enum type, VarKind, that discriminates among
the various kinds of Var, and adds setter/getter methods
for Var's kind field.
Beware: NewVar has a weaker postcondition: the Var objects it
returns are not completely initialized and require a call to
Var.SetKind. This should only affect importers.
No changes are needed to the export data, since the kind can
always be deduced from the context when decoding.
See CL 645656 for the corresponding x/tools changes.
+ test, relnote, API
Updates golang/go#70250
Change-Id: Icde86ad22a880cde6f50bc12bf38004a5c6a1025
Reviewed-on: https://go-review.googlesource.com/c/go/+/645115
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Findley <rfindley@google.com>
These methods were previously used by crypto/ecdsa, but now not even
ecdsa_legacy.go uses them. Neither were ever documented.
Inverse was available only on P256() and only on amd64 and arm64, so
hopefully no one used it. CombinedMult was always available on all
curves, so it's possible some application might have used it, but all
the samples on GitHub I can find copied the old crypto/ecdsa package,
which does a conditional interface upgrade with a fallback, so they
won't break.
Change-Id: I6a6a4656ee1ab98438ca0fb20bea53b229cd7e71
Reviewed-on: https://go-review.googlesource.com/c/go/+/640116
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
SP and HTAB are allowed after a = before the following CRLF.
RFC 2045 section 6.7 describes the ABNF for the quoted-printable encoding:
qp-line := *(qp-segment transport-padding CRLF)
qp-part transport-padding
qp-segment := qp-section *(SPACE / TAB) "="
transport-padding := *LWSP-char
; Composers MUST NOT generate
; non-zero length transport
; padding, but receivers MUST
; be able to handle padding
; added by message transports.
RFC 822 defines LWSP-char as:
LWSP-char = SPACE / HTAB
Dovecot's imaptest contains such a message in
src/tests/fetch-binary-mime-qp.mbox.
Fixes#70952
Change-Id: Ie05921088d7e4d6c92c4bf79b0f4a13586230753
GitHub-Last-Rev: e6e6eee8eb
GitHub-Pull-Request: golang/go#70951
Reviewed-on: https://go-review.googlesource.com/c/go/+/638276
Reviewed-by: Sean Liao <sean@liao.dev>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Sean Liao <sean@liao.dev>
This CL propagates closing the write stream from either side of the
reverse proxy and ensures the proxy waits for both copy-to and the
copy-from the backend to complete.
The new unit test checks communication through the reverse proxy when
the backend or frontend closes either the read or write streams.
That closing the write stream is propagated through the proxy from
either the backend or the frontend. That closing the read stream is
not propagated through the proxy.
Fixes#35892
Change-Id: I83ce377df66a0f17b9ba2b53caf9e4991a95f6a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/637939
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Sean Liao <sean@liao.dev>
Auto-Submit: Sean Liao <sean@liao.dev>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Matej Kramny <matejkramny@gmail.com>
The root cause of issue #65802 is a small race condition that occurs between
two events:
1. During the HTTP server shutdown, a connection in an idle state is identified
and closed.
2. The connection, although idle, has just finished reading a complete request
before being closed and hasn't yet updated its state to active.
In this scenario, despite the connection being closed, the request continues to
be processed. This not only wastes server resources but also prevents the
client request from being retried.
Fixes#65802
Change-Id: Ic22abb4497be04f6c84dff059df00f2c319d8652
GitHub-Last-Rev: 426099a3e7
GitHub-Pull-Request: golang/go#65805
Reviewed-on: https://go-review.googlesource.com/c/go/+/565277
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Sean Liao <sean@liao.dev>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Sean Liao <sean@liao.dev>
This patch updates the compiler to generate DWARF5-style location
lists (e.g. entries that feed into .debug_loclists) as opposed to
DWARF4-style location lists (which wind up in .debug_loc). The DWARF5
format is much more compact, and can make indirect references to text
addresses via the .debug_addr section for further space savings.
Updates #26379.
Change-Id: If2e6fce1136d9cba5125ea51a71419596d1d1691
Reviewed-on: https://go-review.googlesource.com/c/go/+/635836
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
This patch updates the compiler to generate DWARF5-style range lists
(e.g. entries that feed into .debug_rnglists) as opposed to
DWARF4-style range lists (which wind up in .debug_ranges). The DWARF5
format is much more compact, and can make indirect references to text
address via the .debug_addr section for further space savings.
Updates #26379.
Change-Id: I273a6283484b7fe33d79d5412e31c5155b22a7c0
Reviewed-on: https://go-review.googlesource.com/c/go/+/635345
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
CL 629195 strongly favor closure inlining, allowing closures to be
inlined more aggressively.
However, if the closure body contains a call to a function, which itself
is one of the call arguments, it causes the infinite inlining.
Fixing this by prevent this kind of functions from being inlinable.
Fixes#72063
Change-Id: I5fb5723a819b1e2c5aadb57c1023ec84ca9fa53c
Reviewed-on: https://go-review.googlesource.com/c/go/+/654195
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The implementatiom still calls coreType in places and refers to
"core types" in comments, but user-visible error messages don't
know about core types anymore.
This brings the user-visible part of the implementation in sync with
the spec which doesn't have the notion of core types anymore.
For #70128.
Change-Id: I14bc6767a83e8f54b10ebe99a7df0b98cd9fca87
Reviewed-on: https://go-review.googlesource.com/c/go/+/654395
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Griesemer <gri@google.com>
For now, use commonUnder (formerly called sharedUnder) and update
error messages and comments. We can provide better error messages
in individual cases eventually.
Kepp using coreType for make built-in for now because it must accept
different channel types with non-conflicting directions and identical
element types. Added extra test cases.
While at it, rename sharedUnder, sharedUnderOrChan to commonUnder
and commonUnderOrChan, respectively (per suggestion from rfindley).
For #70128.
Change-Id: I11f3d5ce858746574f4302271d8cb763c2cdcf98
Reviewed-on: https://go-review.googlesource.com/c/go/+/653139
Reviewed-by: Robert Findley <rfindley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Div is way faster. We could actually test a lot more primes and still
gain performance despite the diminishing returns, but necessarily it
would have marginal impact overall.
fips140: off
goos: linux
goarch: amd64
pkg: crypto/rsa
cpu: AMD Ryzen 7 PRO 8700GE w/ Radeon 780M Graphics
│ e325b41ad1 │ 0f611af2e1 │
│ sec/op │ sec/op vs base │
GenerateKey/2048-16 124.19m ± 0% 39.93m ± 0% -67.85% (p=0.000 n=20)
Surprisingly, the performance gain is similar on ARM64, which doesn't
have intrinsified math.Div.
fips140: off
goos: darwin
goarch: arm64
pkg: crypto/rsa
cpu: Apple M2
│ e325b41ad1 │ 6276161a7f │
│ sec/op │ sec/op vs base │
GenerateKey/2048-8 136.49m ± 0% 47.97m ± 1% -64.86% (p=0.000 n=20)
Change-Id: I6a6a46560331198312bd09c1cbe4d2b3c370c552
Reviewed-on: https://go-review.googlesource.com/c/go/+/639955
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
CL 652276 reduced the m struct by 8 bytes, which has changed the
allocation class on 64 bit OpenBSD platforms. This results in build
failures due to:
M structure uses sizeclass 1792/0x700 bytes; incompatible with mutex flag mask 0x3ff
Add 128 bytes of padding when spinbitmutex is enabled on 64 bit
architectures, moving the size to the half point between the
1792 and 2048 allocation size.
Change-Id: I71623a1f75714543c302217e619d20cf0e717aeb
Reviewed-on: https://go-review.googlesource.com/c/go/+/653335
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
The tool IDs can be calculated once and reused across multiple
threads. This is a small optimization that helps optimize system
resources.
On a normal Windows machine with 12 virtual CPUs, the time to build
a hello world program is reduced from over 1 second, with spikes of 2
seconds, to a consistent 0.7 seconds.
Updates #71981.
Change-Id: I85f4a19f8ad4230afa32213780c761b7eb22fa29
Reviewed-on: https://go-review.googlesource.com/c/go/+/653715
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
GetPackage is sometimes called with a modroot or pkgdir that ends with a
path separator, and sometimes without. Clean them before passing them to
openIndexPackage to deduplicate calls to mcache.Do and action entry
files.
This shouldn't affect #71698 but was discovered while debugging that
issue.
Change-Id: I6a7fa4de93f45801504abea11bd97f6c6577f296
Reviewed-on: https://go-review.googlesource.com/c/go/+/652435
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The __nl_symbol_ptr is not a common section name anymore. LLVM prefers
__got for GOT symbols in the __DATA_CONST segment.
Note that the Go linker used to place the GOT section in the __DATA
segment, but since CL 644055 we place it in the __DATA_CONST segment.
Updates #71416.
Cq-Include-Trybots: luci.golang.try:gotip-darwin-amd64-longtest
Change-Id: Icb776e19855eaabb4777a9b1eb433497842413b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/652555
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
This change tries increasing the timeout in
TestSpuriousWakeupsNeverHangSemasleep. I'm not entirely sure of the
mechanism, but GODEBUG=gcstoptheworld=2 and GODEBUG=gccheckmark=1 can
cause this test to fail at it's regular timeout. It does not seem to
indicate a deadlock, because bumping the timeout 10x make the problem
go away. I suspect the problem is due to the long STW times these two
modes can induce, plus the fact this test runs in parallel with others.
Let's just bump the timeout. The test is fundamentally sound, and it's
unclear to me how else to test for a deadlock here.
Fixes#71691.
Fixes#71548.
Change-Id: I649531eeec8a8408ba90823ce5223f3a17863124
Reviewed-on: https://go-review.googlesource.com/c/go/+/652756
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Unlike request headers, where we are allowed to leniently accept
a bare LF in place of a CRLF, chunked bodies must always use CRLF
line terminators. We were already enforcing this for chunk-data lines;
do so for chunk-size lines as well. Also reject bare CRs anywhere
other than as part of the CRLF terminator.
Fixes CVE-2025-22871
Fixes#71988
Change-Id: Ib0e21af5a8ba28c2a1ca52b72af8e2265ec79e4a
Reviewed-on: https://go-review.googlesource.com/c/go/+/652998
Reviewed-by: Jonathan Amsterdam <jba@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Add a few more test cases for scanning (integer conversion),
which were helpful in debugging some upcoming changes.
BenchmarkScan currently times converting the value 10**N
represented in base B back into []Word form.
When B = 10, the text is 1 followed by many zeros, which
could hit a "multiply by zero" special case when processing
many digit chunks, misrepresenting the actual time required
depending on whether that case is optimized.
Change the benchmark to use 9**N, which is about as big and
will not cause runs of zeros in any of the tested bases.
The benchmark comparison below is not showing faster code,
since of course the code is not changing at all here. Instead,
it is showing that the new benchmark work is roughly the same
size as the old benchmark work.
goos: darwin
goarch: arm64
pkg: math/big
cpu: Apple M3 Pro
│ old │ new │
│ sec/op │ sec/op vs base │
ScanPi-12 43.35µ ± 1% 43.59µ ± 1% ~ (p=0.069 n=15)
Scan/10/Base2-12 202.3n ± 2% 193.7n ± 1% -4.25% (p=0.000 n=15)
Scan/100/Base2-12 1.512µ ± 3% 1.447µ ± 1% -4.30% (p=0.000 n=15)
Scan/1000/Base2-12 15.06µ ± 2% 14.33µ ± 0% -4.83% (p=0.000 n=15)
Scan/10000/Base2-12 188.0µ ± 5% 177.3µ ± 1% -5.65% (p=0.000 n=15)
Scan/100000/Base2-12 5.814m ± 3% 5.382m ± 1% -7.43% (p=0.000 n=15)
Scan/10/Base8-12 78.57n ± 2% 75.02n ± 1% -4.52% (p=0.000 n=15)
Scan/100/Base8-12 548.2n ± 2% 526.8n ± 1% -3.90% (p=0.000 n=15)
Scan/1000/Base8-12 5.674µ ± 2% 5.421µ ± 0% -4.46% (p=0.000 n=15)
Scan/10000/Base8-12 94.42µ ± 1% 88.61µ ± 1% -6.15% (p=0.000 n=15)
Scan/100000/Base8-12 4.906m ± 2% 4.498m ± 3% -8.31% (p=0.000 n=15)
Scan/10/Base10-12 73.42n ± 1% 69.56n ± 0% -5.26% (p=0.000 n=15)
Scan/100/Base10-12 511.9n ± 1% 488.2n ± 0% -4.63% (p=0.000 n=15)
Scan/1000/Base10-12 5.254µ ± 2% 5.009µ ± 0% -4.66% (p=0.000 n=15)
Scan/10000/Base10-12 90.22µ ± 2% 84.52µ ± 0% -6.32% (p=0.000 n=15)
Scan/100000/Base10-12 4.842m ± 3% 4.471m ± 3% -7.65% (p=0.000 n=15)
Scan/10/Base16-12 62.28n ± 1% 58.70n ± 1% -5.75% (p=0.000 n=15)
Scan/100/Base16-12 398.6n ± 0% 377.9n ± 1% -5.19% (p=0.000 n=15)
Scan/1000/Base16-12 4.108µ ± 1% 3.782µ ± 0% -7.94% (p=0.000 n=15)
Scan/10000/Base16-12 83.78µ ± 2% 80.51µ ± 1% -3.90% (p=0.000 n=15)
Scan/100000/Base16-12 5.080m ± 3% 4.698m ± 3% -7.53% (p=0.000 n=15)
geomean 12.41µ 11.74µ -5.36%
Change-Id: If3ce290ecc7f38672f11b42fd811afb53dee665d
Reviewed-on: https://go-review.googlesource.com/c/go/+/650639
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>
Implement vector unit stride, vector strided, vector indexed and
vector whole register load and store instructions.
The vector unit stride instructions take an optional vector mask
register, which if specified must be register V0. If only two
operands are given, the instruction is encoded as unmasked.
The vector strided and vector indexed instructions also take an
optional vector mask register, which if specified must be register
V0. If only three operands are given, the instruction is encoded as
unmasked.
Cq-Include-Trybots: luci.golang.try:gotip-linux-riscv64
Change-Id: I35e43bb8f1cf6ae8826fbeec384b95ac945da50f
Reviewed-on: https://go-review.googlesource.com/c/go/+/631937
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Mark Ryan <markdryan@rivosinc.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Pengcheng Wang <wangpengcheng.pp@bytedance.com>
CL 527935 optimized []byte(string1 + string2) to use runtime.concatbytes
to prevent concatenating of strings before converting to slices.
However, the optimization is implemented without allowing temporary
buffer for slice on stack, causing un-necessary allocations.
To fix this, optimize concatbytes to use temporary buffer if the result
string length fit to the buffer size.
Fixes#71943
Change-Id: I1d3c374cd46aad8f83a271b8a5ca79094f9fd8db
Reviewed-on: https://go-review.googlesource.com/c/go/+/652395
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
_beginthread is intended to be used together with the C runtime.
The cgo runtime doesn't use it, so better use CreateThread directly,
which is the Windows API for creating threads.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-arm64
Change-Id: Ic6cf75f69f62a3babf5e74155da1aac70961886c
Reviewed-on: https://go-review.googlesource.com/c/go/+/651995
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The lifetime of the variables are identical; capture
them in a single struct to avoid individual allocations.
The inner closure can also avoid allocation by using the
capture of the outer closure.
Escape analysis for OnceValues:
/go/src/sync/oncefunc.go:74:29: moved to heap: sync.f
/go/src/sync/oncefunc.go:76:3: moved to heap: sync.once
/go/src/sync/oncefunc.go:77:3: moved to heap: sync.valid
/go/src/sync/oncefunc.go:78:3: moved to heap: sync.p
/go/src/sync/oncefunc.go:79:3: moved to heap: sync.r1
/go/src/sync/oncefunc.go:80:3: moved to heap: sync.r2
/go/src/sync/oncefunc.go:82:7: func literal escapes to heap
/go/src/sync/oncefunc.go:83:9: func literal does not escape
/go/src/sync/oncefunc.go:93:9: func literal escapes to heap
After provided changes:
/go/src/sync/oncefunc.go:86:2: moved to heap: sync.d
/go/src/sync/oncefunc.go:96:9: func literal escapes to heap
/go/src/sync/oncefunc.go:99:13: func literal does not escape
/go/src/sync/oncefunc.go:100:10: func literal does not escape
Change-Id: Ib06e650fd427b57e0bdbdf1fe759fe436104ff79
Reviewed-on: https://go-review.googlesource.com/c/go/+/601596
Auto-Submit: Austin Clements <austin@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Austin Clements <austin@google.com>
On Darwin, the .got section can be placed in a read-only segment. Only the dynamic linker should modify it at start-up time.
Other read-only sections, like .typelink and .itablink, are already placed in the __DATA_CONST segment. Do the same for the .got section.
Fixes#71416.
Cq-Include-Trybots: luci.golang.try:gotip-darwin-amd64-longtest
Change-Id: I9cd9c20da63b655fabb61d742feb086c3ef3bea7
Reviewed-on: https://go-review.googlesource.com/c/go/+/644055
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL removes the notion of core types from the spec.
Instead of referring to core types, each section that did
so before is reverted to approx. the pre-generics (1.17)
prose, and additional paragraphs cover the type parameter
cases as needed.
The hope is that this makes it easier to read the spec.
When type parameters are involved, the extra prose is
local to the language feature in question and thus more
readily available. When no type parameters are present,
readers do not have to concern themselves with core types.
In contrast to CL 621919, this change is not intended to
loosen the spec in any way and therefore does not change
the language (if the new prose implies otherwise, we will
correct it).
Except for adjustments to compiler error messages
(no mention of core types anymore), no other changes
to the compiler or tools are required.
Future CLs may selectively relax requirements on a language
construct by language construct basis; each such change can
be discussed and proposed independently.
For #70128.
Change-Id: I6ed879a472c615d7c8dbdc7b6bd7eef3d12eff7e
Reviewed-on: https://go-review.googlesource.com/c/go/+/645716
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Bypass: Robert Griesemer <gri@google.com>
this removes the old conditional-on-register-value
handshake from the deferproc/deferprocstack logic.
The "line" for the recovery-exit frame itself (not the defers
that it runs) is the closing brace of the function.
Reduces code size slightly (e.g. go command is 0.2% smaller)
Sample output showing effect of this change, also what sort of
code it requires to observe the effect:
```
package main
import "os"
func main() {
g(len(os.Args) - 1) // stack[0]
}
var gi int
var pi *int = &gi
//go:noinline
func g(i int) {
switch i {
case 0:
defer func() {
println("g0", i)
q() // stack[2] if i == 0
}()
for j := *pi; j < 1; j++ {
defer func() {
println("recover0", recover().(string))
}()
}
default:
for j := *pi; j < 1; j++ {
defer func() {
println("g1", i)
q() // stack[2] if i == 1
}()
}
defer func() {
println("recover1", recover().(string))
}()
}
p()
} // stack[1] (deferreturn)
//go:noinline
func p() {
panic("p()")
}
//go:noinline
func q() {
panic("q()") // stack[3]
}
/* Sample output for "./foo foo":
recover1 p()
g1 1
panic: q()
goroutine 1 [running]:
main.q()
.../main.go:46 +0x2c
main.g.func3()
.../main.go:29 +0x48
main.g(0x1?)
.../main.go:37 +0x68
main.main()
.../main.go:6 +0x28
*/
```
Change-Id: Ie39ea62ecc244213500380ea06d44024cadc2317
Reviewed-on: https://go-review.googlesource.com/c/go/+/650795
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The extension-removing rules for ARM64 were moved to late lower in
CL 568616. This means that the late lower pass can now generate
MOVDreg, however the rules that potentially eliminate MOVDreg only
exist in the earlier pass. Fix this by duplicating the MOVDreg/NOVDnop
rules in late lower, such that we can potentially eliminate conversions.
Removes 400+ instructions from the Go binary on openbsd/arm64.
Change-Id: I14aad06b994c9179f3ecdda566629793ba167511
Reviewed-on: https://go-review.googlesource.com/c/go/+/651819
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
This patch rolls the main .debug_info DWARF section from version 4 to
version 5, and also introduces machinery in the Go compiler and linker
for taking advantage of the DWARF5 ".debug_addr" section for
subprogram DIE "high" and "low" PC attributes. All functionality is
gated by GOEXPERIMENT=dwarf5.
For the compiler portion of this patch, we add a new DIE attribute
form "DW_FORM_addrx", which accepts as an argument a function (text)
symbol. The dwarf "putattr" function is enhanced to handle this
format by invoking a new dwarf context method "AddIndirectTextRef".
Under the hood, this method invokes the Lsym method WriteDwTxtAddrx,
which emits a new objabi.R_DWTXTADDR_* relocation. The size of the
relocation is dependent on the number of functions in the package; we
pick a size that is just big enough for the largest func index.
In the linker portion of this patch, we now switch over to writing out
a version number of 5 (instead of 4) in the compile unit header (this
is required if we want to use addrx attributes). In the parallel portion
of DWARF gen, within each compilation unit we scan subprogram DIEs to
look for R_DWTXTADDR_* relocations, and when we find such a reloc,
we assign a slot in the .debug_addr section for the func targeted.
After the parallel portion is complete, we then walk through all of the
compilation units to assign a value to their DW_AT_addr_base attribute,
which points to the portion of the single .debug_addr section containing
the text addrs for that compilation unit.
Note that once this patch is in, programs built with GOEXPERIMENT=dwarf5
will have broken/damaged DWARF info; in particular, since we've changed
only the CU and subprogram DIEs and haven't incorported the other
changes mandated by DWARF5 (ex: .debug_ranges => .debug_rnglists)
a lot of the variable location info will be missing/incorrect. This
will obviously change in subsequent patches.
Note also that R_DWTXTADDR_* can't be used effectively for lexical
scope DIE hi/lo PC attrs, since there isn't a viable way to encode
"addrx + constant" in the attribute value (you would need a new entry
for each attr endpoint in .debug_addr, which would defeat the point).
Updates #26379.
Change-Id: I2dfc45c9a8333e7b2a58f8e3b88fc8701fefd006
Reviewed-on: https://go-review.googlesource.com/c/go/+/635337
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Add a new symbol type: SDWARFADDR. This kind of symbol stores content
to be added to the DWARF .debug_addr section (new with DWARF5). At the
moment these symbols are created only in the linker, but it's not hard to
imagine other implementations in which the compiler would create them,
so they are added to both the compiler and linker symbol kind space.
Updates #26379.
Change-Id: I4a82ead0d59fe6028abfd6d6e3fc3df2e28c0ef6
Reviewed-on: https://go-review.googlesource.com/c/go/+/634415
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Add a set of new relocations to be used when the compiler is writing
debug information using DWARF version 5. No changes in compiler or
linker functionality, this patch just adds the relocations themselves
and some helper functions; uses will appear in a later patch. These
relocations are generated by the compiler when writing a DWARF DIE
attribute of form DW_FORM_addrx, or when writing a .debug_addr index
reference in a SDWARFRANGE or SDWARFLOC section. The target symbol of
the relocation is a function (STEXT symbol); the linker resolves the
relocation by replacing the target of the reloc with an index of a
slot in the .debug_addr section (.debug_addr is new with DWARF5).
Updates #26379.
Change-Id: I43c587d25d0836972dac487d09c8924d77345f4e
Reviewed-on: https://go-review.googlesource.com/c/go/+/633880
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
This patch rolls out the necessary changes to migrate the DWARF line
table support in the compiler and linker to DWARF version 5, gated by
the "dwarf5" GOEXPERIMENT.
DWARF version 5 includes a number of changes to the line table,
notably a revamped prolog section and a change in the indexing system
used to refer to files and directories within the line table
program. Specifically, prior to DWARF 4 a compilation's directory
table was considered to have an implicit zero entry containing the
compilation directory of the translation unit (package), and the file
table was considered to have an implicit zero entry storing the
"primary source file" (stored in the compilation unit DIE name).
DWARF 5 does away with these implicity entries meaning that files and
dirs are now effectively a 0-based index.
Updates #26379.
Change-Id: I9b4f1be5415aacec1ba57366d60bd48819c56ea5
Reviewed-on: https://go-review.googlesource.com/c/go/+/633879
Reviewed-by: Alessandro Arzilli <alessandro.arzilli@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Similarly to CL 648035, SetBytes doesn't need to be constant time for
the uses we make of it in the standard library (ECDH and ECDSA public
keys), but it doesn't cost much to make it constant time for users of
the re-exported package, or even just to save the next person from
convincing themselves that it's ok for it not to be constant time.
Change-Id: I6a6a465622a0de08d9fc71db75c63185a82aa54a
Reviewed-on: https://go-review.googlesource.com/c/go/+/650579
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Currently method values aren't correctly handled in Seq because we call
canRangeFunc on the reciever type, not the method value type, when we're
handling a method value. reflect.Value.Type has the logic to obtain the
method value type from the Value.
This change slightly refactors reflect.Value.Type into a separate
function so we can obtain the correct type as an abi.Type and pass it
off to canRangeFunc (and canRangeFunc2).
Fixes#71874.
Change-Id: Ie62dfca2a84b8f2f816bb87ff1ed1a58a7bb8122
Reviewed-on: https://go-review.googlesource.com/c/go/+/651416
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
As of CL 650835, the pidfd test child no longer sends SIGCHLD on exit.
Per clone(2), "If [the child termination] signal is specified as
anything other than SIGCHLD, then the parent process must specify the
__WALL or __WCLONE options when waiting for the child with wait(2)."
Align with this requirement.
For #71828.
Change-Id: I6a6a636c739e4a59abe1533fe429a433e8588939
Reviewed-on: https://go-review.googlesource.com/c/go/+/651415
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Overzealous security scanners don't like the Go 1.17 binary because they
think it has every 1.17 security vulnerability. base64-encode the binary
to hide from them.
I've also extended the instructions to make the binary easier to
reproduce.
Since we do the Go binary, we might as well do the C binary too, as it
apparently makes some virus scanners unhappy.
Fixes#71753.
For #71734.
For #71821.
Change-Id: I6a6a636cccbf5312522f52f27f74eded64048fb7
Reviewed-on: https://go-review.googlesource.com/c/go/+/651175
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
We used to generate register GC maps as an experimental approach
for asynchronous preemption, which later we chose not to take.
Most of the register GC map code are already removed. One
exception is that the ssa.Register type still contains a field
for the register map index. Remove it.
Change-Id: Ib177ebce9548aa5ffbcaedd4b507240ea7df8afe
Reviewed-on: https://go-review.googlesource.com/c/go/+/651076
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently Make panics when passed a linker-allocated object. This is
inconsistent with both runtime.AddCleanup and runtime.SetFinalizer. Not
panicking in this case is important so that all pointers can be treated
equally by these APIs. Libraries should not have to worry where a
pointer came from to still make weak pointers.
Supporting this behavior is a bit complex for weak pointers versus
finalizers and cleanups. For the latter two, it means a function is
never called, so we can just drop everything on the floor. For weak
pointers, we still need to produce pointers that compare as per the API.
To do this, copy the tiny lock-free trace map implementation and use it
to store weak handles for "immortal" objects. These paths in the
runtime should be rare, so it's OK if it's not incredibly fast, but we
should keep the memory footprint relatively low (at least not have it be
any worse than specials), so this change tweaks the map implementation a
little bit to ensure that's the case.
Fixes#71726.
Change-Id: I0c87c9d90656d81659ac8d70f511773d0093ce27
Reviewed-on: https://go-review.googlesource.com/c/go/+/649460
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
While looking at the SSA of following code, i noticed
that these rules do not work properly, and the types
are loaded indirectly through an itab, instead of statically.
type M interface{ M() }
type A interface{ A() }
type Impl struct{}
func (*Impl) M() {}
func (*Impl) A() {}
func main() {
var a M = &Impl{}
a.(A).A()
}
Change-Id: Ia275993f81a2e7302102d4ff87ac28586023d13c
GitHub-Last-Rev: 4bfc901917
GitHub-Pull-Request: golang/go#71784
Reviewed-on: https://go-review.googlesource.com/c/go/+/649500
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
The go command assumes that GOMODCACHE is immutable. As an example of
one place the assumption is made, the modindex won't stat the files in
GOMODCACHE when getting the cache key for the index entry and just uses
the path of the module in the modcache (basically the module's name and
version). Explicitly reject overlays affecting GOMODCACHE to avoid
surprising and incorrect behavior.
For #71783
For #71075
Change-Id: I21dd5d39d71037de473b09ac8482a1867864e11f
Reviewed-on: https://go-review.googlesource.com/c/go/+/650475
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
RtlIsDosDeviceName_U is specifically designed to detect Windows devices.
We were using GetFullPathName to do this, but it's not the right API
for the job, as it is slower and allocates more memory.
goos: windows
goarch: amd64
pkg: path/filepath
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
│ old.txt │ new.txt │
│ sec/op │ sec/op vs base │
IsLocal-12 5.685µ ± 59% 1.853µ ± 12% -67.41% (p=0.000 n=10)
│ old.txt │ new.txt │
│ B/op │ B/op vs base │
IsLocal-12 496.00 ± 0% 48.00 ± 0% -90.32% (p=0.000 n=10)
│ old.txt │ new.txt │
│ allocs/op │ allocs/op vs base │
IsLocal-12 10.000 ± 0% 6.000 ± 0% -40.00% (p=0.000 n=10)
Change-Id: Ib40ad7a90ab93cf7051c8d6becbce4d287f10f4e
Reviewed-on: https://go-review.googlesource.com/c/go/+/650578
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
The existing code for recover from deferrangefunc was broken in
several ways.
1. the code following a deferrangefunc call did not check the return
value for an out-of-band value indicating "return now" (i.e., recover
was called)
2. the returned value was delivered using a bespoke ABI that happened
to match on register-ABI platforms, but not on older stack-based
ABI.
3. the returned value was the wrong width (1 word versus 2) and
type/value(integer 1, not a pointer to anything) for deferrangefunc's
any-typed return value (in practice, the OOB value check could catch
this, but still, it's sketchy).
This -- using the deferreturn lookup method already in place for
open-coded defers -- turned out to be a much-less-ugly way of
obtaining the desired transfer of control for recover().
TODO: we also could do this for regular defer, and delete some code.
Fixes#71675
Change-Id: If7d7ea789ad4320821aab3b443759a7d71647ff0
Reviewed-on: https://go-review.googlesource.com/c/go/+/650476
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
CL 630696 changes budget for once-called closures, making them more
inlinable. However, when recursive inlining involve both the closure and
its parent, the inliner goes into an infinite loop:
parent (a closure) -> closure -> parent -> ...
The problem here dues to the closure name mangling, causing the inlined
checking condition failed, since the closure name affects how the
linker symbol generated.
To fix this, just prevent the closure from inlining its parent into
itself, avoid the infinite inlining loop.
Fixes#71680
Change-Id: Ib27626d70f95e5f1c24a3eb1c8e6c3443b7d90c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/649656
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Currently, exported Go functions with no parameters generate C functions
with an empty parameter list. In C, a function with an empty parameter
list can accept any number of arguments, whereas a function with a single
void parameter explicitly declares that it takes no arguments.
To align the generated C functions with their Go prototypes, update the
code generation to explicitly include a void parameter for functions
with no parameters.
Fixes#68411
Change-Id: Iab9456aa0236200bf21d1181a2e18e82869df63f
GitHub-Last-Rev: 6ff21a98df
GitHub-Pull-Request: golang/go#70981
Reviewed-on: https://go-review.googlesource.com/c/go/+/638635
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Provides, on one line, an approximation of P scheduling throughput: how
many times execute() was called for a given P. Said another way: how
many RUNNABLE to RUNNING transitions have happened for this P.
This allows discerning whether a P actually did anything, and how it
compares to other periods of a processes operation.
This should be useful to analyze (kernel) scheduler hiccups.
Investigators will want to subtract the tick values from subsequent
schedtrace lines to get a rate of schedulings. I've opted to add a space
around the first and last element as well to make it more uniform to do
the proposed subtracting with tools like AWK.
Change-Id: I69d6dae1509ad285d43799f38bcaa3aa0fb2352e
Reviewed-on: https://go-review.googlesource.com/c/go/+/635636
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Nicolas Hillegeer <aktau@google.com>
In particular, we apply it only to functions where it is always
a code improvement to inline the call.
We also apply it to some constants.
In a few cases this may introduce a panic statement at the
caller, which is debatable, but making the potential for panic
evident is the purpose of the deprecation.
The gofix analyzer in gopls v0.18 will show a diagnostic for calls
to the annotated functions, and will offer to inline the call.
The new //go:fix annotation needs a special exemption in the
pragma check in the compiler.
Updates #32816
Change-Id: I43bf15648ac12251734109eb7102394f8a76d55e
Reviewed-on: https://go-review.googlesource.com/c/go/+/648995
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Commit-Queue: Alan Donovan <adonovan@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Use return with register for async preemption resumption on arm64.
This has the same behaviour as the current use of JMP, however
is permitted when Branch Target Identification is being enforced,
while a JMP with register is considered an indirect call and
requires a `BTI J` marker at the resumption address.
Updates #66054
Change-Id: I135ac577073467bedd9efd8df15b76c97dc08767
Reviewed-on: https://go-review.googlesource.com/c/go/+/646782
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Internally we only use SetCanonicalBytes as part of Ed25519
verification, where all inputs are public, so it doesn't need to be
constant time.
However, this code is replicated outside of the standard library. Even
there, an attack is not practical, so this should not be considered a
security vulnerability:
- For specific scalars, this only leaks at most four bits of
information, and always the same four bits (so it's not an adaptive
attack).
- For derived scalars, assuming they are valid and uniformly
distributed, the loop would return true on the first iteration with
probability (1 - 2⁻¹²⁷) due to the shape of the scalar field order.
Still, making it constant time is easy enough and saves the next person
from having to think about it.
This was previously reported by Yawning Angel, and then as part of a
security audit.
Change-Id: I6a6a46563c8abecb0b4a6f12033a71c4c4da6fa7
Reviewed-on: https://go-review.googlesource.com/c/go/+/648035
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
CL 648315 and CL 648195 fixed#71615 in the case where we fail to read
the next generation by emitting an extra sync event before returning an
error. But, it's possible we failed to even read the next spilled batch
when we read the first generation, and have been carrying the error from
trying to read a spilled batch since the last generation. In this case,
we don't emit a final sync event, meaning that there are still some
cases where #71615 happens.
This change emits the final sync event in this corner case. I believe
this is the final corner case. I could previously reproduce the issue
by running the test under stress2, but I can no longer reproduce any
failures after this change.
Fixes#71615, for real this time.
Change-Id: I10688a3c0e4b8327a95f31add365338c77c091ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/649259
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Benchmarking key generation is a pain. The number of random candidates
explored before finding a prime varies greatly, and on top of that some
rejections happen in the trial divisions step and some in the
Miller-Rabin step.
However, we can calculate on average how many candidates we should
reject before finding a prime, and of those how many should be divisible
by small primes. (And even the number of multiplications in a
Miller-Rabin iteration.) The new checked in sequence of candidates is
normalized to represent the average case.
It doesn't normalize the runtime of GCD, but running the benchmark with
20 different randomly generated "average cases" produces very consistent
results, so it must not matter much.
goos: darwin
goarch: arm64
pkg: crypto/rsa
cpu: Apple M2
│ regen.txt │
│ sec/op │
GenerateKey/2048-8 136.4m ± 0%
Changed slightly the excess masking in keygen.go to make it easier to
feed fixed candidates. This might also make it easier to share test
vectors in the future.
Change-Id: I66696c693f35da7bda27db537aa3bf3b991e970e
Reviewed-on: https://go-review.googlesource.com/c/go/+/639335
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Implement vector configuration setting instructions (VSETVLI,
VSETIVLI, VSETL). These allow the vector length (vl) and vector
type (vtype) CSRs to be configured via a single instruction.
Unfortunately each instruction has its own dedicated encoding.
In the case of VSETVLI/VSETIVLI, the vector type is specified via
a series of special operands, which specify the selected element
width (E8, E16, E32, E64), the vector register group multiplier
(M1, M2, M4, M8, MF2, MF4, MF8), the vector tail policy (TU, TA)
and vector mask policy (MU, MA). Note that the order of these
special operands matches non-Go assemblers.
Partially based on work by Pengcheng Wang <wangpengcheng.pp@bytedance.com>.
Cq-Include-Trybots: luci.golang.try:gotip-linux-riscv64
Change-Id: I431f59c1e048a3e84754f0643a963da473a741fe
Reviewed-on: https://go-review.googlesource.com/c/go/+/631936
Reviewed-by: Mark Ryan <markdryan@rivosinc.com>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
This CL adds new relocation type for riscv64: R_GOT_PCREL_ITYPE_RELOC
which generate an AUIPC + I-type pair with relocation type of GOT_HI20
and PCREL_LO12_I.
According to RISCV elf psabi doc, medium position independent code
model, the GNU as example is:
```
# Calculate address of non-local symbol
.Ltmp3: aupipc a0, %got_pcrel_hi(symbol)
ld a0, %pcrel_lo(.Ltmp3)(a0)
```
Change-Id: I719dd05e009ca2d9291f0689b346c059f9c56918
Reviewed-on: https://go-review.googlesource.com/c/go/+/612635
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Particularly with 2-word load instructions, this becomes important.
Classic example is:
func f(p *string) string {
return *p
}
We want the two loads to put the return values directly into
the two ABI return registers.
At this point in the stack, cmd/go is 1.1% smaller.
Change-Id: I51fd1710238e81d15aab2bfb816d73c8e7c207b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/631137
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
If we call slicebytetostring immediately (with no intervening writes)
before calling map access or delete functions with the resulting
string as the key, then we can just use the ptr/len of the
slicebytetostring argument as the key. This avoids an allocation.
Fixes#44898
Update #71132
There's old code in cmd/compile/internal/walk/order.go that handles
some of these cases.
1. m[string(b)]
2. s := string(b); m[s]
3. m[[2]string{string(b1),string(b2)}]
The old code handled cases 1&3. The new code handles cases 1&2.
We'll leave the old code around to keep 3 working, although it seems
not terribly common.
Case 2 happens particularly after inlining, so it is pretty common.
Change-Id: I8913226ca79d2c65f4e2bd69a38ac8c976a57e43
Reviewed-on: https://go-review.googlesource.com/c/go/+/640656
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Fix TestACVP environment construction to include both ACVP_WRAPPER and
GODEBUG.
Previously we were accidentally overwriting the cmd.Env, stomping the
ACVP_WRAPPER env var and replacing it with just the GODEBUG env var.
This in turn makes the tests start to fail when the test binary
subprocess is invoked without knowing it's fulfilling the role of the
wrapper, and not the test driver.
Change-Id: Ie6ee30c8b93b2051a671e12aaa63d2116c5eb8c8
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/649016
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
If there's an error getting the version of the c compiler, add the error
to the input used to produce the cache key. In the case where we can't
parse the version, the text of the output of the command is part of the
error, so different unparseable versions will produce different cache
keys. Before, we wouldn't add anything to the key when there was an
error getting the version, so we wouldn't distinguish a missing compiler
from one where we couldn't parse the version.
Fixes#64589
Change-Id: I27f853e8ff40002e2f045b2210633b38f93d0130
Reviewed-on: https://go-review.googlesource.com/c/go/+/648196
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
The tree has opened for Go 1.25 development. This is a time to update
all golang.org/x/... module versions that contribute packages to the
std and cmd modules in the standard library to latest master versions.
For #36905.
[git-generate]
go install golang.org/x/build/cmd/updatestd@latest
go install golang.org/x/tools/cmd/bundle@latest
updatestd -goroot=$(pwd) -branch=master
# Update a cmd/vet test case.
cat <<EOF | patch
diff --git a/src/cmd/vet/testdata/print/print.go b/src/cmd/vet/testdata/print/print.go
index a2ad0f1298..fffe571163 100644
--- a/src/cmd/vet/testdata/print/print.go
+++ b/src/cmd/vet/testdata/print/print.go
@@ -200,8 +200,8 @@ func PrintfTests() {
// Bad argument reorderings.
Printf("%[xd", 3) // ERROR "Printf format %\[xd is missing closing \]"
Printf("%[x]d x", 3) // ERROR "Printf format has invalid argument index \[x\]"
- Printf("%[3]*s x", "hi", 2) // ERROR "Printf format has invalid argument index \[3\]"
- _ = fmt.Sprintf("%[3]d x", 2) // ERROR "Sprintf format has invalid argument index \[3\]"
+ Printf("%[3]*s x", "hi", 2) // ERROR "Printf format %\[3\]\*s reads arg #3, but call has 2 args"
+ _ = fmt.Sprintf("%[3]d x", 2) // ERROR "Sprintf format %\[3\]d reads arg #3, but call has 1 arg"
Printf("%[2]*.[1]*[3]d x", 2, "hi", 4) // ERROR "Printf format %\[2]\*\.\[1\]\*\[3\]d uses non-int \x22hi\x22 as argument of \*"
Printf("%[0]s x", "arg1") // ERROR "Printf format has invalid argument index \[0\]"
Printf("%[0]d x", 1) // ERROR "Printf format has invalid argument index \[0\]"
EOF
# Temporarily hold x/net back to leave out CL 643780 because it's
# causing an import cycle in net/http's generated h2_bundle.go.
cd src
sed -i '' 's|"golang.org/x/net/internal/httpcommon"||' net/http/h2_bundle.go
go get golang.org/x/net@v0.34.1-0.20250123000230-c72e89d6a9e4 # version before CL 643780
go mod tidy
go mod vendor
go generate net/http
Change-Id: I91967ceb797bbc741af024cd2d2dba29dc558384
Reviewed-on: https://go-review.googlesource.com/c/go/+/648735
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
This change modifies this test (which involves an arbitrary timeout) to
be a little less flaky by double-checking that our subprocess program
completed even if the ticker fires and we've exceeded our timeout. The
logic behind this change is that the testing goroutine might get delayed
for any number of reasons, but the subprocess could still complete in
time. Still, the goroutine will wake up to handle the ticker and see its
over time, even though the event it was waiting for did actually happen.
I can't reproduce #71548 locally, so I suspect because this test calls
t.Parallel other load can delay the testing goroutine enough for this to
happen (especially with GODEBUG=gccheckmark=1, which pauses
everything to perform a full mark and sweep, and runtime tests love to
call runtime.GC).
For #71548.
Change-Id: I83e86a0115f65950886b57b5af0b4a517ef5f90f
Reviewed-on: https://go-review.googlesource.com/c/go/+/648576
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Commit-Queue: Michael Knyszek <mknyszek@google.com>
make.bat accepts the --dist-tool flag on multiple flag positions
and also allows omitting the trailing dash. Doing so adds complexity
and is not aligned with the make.bash and make.rc behavior. Remove that
flexibility to simplify the code and make it more consistent. This also
fixes a bug where dist.exe wouldn't be removed from cmd\dist when
running make.bat --dist-tool.
Also, there is no need for race.bat to invoke make.bat with --dist-tool.
It uses it to get the GOHOSTARCH env value, but we can already get
that from the built-in PROCESSOR_ARCHITECTURE env variable.
Change-Id: Ia673562c1ae6aff9bd3ec7aa8cdd25ff187eeb79
Reviewed-on: https://go-review.googlesource.com/c/go/+/648615
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Currently one of the reasons experimental events are tricky to use is
because:
- There's no way to take advantage of the existing infrastructure, like
strings and stacks, and
- There's no way to attach arbitrary data to an event (except through
strings, possibly).
Fix this by abstracting away the raw arguments in an ExperimentalEvent
and requiring access to the arguments via a new method, ArgValue. This
returns a Value, which gives us an opportunity to construct a typed
value for the raw argument dynamically, and a way to access existing
tables. The type of the argument is deduced from conventions for the
argument's name. This seems more than sufficient for experimental
events.
To make this work, we also need to add a "string" variant to the Value
type. This may be a little confusing since they're primarily used for
metrics, but one could imagine other scenarios in which this is useful,
such as including build information in the trace as a metric, so I think
this is fine.
This change also updates the Value API to accomodate a String method for
use with things that expect a fmt.Stringer, which means renaming the
value assertion methods to have a "To" prefix.
Change-Id: I43a2334f6cd306122c5b94641a6252ca4258b39f
Reviewed-on: https://go-review.googlesource.com/c/go/+/645135
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
parser.go is an old file that contains trace v1 definitions and a second
equivalent definition for stack frames. These are redundant and useless.
Delete these definitions and rename the file to fakep.go, which
describes the only thing left in this file, a bunch of fake P IDs used
by the trace viewer.
We should consider moving the fake P definitions elsewhere, too.
Change-Id: Ifd0768bd73c39009069445afe0155f1e352f00c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/644875
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This change moves maxArgs to tracev2 and renames it MaxTimedEventArgs.
It also updates the tests to make sure the specs conform to this
requirement.
Change-Id: I7b0c888a4dfd83306a470a4c9b0f9e44fe2e7818
Reviewed-on: https://go-review.googlesource.com/c/go/+/646016
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Adds ACVP test coverage for the SP 800-108r1 KDF feedback mode algorithm
based on the NIST spec:
https://pages.nist.gov/ACVP/draft-celi-acvp-kbkdf.html
The HKDF-based implementation in our FIPS module fixes some parameters,
requiring tailoring of the advertised capability to match. Notably:
* We only support fixedDataOrder "after fixed data"
* We only support a counter length of 8 bits
* We only support empty IVs
No acvp_test.config.json update accompanies this support because the
ACVP tests for this algorithm aren't amenable to fixed data testing.
Updates #69642
Change-Id: I729e899377a64d2b613d6435241aebabeef93bca
Reviewed-on: https://go-review.googlesource.com/c/go/+/640016
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Adds ACVP test coverage for the SP 800-108r1 KDF counter mode algorithm
based on the NIST spec:
https://pages.nist.gov/ACVP/draft-celi-acvp-kbkdf.html
The implementation in our FIPS module fixes some parameters, requiring
tailoring of the advertised capability to match. Notably:
* We only support macModes CMAC-AES-128, -192, and -256
* We only support supportedLengths 256 (matching the [32]byte output
from CounterKDF.DeriveKey)
* We only support fixedDataOrder "before fixed data"
* We only support counterLength 16
No acvp_test.config.json update accompanies this support because the
ACVP tests for this algorithm aren't amenable to fixed data testing.
Updates #69642
Change-Id: I9e02d6c8cb6e209ac8e4c9fba926fffbad916098
Reviewed-on: https://go-review.googlesource.com/c/go/+/639776
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Adds ACVP test coverage for the SP 800-90A rev 1 ctrDRBG algorithm based
on the NIST spec:
https://pages.nist.gov/ACVP/draft-vassilev-acvp-drbg.html#section-7.2
The implementation in our FIPS module is a minimal implementation
tailored to the specific needs of stdlib crypto. As a result we
customize the ACVP capability registration so that:
* predResistanceEnabled is false
* only mode AES-256 is supported
* for that mode,
* derFuncEnabled is false
* persoStringLen is 0 to disable personalization
* additionalInputLen is 384 to match the [48]byte argument in our API
Other capability values are chosen based on Table 4's ctrDRBG AES-256
w/o `derFuncEnabled` row:
https://pages.nist.gov/ACVP/draft-vassilev-acvp-drbg.html#section-7.4
We do enable reseed in the capability, necessitating two acvptool
commands: one that expects only 6 args and doesn't reseed
("ctrDRBG/AES-256"), and one that expects 8 args and does
("ctrDRBG-reseed/AES-256").
Updates #69642
Change-Id: I0f01a2f9496f45b130ee7d10916708093236f473
Reviewed-on: https://go-review.googlesource.com/c/go/+/639795
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Since CL 644215 each Sync event now represents the coming generation,
with a final Sync event emitted even when there's nothing ahead. This
change however failed to emit a Sync event at the end of a completely
valid generation when the next generation was invalid, causing the
runtime test TestCrashWhileTracing to start failing.
Fix this by emitting a final Sync event even when the next generation is
broken. We hold onto the error in parsing the next generation and emit
it after that final Sync event.
(Should these "final" Sync events distinguish themselves in some way?)
Fixes#71615.
Change-Id: I1f8abee5abaa39e1219e6fa05e9f82f1478db4c9
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/648195
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
These two packages were historically separate in an attempt to provide a
unified description of trace v1 and trace v2 formats. In practice this
turned out to be pointless, since it made more sense to keep the trace
v1 parser in a self-contained bubble with a converter to v2. Future
trace wire format migrations should probably just follow the same
general strategy, if there's a substantial change. (Minor changes can be
handled more organically.)
Change-Id: Ic765df62065fe53cfae59b505297527c3fa42dfb
Reviewed-on: https://go-review.googlesource.com/c/go/+/645395
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Currently all v2 trace versions, Go 1.22 and Go 1.23, share a full set
of specs. This is mostly OK, but it means quite a few events will be
accepted for 1.22 traces that should be rejected. This change fixes that
by limiting which event specs are returned by version.Version.Specs for
Go 1.22.
While we're here, let's be stricter about event names too, and move
tracev2.EventString to be a method on the version, so we can be more
precise. An intended consequence of this move is that tracev2 no longer
depends on fmt, since we will want the runtime to depend on tracev2 in
the near future.
Change-Id: If7285460c8ba59ab73da00993b7b12e61cdfe6a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/644219
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
This change follows up from the previous one which renamed oldtrace to
tracev1, defining everything Go 1.22+ as trace v2.
This change also re-maps some packages in preparation for sharing with
other parts of the standard library, like the runtime. It also cleans up
some other uses of 'go122' that are just a bit misleading. The mappings
are as follows:
- internal/trace/event -> internal/trace/tracev2/event
- internal/trace/event/go122 -> internal/trace/tracev2
- internal/trace/internal/testgen/go122 ->
internal/trace/internal/testgen
The CL updates all import paths and runs gofmt -w -s on the entire
subdirectory.
Change-Id: I35476c679a96d4eafad6b94bac5f88aa7b085d2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/644218
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently Root embeds a root and calls SetFinalizer on &r.root. This
sets the finalizer on the outer root, which is visible to users of
os.Root, and thus they can mutate the finalizer attached to it.
This change modifies Root to not embed its inner root, but rather to
refer to it by pointer. This allows us to set the finalizer on this
independent inner object, preventing users of os.Root from changing the
finalizer. This follows the same pattern as os.File's finalizer.
Fixes#71617.
Change-Id: Ibd199bab1b3c877d5e12ef380fd4647b4e10221f
Reviewed-on: https://go-review.googlesource.com/c/go/+/647876
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This is part of a refactoring to better distinguish trace wire format
versions. Even though details may change between Go versions and they
might be backwards-incompatible, the trace format still broadly has two
wire formats: v1 and v2.
A follow-up change will rename go122 to v2 to make this more consistent.
Change-Id: If4fe1c82d8aeabc8baa05f525e08a9e7d469a5c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/644217
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
This change modifies how per-generation experimental batches are
exposed. Rather than expose them on the ExperimentalEvent, it exposes it
as part of the Sync event, so it's clear to the caller when the
information becomes relevant and when it should be parsed.
This change also adds a field to each ExperimentalEvent indicating which
experiment the event is a part of.
Because this information needs to appear *before* a generation is
observed, we now ensure there is a sync event both before and after each
generation. This means the final sync event is now a special case;
previously we would only emit a sync event after each generation.
This change is based on feedback from Austin Clements on the
experimental events functionality.
For #62627.
Change-Id: I48b0fe12b22abb7ac8820a9e73447bfed8419856
Reviewed-on: https://go-review.googlesource.com/c/go/+/644215
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
nolocal is (almost) no longer needed after CL 647115. If we remove it,
then we can pass through all arguments to the Go command, which is
useful for running tests with additional flags, like -json or -v.
This CL also updates all.bat to use "go tool dist" instead of
"%GOTOOLDIR%/dist", as %GOTOOLDIR% is no longer set after making
make.bat uncoditionally set nolocal.
Change-Id: I97dc687faa5686d023f7d7d2b96637295995fe67
Reviewed-on: https://go-review.googlesource.com/c/go/+/647117
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
"if errorlevel 1" is and old construct that returns true if the
errorlevel is greater than or equal to 1. There are better alternatives
since Windows NT. For example, the || operator runs the RHS operand if
the preceding command failed, determined by checking that the errorlevel
is different from 0. This approach is more robust -it also works with
negative errorlevels- and is less verbose.
Change-Id: I2070d654d8f9dd41a6cd586ba5ad5f4fea0638ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/647136
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Adds ACVP test coverage for deterministic ECDSA based on the NIST spec:
https://pages.nist.gov/ACVP/draft-fussell-acvp-ecdsa.html
Notably there is no corresponding acvp_test.config.json update in this
commit because ACVP DetECDSA only specifies sigGen mode.
The ACVP ECDSA sigGen tests are not amenable to testing against
static data because the test vectors don't provide a key pair to use for
the signature, just the message. The module wrapper has to generate its
own keypair and return the public key components with the signature.
DetECDSA produces deterministic signatures only when signing the same
message with the same key.
Change-Id: I9921f52e943c96b32e02e79cb5556ba0fabeae17
Reviewed-on: https://go-review.googlesource.com/c/go/+/635341
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
%GOBUILDEXIT% is used to avoid closing the terminal window when the
build or the tests fail on a dev machine. It is only set in CI to get
a non-zero exit code in case of failure.
%GOBUILDFAIL% is used to pass the exit code from a child batch file to
the parent batch file. It is set to 1 in the child batch file if the
build or the tests fail.
These two variables add complexity to the batch files and impose some
limitations on how they are implemented. For example, the child files
can't use setlocal, as it would make the parent file unable to read the
%GOBUILDFAIL% variable.
This CL removes these two variables and replaces them with unconditional
calls to "exit /b 1" in case of failure, which is more idiomatic and
composable. The trick is that the "/b" parameter makes the exit only
apply to the current batch file, not the entire shell session (unless
the bat file is the root, in which case the parameter is ignored), so
the parent batch file can continue executing, potentially checking the
errorlevel of the child batch file (which we always set to 1).
Change-Id: Ib053fb181ab14d58679551e03485700de77878d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/647115
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
The existing implementation would either fail or bind to the wrong interface
when the requested interface had no IPv4 address, such as when the Ethernet cable
was unplugged.
Now on Linux, it will always bind to the requested interface.
On other operating systems, it will consistently fail if the requested interface
has no IPv4 address.
Fixes#70132
Change-Id: I22ec7f9d4adaa4b5afb21fc448050fb4219cacee
Reviewed-on: https://go-review.googlesource.com/c/go/+/644375
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
It appears to be quite easy to end up with a broken 'bzr' installation.
For example, if bzr was installed via a system-wide package manager and
intends to work with a system-wide Python installation, it may break if
another 'python3' binary is added to PATH.
If something as simple as 'bzr help' fails to exit with zero code,
consider it broken and skip tests that require a working bzr binary
just like if the 'bzr' binary isn't present in PATH at all.
This makes these tests more robust and capable of producing useful
signal in more environments. Separately from this, we'll want to
restore a working bzr installation on the linux-arm64 builders, but
at least there's still one on linux-amd64 builders.
For #71563.
Fixes#71504.
Change-Id: Ia147196f12b90a0731ebbfab63b5de308212ed65
Cq-Include-Trybots: luci.golang.try:gotip-linux-arm64-longtest,gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/646715
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Add a RISCV64 variable to cpu/internal that indicates both the presence
of RISC-V extensions and performance information about the underlying
RISC-V cores. The variable is only populated with non false values on
Linux. The detection code relies on the riscv_hwprobe syscall
introduced in Linux 6.4. The patch can detect RVV 1.0 and whether
the CPU supports fast misaligned accesses. It can only detect RVV 1.0
on a 6.5 kernel or later (without backports).
Updates #61416
Change-Id: I2d8289345c885b699afff441d417cae38f6bdc54
Reviewed-on: https://go-review.googlesource.com/c/go/+/522995
Reviewed-by: Joel Sing <joel@sing.id.au>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
This change ensures that dirinfo in the File struct is initialized atomically,
avoiding redundant allocations when multiple goroutines access it concurrently.
Instead of creating separate buffers, we now use CompareAndSwap to guarantee
thread-safe initialization and reduce unnecessary memory usage.
Although this is not a strict race condition, the update enhances efficiency by
eliminating duplicate allocations and ensuring safer concurrent access.
Fixes#71496.
Change-Id: If08699a94afa05611cdf67e82a5957a8d8f9d5c8
GitHub-Last-Rev: 1e1f619143
GitHub-Pull-Request: golang/go#71501
Reviewed-on: https://go-review.googlesource.com/c/go/+/645720
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
The linux-arm64 trybots are consistently failing with
vcstest_test.go:155: 2025/01/30 21:50:41 hello.txt:
> handle bzr
> env BZR_EMAIL='Russ Cox <rsc@google.com>'
> env EMAIL='Russ Cox <rsc@google.com>'
> bzr init-repo .
[stderr]
brz: ERROR: Couldn't import breezy and dependencies.
Please check the directory containing breezy is on your PYTHONPATH.
Error: PyErr { type: <class 'ModuleNotFoundError'>, value: ModuleNotFoundError("No module named 'breezy'"), traceback: None }
vcstest_test.go:161: hello.txt:6: bzr init-repo .: exit status 1
This seems to be a problem with the builder.
For now, skip the test if we see that error message, just as we already
skip the test if the bzr executable is not found.
For #71504
Change-Id: If8b6d4dea02dc16198ba6067595dff3340a81299
Reviewed-on: https://go-review.googlesource.com/c/go/+/646635
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Check correct use of ...'s in parameter lists in parsers.
This allows the type checkers to assume correct ASTs with
respect to ... use.
Adjust some error messages: if a ... is used in a result
parameter list, the error is now more accurate.
Eliminate a now unused error code.
Change-Id: I66058e114e84805e24c59e570604b607ef5ff1fe
Reviewed-on: https://go-review.googlesource.com/c/go/+/631135
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Bypass: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Refactor parser.parseParameters to only parse
ordinary parameters. Introduce a variant to
parse type parameters.
In the two places where we need ordinary and type
parameters, call the function twice.
Also, use a range loop in two places which is a
bit easier to read.
Change-Id: I0a62e1c508d6ccd16b7cb6e1b852ab1d32224ec2
Reviewed-on: https://go-review.googlesource.com/c/go/+/630816
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Bypass: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Commit-Queue: Robert Griesemer <gri@google.com>
For some reason the ESTALE error message differed on Linux systems.
On Linux strerror normally returns "Stale file handle" for ESTALE,
except possibly in the en_GB locale. The mkerrors.sh script sets
LC_ALL=C, so it should always produces "stale file handle".
However, for some reason, several targets use "stale NFS file handle"
instead.
Clean this up so that we use the same string on all Linux systems.
This is also consistent with golang.org/x/sys/unix.
Fixes#71309
Change-Id: Ic2ffaf114c85112bc6d0831e43dd5fd2f4237bc2
Reviewed-on: https://go-review.googlesource.com/c/go/+/643335
Auto-Submit: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
This is a simple move of the contents of the vendored x/net/route
to internal/routebsd. I've also added some test files that
were not previously vendored.
This next CL will simplify the new internal/routebsd, removing the
code that is not needed by the new package.
This is a step toward simplifying the x/net/route package by
permitting it to import x/sys/unix.
Change-Id: I4d13df11fa9738cd68876b2ea456d03f82d8d64a
Reviewed-on: https://go-review.googlesource.com/c/go/+/637695
Commit-Queue: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
The coverageredesign experiment was turned on by default by
CL 436236 in September, 2022. We've documented it and people
are using it. This CL removes the ability to turn off the experiment.
This removes some old code that is no longer being executed.
For #51430
Change-Id: I88d4998c8b5ea98eef8145d7ca6ebd96f64fbc2b
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-darwin-amd64-longtest,gotip-linux-arm64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/644997
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
After using strings.FieldsFuncSeq, the number of memory allocations has been reduced from 2 to 0.
The following is the complete benchamark code and results:
package main
import (
"strings"
"testing"
)
func isSlashRune(r rune) bool { return r == '/' || r == '\\' }
func containsDotDotLoop(v string) bool {
if !strings.Contains(v, "..") {
return false
}
for _, ent := range strings.FieldsFunc(v, isSlashRune) {
if ent == ".." {
return true
}
}
return false
}
func containsDotDotSeq(v string) bool {
if !strings.Contains(v, "..") {
return false
}
for ent := range strings.FieldsFuncSeq(v, isSlashRune) {
if ent == ".." {
return true
}
}
return false
}
func BenchmarkDotDot(b *testing.B) {
testCases := []string{
"/path/to/somewhere",
"/path/../to/somewhere",
"/really/long/path/with/many/segments",
"../../../deep/path",
}
b.Run("Loop", func(b *testing.B) {
for i := 0; i < b.N; i++ {
for _, tc := range testCases {
containsDotDotLoop(tc)
}
}
})
b.Run("Seq", func(b *testing.B) {
for i := 0; i < b.N; i++ {
for _, tc := range testCases {
containsDotDotSeq(tc)
}
}
})
}
go test -bench=. -benchmem
goos: darwin
goarch: arm64
pkg: bc
cpu: Apple M1
BenchmarkDotDot/Loop-8 6133270 193.7 ns/op 144 B/op 2 allocs/op
BenchmarkDotDot/Seq-8 23172360 51.19 ns/op 0 B/op 0 allocs/op
PASS
ok bc 2.633s
Change-Id: I529c296e701b22710e21b53877aa798799980a3b
Reviewed-on: https://go-review.googlesource.com/c/go/+/639536
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Previously, the implementation used bit manipulation to approximate
counters to the nearest power of two.
Given the 0-255 range of byte, we can precompute values at
initialization and use a lookup table, reducing runtime computation.
Benchmarks show an 18% performance gain on AMD64 and 5% on ARM64.
* net/netip/FuzzParse (n=10, t=60s, state reset per run)
* AMD64 (Intel Alder Lake i5-12600k):
17,349,217 -> 20,487,756 execs/s
* ARM64 (M3 Pro):
19,606,471 -> 20,657,041 execs/s
* compress/gzip/FuzzReader (n=10, t=60s, mature corpus)
* AMD64 (Intel Alder Lake i5-12600k):
5,655,956 -> 6,707,035 execs/s
Change-Id: If11f7fe866f54c7cd2c5a48e251c027b67980df7
Reviewed-on: https://go-review.googlesource.com/c/go/+/627378
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>
Added implementations for *io/fs.subFS, os.DirFS, and testing/fstest.MapFS.
Amended testing/fstest.TestFS to check behavior.
Addressed TODOs in archive/tar and os.CopyFS around symbolic links.
I am deliberately not changing archive/zip in this CL,
since it currently does not resolve symlinks
as part of its filesystem implementation.
I am unsure of the compatibility restrictions on doing so,
so figured it would be better to address independently.
testing/fstest.MapFS now includes resolution of symlinks,
with MapFile.Data storing the symlink data.
The behavior change there seemed less intrusive,
especially given its intended usage in tests,
and it is especially helpful in testing the io/fs function implementations.
Fixes#49580
Change-Id: I58ec6915e8cc97341cdbfd9c24c67d1b60139447
Reviewed-on: https://go-review.googlesource.com/c/go/+/385534
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Funda Secgin <fundasecgin33@gmail.com>
Methods declared in an interface have a signature and FuncType in the
AST, but they do not express a syntactic function type expression.
Treat them like ordinary function/method declarations and do not record
them in the Info.Types map. This removes an inconsistency in the way
function types are recorded.
Follow-up on CL 640776.
For #70908.
Change-Id: I60848f209b40b008039c014fb8b7b279361487b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/640596
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
This change fixes GODEBUG=gccheckmark=1 which seems to have bit-rotted.
Because the root jobs weren't being reset, it wasn't doing anything.
Then, it turned out that checkmark mode would queue up noscan objects in
workbufs, which caused it to fail. Then it turned out checkmark mode was
broken with user arenas, since their heap arenas are not registered
anywhere. Then, it turned out that checkmark mode could just not run
properly if the goroutine's preemption flag was set (since
sched.gcwaiting is true during the STW). And lastly, it turned out that
async preemption could cause erroneous checkmark failures.
This change fixes all these issues and adds a simple smoke test to dist
to run the runtime tests under gccheckmark, which exercises all of these
issues.
Fixes#69074.
Fixes#69377.
Fixes#69376.
Change-Id: Iaa0bb7b9e63ed4ba34d222b47510d6292ce168bc
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/608915
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Enable the cmd/go fips test now that v1.0.0.zip has been checked in.
Will still need to enable the alias half when the alias is checked in.
Also fix a problem that was causing spurious failures, by fixing
repeated unpackings and also disabling modindex reads of the
virtual fips140 snapshot directories.
Fixes#71491.
Change-Id: I7fa21e9bde07ff4eb6c3483e99d49316ee0ea7f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/645835
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
In ProcessCoverTestDir pass the selected set of packages to
EmitTextual in addition to EmitPercent, so that when we have runs with
multiple packages selected but without -coverpkg, text format output
for package P was incorrectly including output for P's covered
dependencies. This is in effect an extension of the fix for issue
65570.
Includes a cmd/go script test to verify correct behavior; ideally it
would be nice to locate this test in .../internal/coverage somewhere
but at the moment script tests are only supported for
cmd/{go,compile,link}.
Updates #65570.
Fixes#70244.
Change-Id: Ia0bb10155353aa0f2ead46e81a2aaa71bde4ef82
Reviewed-on: https://go-review.googlesource.com/c/go/+/627316
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Than McIntosh <thanm@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL reintroduces the various mapiter* linkname functions with a
compatibility layer that is careful to maintain compatibility with users
of the linkname.
The wrappers are straightforward. Callers of these APIs get an extra
layer of indirection, with their hiter containing a pointer to the real
maps.Iter. These users will take a minor performance hit from the extra
allocation, but this approach should have good long-term
maintainability.
Fixes#71408.
Change-Id: I6a6a636c7574bbd670ff5243dfeb63dfba6dc611
Reviewed-on: https://go-review.googlesource.com/c/go/+/643899
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
mapiterinit allows external linkname. These users must allocate their
own iter struct for initialization by mapiterinit. Since the type is
unexported, they also must define the struct themselves. As a result,
they of course define the struct matching the old hiter definition (in
map_noswiss.go).
The old definition is smaller on 32-bit platforms. On those platforms,
mapiternext will clobber memory outside of the caller's allocation.
On all platforms, the pointer layout between the old hiter and new
maps.Iter does not match. Thus the GC may miss pointers and free
reachable objects early, or it may see non-pointers that look like heap
pointers and throw due to invalid references to free objects.
To avoid these issues, we must keep mapiterinit and mapiternext with the
old hiter definition. The most straightforward way to do this is to use
mapiterinit and mapiternext as a compatibility layer between the old and
new iter types.
The first step to that is to move normal map use off of these functions,
which is what this CL does.
Introduce new mapIterStart and mapIterNext functions that replace the
former functions everywhere in the toolchain. These have the same
behavior as the old functions.
This CL temporarily makes the old functions throw to ensure we don't
have hidden dependencies on them. We cannot remove them entirely because
GOEXPERIMENT=noswissmap still uses the old names, and internal/goobj
requires all builtins to exist regardless of GOEXPERIMENT. The next CL
will introduce the compatibility layer.
I want to avoid using linkname between runtime and reflect, as that
would also allow external linknames. So mapIterStart and mapIterNext are
duplicated in reflect, which can be done trivially, as it imports
internal/runtime/maps.
For #71408.
Change-Id: I6a6a636c6d4bd1392618c67ca648d3f061afe669
Reviewed-on: https://go-review.googlesource.com/c/go/+/643898
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
The TestRootConcurrentClose test can fail when GOARCH=WASM because of
goroutine starvation. The spawned goroutine will sometimes run in a
loop and never have the main goroutine be scheduled. This causes the
test to fail due to a timeout. This change forces the goroutine to be
scheduled with each iteration of the loop when GOARCH=WASM.
For #71134Fixes#71117
Change-Id: I4fb68907c9ac3b33bd0572d5e5db2974a3379191
Reviewed-on: https://go-review.googlesource.com/c/go/+/640195
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Pull in x/net CL 642606 and CL 643256 and regenerate h2_bundle.go:
http2: disable extended CONNECT by default
http2: encode :protocol pseudo-header before regular headers
For #36905.
Fixes#70728.
Fixes#71128.
[git-generate]
go install golang.org/x/build/cmd/updatestd@latest
go install golang.org/x/tools/cmd/bundle@latest
updatestd -goroot=$(pwd) -branch=internal-branch.go1.24-vendor
Change-Id: Id853cb96f8fc410956666f5c3ab4c5889c703503
Reviewed-on: https://go-review.googlesource.com/c/go/+/642398
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
If a wasmexport function is called from the host before
initializing the Go Wasm module, currently it will likely fail
with a bounds error, because the uninitialized SP is 0, and any
SP decrement will make it out of bounds.
As at least some Wasm runtime doesn't call _initialize by default,
This error can be common. And the bounds error looks confusing to
the users. Therefore, we detect this case and emit a clearer error.
Fixes#71240.
Updates #65199.
Change-Id: I107095f08c76cdceb7781ab0304218eab7029ab6
Reviewed-on: https://go-review.googlesource.com/c/go/+/643115
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Newer git versions (at least git 2.47.1) do not send all the matching tags
for a shallow fetch of a specific hash anymore. The go command assumes
that git servers do this. Since that assumption is broken, use the local
copy of the remote refs list to augment the tags sent by the server.
This makes the cmd/go/internal/modfetch tests pass again with newer git.
Fixes#71261.
Change-Id: I9fd4f3fd7beeb68a522938599f8f3acd887d0b26
Reviewed-on: https://go-review.googlesource.com/c/go/+/642437
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>
https://ci.chromium.org/ui/inv/build-8725798219051312433/test-results?sortby=&groupby=
shows a mysterious failure with this stack:
=== RUN BenchmarkAll
BenchmarkAll
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x7c497f]
goroutine 20 gp=0xc000004000 m=7 mp=0xc000182808 [running]:
panic({0x81c5e0?, 0xabc6b0?})
/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:806 +0x168 fp=0xc00c7ffce0 sp=0xc00c7ffc30 pc=0x4ad4c8
runtime.panicmem(...)
/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/panic.go:262
runtime.sigpanic()
/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/signal_unix.go:925 +0x359 fp=0xc00c7ffd40 sp=0xc00c7ffce0 pc=0x4af6d9
cmd/api.(*Walker).export(0xc000034100, 0x0)
/home/swarming/.swarming/w/ir/x/w/goroot/src/cmd/api/main_test.go:193 +0x3f fp=0xc00c7ffe08 sp=0xc00c7ffd40 pc=0x7c497f
cmd/api.BenchmarkAll(0xc000214288)
/home/swarming/.swarming/w/ir/x/w/goroot/src/cmd/api/api_test.go:205 +0x207 fp=0xc00c7ffeb0 sp=0xc00c7ffe08 pc=0x7c1c07
testing.(*B).runN(0xc000214288, 0x1)
/home/swarming/.swarming/w/ir/x/w/goroot/src/testing/benchmark.go:202 +0x291 fp=0xc00c7fff78 sp=0xc00c7ffeb0 pc=0x57e611
testing.(*B).run1.func1()
/home/swarming/.swarming/w/ir/x/w/goroot/src/testing/benchmark.go:224 +0x7c fp=0xc00c7fffe0 sp=0xc00c7fff78 pc=0x57f11c
runtime.goexit({})
/home/swarming/.swarming/w/ir/x/w/goroot/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00c7fffe8 sp=0xc00c7fffe0 pc=0x4b4a61
created by testing.(*B).run1 in goroutine 1
/home/swarming/.swarming/w/ir/x/w/goroot/src/testing/benchmark.go:217 +0x173
So import_ must have returned an error, making pkg nil. Show that error.
Also do the same at the other calls to import_.
Change-Id: Ie782571c4bda3334a86b303f61969cf1cc7d3c32
Reviewed-on: https://go-review.googlesource.com/c/go/+/642438
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
This case recently started happening on the builders.
The synctest experiment was recently enabled for some targets (CL 642422).
This caused the list of standard packages to include testing/synctest.
However, BenchmarkAll tests for all configurations;
some did not include testing/synctest. That caused the test to crash.
Change-Id: Icade10af147c2e2bcbac25bf744919083db3e70f
Reviewed-on: https://go-review.googlesource.com/c/go/+/642397
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
The existing code assumed the type argument count check in
Checker.instance couldn't fail for generic alias types
(matching the code for generic signatures), but it actually
can.
Adjust the code accordingly and document that the result of
Checker.instance may be invalid.
Review all call sites of Checker.instance and make sure we
handle the failure case, or document the code accordingly
(in the case of generic signatures).
When reporting an type argument count error, use the alias
name rather than the alias string representation to match
the error we get for a non-alias type.
While at it, update the manual.go template for ease of use.
Fixes#71198.
Change-Id: I6d19ec6418440e9b49574a2d7dd9825e0af6c2fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/641857
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Even if an error occurs during unmarshal, check the resulting Go value.
The documented API specifies no guarantees on how much of a Go value
will be populated when an error occurs and the "json" package
is technically not bounded by the Go compatibility agreement
to ensure this behavior never changes.
However, there is still value in running checks for
what exactly what is partially mutated in the event of an error
even if this is not guaranteed behavior.
Change-Id: I6e923a31f77768a14c4adfb0d37dbeee5807a4a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/642275
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Adds ACVP test coverage for the hmacDRBG algorithm based on the NIST
spec:
https://pages.nist.gov/ACVP/draft-vassilev-acvp-drbg.html#section-7.2
The HMAC DRBG algorithm in our fips module is a minimal implementation
tailored for use for generating ECDSA nonces and so lives in
crypto/internal/fips140/ecdsa.
In order to be testable by crypto/internal/fips140test this changeset
exports a ecdsa.TestingOnlyNewDrbg() constructor to support the ACVP use-case.
All FIPS-compatible SHA2 and SHA3 digests are tested.
The ACVP capability registration is customized to match the limited
capabilities of our ecdsa-focused impl. Most notably:
* reseedImplemented is false - we expect this impl to be invoked
only once or twice per instantiation and do not support explicit
reseeding.
* predResistanceEnabled is false - this requires reseeding.
* Per mode:
* derFuncEnabled is always false - this is only used by ctrDRBG.
* additionalInputLen is 0 for all modes - this is only used with
preResistanceEnabled.
The other capability values are chosen based on Table 4:
https://pages.nist.gov/ACVP/draft-vassilev-acvp-drbg.html#section-7.4
Updates #69642
Change-Id: Ia58979d691f912e2ed739a05efb719f580fbbf89
Reviewed-on: https://go-review.googlesource.com/c/go/+/639775
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Adds ACVP test coverage for ML-KEM based on the NIST spec:
https://pages.nist.gov/ACVP/draft-celi-acvp-ml-kem.html
Notably we need to update the BoringSSL module version because the
acvptool was only recently updated to support testing ML-KEM.
A few non-test updates are also required for the
crypto/internal/fips140/mlkem package:
* For keyGen tests a new ExpandedBytes768() function is added that
converts a DecapsualtionKey768 struct into the expanded NIST
serialization. The existing Bytes() function returns the
key's seed, while ACVP testing requires the more cumbersome format.
* For decap tests a new TestingOnlyNewDecapsulationKey768()
constructor is added to produce a DecapsulationKey768 struct from the
expanded FIPS 203 serialization provided by the ACVP test vector. The
pre-existing NewDecapsulationKey768() function expects a seed as
input.
The generate1024.go helper is updated to translate the above changes to
the generated mlkem1024.go implementation.
Both of these new functions are exclusively for ACVP usage and so not
present in the public mlkem API. End users should always prefer to work
with seeds.
Updates #69642
Change-Id: I79784f8a8db00a2ddefdcece4b8de50b033c8f69
Reviewed-on: https://go-review.googlesource.com/c/go/+/637439
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Heap profiles hide "runtime" frames like runtime.mapassign. This broke
in 1.24 because the map implementation moved to internal/runtime/maps,
and runtime/pprof only considered literal "runtime." when looking for
runtime frames.
It would be nice to use cmd/internal/objabi.PkgSpecial to find runtime
packages, but that is hidden away in cmd.
Fixes#71174.
Change-Id: I6a6a636cb42aa17539e47da16854bd3fd8cb1bfe
Reviewed-on: https://go-review.googlesource.com/c/go/+/641775
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
The Mmap function returns a Data struct containing a slice with the
mapped contents of the file. Before this change, on Windows, the slice
contained the contents of all the pages of the mapping, including past
the end of the file. Re-slice the slice to the length of the file (if
if the slice is longer) so that the slice contains only the data in the
file.
For #71059
Change-Id: I389b752505b6fa1252b5c6d836a37bc7e662a45d
Reviewed-on: https://go-review.googlesource.com/c/go/+/640155
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Both fips140only and the service indicator checks in
crypto/internal/fips140/... expect to type assert to
crypto/internal/fips140/{sha256,sha512,sha3}.Digest.
However, crypto/sha3 returns a wrapper concrete type around sha3.Digest.
Add a new fips140hash.Unwrap function to turn the wrapper into the
underlying sha3.Digest, and use it consistently before calling into
fips140only or the FIPS 140-3 module.
In crypto/rsa, also made the fips140only checks apply consistently after
the Go+BoringCrypto shims, so we can instantiate the hash, and avoid
having to wrap the New function. Note that fips140=only is incompatible
with Go+BoringCrypto.
Fixes#70879
Change-Id: I6a6a4656ec55c3e13f6cbfadb9cf89c0f9183bdc
Reviewed-on: https://go-review.googlesource.com/c/go/+/640855
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The constants appeared badly ordered and grouped in the godoc before
const (
CiphertextSize1024 = 1568
EncapsulationKeySize1024 = 1568
)
const (
SharedKeySize = 32
SeedSize = 64
CiphertextSize768 = 1088
EncapsulationKeySize768 = 1184
)
while now they are a single group with the good size first
const (
SharedKeySize = 32
SeedSize = 64
CiphertextSize768 = 1088
EncapsulationKeySize768 = 1184
CiphertextSize1024 = 1568
EncapsulationKeySize1024 = 1568
)
No code changes.
Change-Id: I6a6a4656961b1e8c8bca3992aafa33e0575af8a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/640997
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Having a global lock on the random state (used only in FIPS-140 mode)
introduces contention in concurrent programs. Use an approximately
per-P random state instead, using sync.Pool to manage per-P state.
This code is important to land for the Go 1.24 release because it is
part of the FIPS-140 module that will be validated and certified,
so it will live for a long time. We otherwise wouldn't be able to
correct this contention for at least a year, perhaps more.
At the same time, the code is only used in the FIPS-140 mode,
so there is no risk to normal programs.
Fixes#71155.
Change-Id: I6b779f15ddfdf232f608f5cda08f75906e58114f
Reviewed-on: https://go-review.googlesource.com/c/go/+/641097
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The backslashes on the windows paths will be escaped, so when checking
for them in the regular expression we'd have to have quadruple
backslashes '\\\\'. Since it's difficult to escape $GOCACHEPROG properly
for both json and regexp, just check for a string that ends in
cacheprog$GOEXE. We already check that the proper value is reported in
go env and go env -changed, and the json test case is mostly useful to
verify that GOCACHEPROG shows up in the json output.
For #71059
Change-Id: I52d49de61f2309a139f84c4d232b4cd94546ec8c
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/641375
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Currently injectglist emits all the trace events before actually calling
casgstatus on each goroutine. This is a problem, since tracing can
observe an inconsistent state (gstatus does not match tracer's 'emitted
an event' state).
This change fixes the problem by having injectglist do what every other
scheduler function does, and that's wrap each call to casgstatus in
traceAcquire/traceRelease.
Fixes#70883.
Change-Id: I857e96cec01688013597e8efc0c4c3d0b72d3a70
Reviewed-on: https://go-review.googlesource.com/c/go/+/638558
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
On Windows, we can't open a file that's already been opened. Before this
change, we'd try to write an index entry if mmapping the entry failed.
But that could happen either if the file doesn't exist or if there was a
problem mmapping an already opened file. Pass through information about
whether the file was actually opened so that we don't try to write to an
already opened file.
For #71059
Change-Id: I6adabe1093fed9ec37e7fafb13384c102786cbce
Reviewed-on: https://go-review.googlesource.com/c/go/+/640577
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
These break if the tools are run with GODEBUG=fips140=only,
which happens if someone sets that during 'go test' (and a test
binary must be built).
The easiest fix is to make the tools compatible with this GODEBUG
by just using sha256 as the underlying hash always. Just in case,
I made the wrappers select different sections of the hash, but
none of the call sites really care.
This CL is for the Go 1.24 release, but a follow-up during the Go 1.25
dev cycle could change all the usage sites to only use Sum32/New32.
For #70514Fixes#70878
Change-Id: Id5fea779c83df51d1680dbe561e0949c56e8d1e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/641096
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Allow a gocacheprog to not respond to close. The intention of the code
is that after we send the close message we'd ignore errors reading from
the cacheprog's stdout. But before this change if a cacheprog
did not respond to close and we got an EOF reading from the cacheprog's
stdout we'd just ignore all pending requests. The send operation would
then block forever waiting for a response. With this change, we close
all response channels for pending responses if there's an error reading
from the cacheprog's stdout while we're closing. The receives from the
response channels would then proceed (but now have to handle a nil
value). Then the send operation would return and the (*ProgCache).Close
function can proceed.
Fixes#70848
Change-Id: I6631d317ba7aea3f25f714f31cd2aeef0f4d4e3e
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/640516
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Function types for function (and method) declarations do not
appear in Info.Types maps, only Info.Defs maps, because the
function type is implicit in the declaration and not a proper
(function) type expression. This is true even though the AST
represents these types via an (artificial) FuncType node.
Document this explicitly in the API.
No functional code changes.
Fixes#70908.
Change-Id: I2aa897daed04e7ad0fa8b625d9adc7b423c57387
Reviewed-on: https://go-review.googlesource.com/c/go/+/640776
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
disasm_riscv64 currently always returns an instruction length of four,
which is not correct if compressed instructions are in use. Return the
length of the decoded instruction, defaulting to two bytes if the
instruction is unknown.
With this change it is possible to correctly objdump a binary that is
written in C and includes compressed instructions:
$ go tool objdump ./hello
TEXT _start(SB)
:0 0x5b0 ef002002 CALL 8(PC)
:0 0x5b4 aa87 ADD X10, X0, X15
:0 0x5b6 17250000 AUIPC $2, X10
:0 0x5ba 033525a3 MOV -1486(X10), X10
:0 0x5be 8265 MOV (X2), X11
:0 0x5c0 3000 ADDI $8, X2, X12
...
Fixes#71102
Change-Id: Ia99eb114a98c6d535de872ce8a526cd5e6203fff
Reviewed-on: https://go-review.googlesource.com/c/go/+/639995
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Jorropo <jorropo.pgm@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
This has no practical advantage, and requires extra variable time code,
but is an explicit FIPS 186-5 requirement.
Note that the new behavior is consistent with Go+BoringCrypto, but not
with Go 1.23. The resulting keys are essentially interchangeable, but
it's not impossible for applications to notice (google/go-tpm#383).
gcd_lcm_tests.txt is from BoringSSL.
Change-Id: I6a6a4656fd5e92912c87bedc667456d0e8787023
Reviewed-on: https://go-review.googlesource.com/c/go/+/639936
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
This test checks a use-case of sync.Map that's expected to be more
common in Go 1.24 and beyond, as a concurrent weak cache.
The test will also fail if CompareAndSwap is not properly atomic with
CompareAndDelete, which is what #70970 is actually about. We should have
more explicit tests checking mutual atomicity of operations, but
for now this is OK, and still useful.
For #70970.
Change-Id: I6db508660691586a8af9ad511c9a96432d333343
Reviewed-on: https://go-review.googlesource.com/c/go/+/640737
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
In documentation, we've usually but not always referred to a
context with a closed Done channel as "done" rather than
"canceled", to avoid ambiguity between a context canceled
by calling a CancelFunc and one past its deadline.
This actually adds ambiguity, however, since it's common to
see references to a "canceled context" that are intended to
cover contexts past their deadline. If you see "function F
returns if its context is canceled", you can reasonably
assume that F will return if its context passes its
deadline, unless something says otherwise.
Update the context package docs to explicitly state that
a context is canceled when its deadline passes. Drop references
to contexts becoming "done" and just use "canceled" throughout.
Fixes#70945
Change-Id: I99fbd800c6049deaa37015a304f7f9d9a84100e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/640095
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
When ECH is rejected, properly take retry configs from the encrypted
extensions message. Also fix the bogo shim to properly test for this
behavior.
We should properly map the full BoringSSL -> Go errors so that we don't
run into a similar failure in the future, but this is left for a follow
up CL.
Fixes#70915
Change-Id: Icc1878ff6f87df059e7b83e0a431f50f1fea833c
Reviewed-on: https://go-review.googlesource.com/c/go/+/638583
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Add a section on the representation of values:
distinguish between values that are self-contained
and values that contain references while avoiding
the notion of "reference types" which is misleading.
Also, use "predeclared identifier nil" rather than
"predeclared value nil" because it is the identifier
that is predeclared.
Fixes#5083.
Change-Id: I2235673c6404f2c055f195e879f198c7ab246d58
Reviewed-on: https://go-review.googlesource.com/c/go/+/635801
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Bypass: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
It is not true that Unmarshal always treats a JSON null
as being equivalent to a no-op.
For bools, ints, uints, floats, strings, arrays, and structs,
it treats a JSON null as a no-op. However, for []byte, slice,
map, pointer, or interface, it zeros the underlying value.
Remove this suggestion as the actual behavior is inconsistent.
Note that the proposed behavior in v2 Unmarshal is to consistently
zero out the underlying value.
Change-Id: I02cef0bf7919f25cfd0aceb04486d37498761181
Reviewed-on: https://go-review.googlesource.com/c/go/+/638416
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
The GNU linker interprets @file as "read command-line options from file".
Thus, we forbid values starting with @ on linker flags. However, this
causes a problem when targeting Darwin. @executable_path, @loader_path, and
@rpath are special values used in Mach-O to change the library search path
and can be used in conjunction with the -install_name and -rpath linker
flags. Since the GNU linker does not support Mach-O, targeting Darwin
implies not using the GNU linker. Therefore, we allow @ in the linker flags
if and only if cfg.Goos == "darwin".
Fixes#40559
Change-Id: I0896758f0835e444ea0d501ea3fd8423cff97a27
GitHub-Last-Rev: 2b81dcd12e
GitHub-Pull-Request: golang/go#70939
Reviewed-on: https://go-review.googlesource.com/c/go/+/638075
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
In introducing iterators, package iter gives an example of how to
use an iterator in a range-over-func loop, but currently does not
give an example of what an iterator implementation might look like.
This change adds the example of map.Keys() before the usage example.
Additionally, it references to the Go blog for further examples,
as well as the language spec about for-range loops.
Fixes#70986
Change-Id: I7108d341d314d7de146b4c221700736c943a9f5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/638895
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
On AIX, an R_ADDR relocation from an RODATA symbol to a DATA
symbol does not work, as the dynamic loader can change the address
of the data section, and it is not possible to apply a dynamic
relocation to RODATA. In order to get the correct address, we
apply the delta between unrelocated and relocated data section
addresses at run time. The linker saves both the unrelocated and
the relocated addresses, so we can compute the delta.
This is possible because RODATA symbols are generated by the
compiler and so we have full control of. On AIX, the only case
is the on-demand GC pointer masks from the type descriptors, for
very large types.
Perhaps there is a better way.
Fixes#70483.
Change-Id: I2664c0a813b38f7b146794cb1e73ccf5e238ca65
Reviewed-on: https://go-review.googlesource.com/c/go/+/638016
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
len(map) is lowered to loading the first field of the map
structure, which is the length. Currently it is a load of an int.
With the old map, the first field is indeed an int. With Swiss
map, however, it is a uint64. On big-endian 32-bit machine,
loading an (32-bit) int from a uint64 would load just the high
bits, which are (probably) all 0. Change to a load with the proper
type.
Fixes#70248.
Change-Id: I39cf2d1e6658dac5a8de25c858e1580e2a14b894
Reviewed-on: https://go-review.googlesource.com/c/go/+/638375
Run-TryBot: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
We were trying to keep all binaries stale in fips140 mode
so that every build would write and leave behind a fips.o
in the work directory for use by validating labs.
That breaks various staleness checks, including the one
in cmd/dist during GOFIPS140=latest ./make.bash.
Revert the fips140 hack. Validating labs will still be able
to find the fips.o when building against a clean cache.
Add the default godebug to the link hash though,
so that it is clear that GOFIPS140=latest
and GOFIPS140=off binaries have different hashes.
(The only effect is the default GODEBUG setting.)
They already had different hashes, because the
default GODEBUG ends up in p.Internal.BuildInfo,
and that gets hashed in a "modinfo" line,
but better to be explicit.
Fixes#70873.
Change-Id: I49a38c180208098c2b6720facef48f4e96d44c54
Reviewed-on: https://go-review.googlesource.com/c/go/+/637116
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
If we do one upgrade because of a go install target's go.mod file,
we still might need a second upgrade to implement the GOTOOLCHAIN
minimum. Instead of allowing a two-step switch (which we were
cutting off anyway), skip the first step and go straight to the
GOTOOLCHAIN min upgrade.
Fixes#69051.
Change-Id: I16f060f473574d8b8f84c55fae2fd0cdabc8aa19
Reviewed-on: https://go-review.googlesource.com/c/go/+/637496
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
CL 603959 added T.Context for #36532.
The discussion on the proposal only mentions t.Context.
However, the implementation of CL 603959 also added B.Context and F.Context.
They were added to the API listing, and B.Context was mentioned in
the release notes.
Unfortunately, the new B.Context and F.Context methods always
returned nil, rather than a context.Context value.
This change adds a working implementation of B.Context and F.Context.
For #36532Fixes#70866
Change-Id: I8a44e6649fb658e4f641ffb7efd08b4374f578ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/637236
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Per ISO/IEC 19790:2012, Section 7.4.3.1.
> A cryptographic module shall [04.12] provide the following services to
> operators.
>
> a) Show module’s versioning information. The cryptographic module
> shall [04.13] output the name or module identifier and the versioning
> information that can be correlated with a validation record (e.g.
> hardware, software and/or firmware versioning information)."
For #69536
Change-Id: I8061f64e4ae60a4666f6abd892cb1301d6bf2452
Reviewed-on: https://go-review.googlesource.com/c/go/+/636558
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
On linux/mips64, the syscall.Stat_t struct does not match the
kernel version of the struct. Functions that operate on a Stat_t
translate between it and the kernel struct.
The fstatat function was not doing this translation.
Make it do so.
Export a syscall.Fstatat on mips64 for usage by
internal/syscall/unix. Perhaps we should just do this on all
architectures, but this is the smaller change for now.
Fixes#70659
Change-Id: I38e36473689be25861953b418c9abc5b270a7bcf
Reviewed-on: https://go-review.googlesource.com/c/go/+/633280
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
- Describe that function invocation allocates space for a functions'
variables.
- Explain parameter passing in terms of assignments.
Change-Id: Ia693d73a570f7d1aa2ac05e6095b4e602e4e9bf2
Reviewed-on: https://go-review.googlesource.com/c/go/+/635800
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Bypass: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Rob Pike <r@golang.org>
NIST SP 800-131Ar3 ipd, scheduled for publication in 2025Q1, marks
AES-ECB as disallowed for encryption, and legacy use for decryption.
There are apparently no details on how the transition is going to work,
so to avoid surprises we just mark direct use of the Block as
non-approved.
We need to use Encrypt from higher level modes without tripping the
service indicator. Within the aes package, we just use the internal
function. For the gcm package we could do something more clever, but
this deep into the freeze, just make an exported function that we commit
to use nowhere else.
I could not figure out a decent way to block ECB on GODEBUG=fips140=only.
For #69536
Change-Id: I972a4b5da8efd0a0ab68d7dd509bec73aa2d6b68
Reviewed-on: https://go-review.googlesource.com/c/go/+/636775
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
If a package is incomplete, don't create the actions for building and
testing it. Instead report the errors for the package's dependencies
and report a setup failed error (similar to what we'd to for a load
error when producing the test packages). This produces similar errors to
what were produced by load.CheckPackageErrors while still produing the
test failure actions per package under test.
(I wasn't sure what to do about the last test case in test_setup_error.
I think it should be treated the same as other load errors?)
Fixes#70820
Change-Id: Ie95e3c158c50ed35a1f27237ef3db40502719993
Reviewed-on: https://go-review.googlesource.com/c/go/+/635856
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Matloob <matloob@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
The current b.Loop example doesn't focus on the basic usage of b.Loop.
Replace this with a new example that uses (slightly) more realistic
things to demonstrate the most salient points of b.Loop.
We also move the example into an example file so that we can write a
real Benchmark function and a real function to be benchmarks, which
makes this much closer to what a user would actually write.
Updates #61515.
Change-Id: I4d830b3bfe3eb3cd8cdecef469fea0541baebb43
Reviewed-on: https://go-review.googlesource.com/c/go/+/635896
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This updates the testing documentation to frame B.Loop as the
canonical way to write benchmarks. We retain documentation on b.N
benchmarks because people will definitely continue to see them (and
write them), but it's demoted to clearly second class.
This also attempts to clarify and refine the B.Loop documentation
itself.
Updates #61515Fixes#70787
Change-Id: If5123435bfe3a5883a753119ecdf7bbc41afd499
Reviewed-on: https://go-review.googlesource.com/c/go/+/635895
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Caleb Spare <cespare@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Austin Clements <austin@google.com>
go test -json has two new effects in Go 1.24: it implies go build
-json, and it adds a FailedBuild field in test events. For
compatibility, CL 629335 added gotestjsonbuildtext=1, which disables
the implicit go build -json, but that CL didn't affect the FailedBuild
field. In principle this shouldn't matter because it's just another
JSON field, but just so we don't have to worry about some intermediate
behavior, this CL makes gotestjsonbuildtext=1 disable the FailedBuild
field as well.
Updates #62067
Updates #70402
Change-Id: I006d1ea0468980ee2564495324e8b4ed082898af
Reviewed-on: https://go-review.googlesource.com/c/go/+/635899
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
This is part of a series of CLs that aim to reduce how often interface
arguments escape for the print functions in fmt.
Currently, method values are one of two reasons reflect.Value.Interface
always escapes its reflect.Value.
Our later CLs modify behavior around method values, so we add some tests
of function formatting (including method values) to help reduce the
chances of breaking behavior later.
We also add in some allocation tests focused on interface arguments for
the print functions. These currently do not show any improvements
compared to Go 1.21.
These tests were originally in a later CL in our stack (CL 528538),
but we split them out into this CL and moved them earlier in the stack.
Updates #8618
Change-Id: Iec51abc3b7f86a2711e7497fc2fb7a678b9f8f73
Reviewed-on: https://go-review.googlesource.com/c/go/+/529575
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Testing on the freebsd-386 gomote seems to show that sendfile returns
a non-zero number of bytes written even when it returns EINVAL.
This confuses the caller. Change the Go code to only return non-zero
on success or EINTR or EAGAIN, which are the only cases where the
man page says that sendfile updates the number of bytes.
For #70763
Change-Id: Icc04e6286b5b29a2029237711d50fe4973234f0a
Reviewed-on: https://go-review.googlesource.com/c/go/+/635815
Reviewed-by: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Currently, a symbol reference is counted as a reference to a
builtin symbol if the name matches a builtin. Usually builtin
references are generated by the compiler. But one could manually
write one with linkname. Since the list of builtin functions are
subject to change from time to time, we don't want users to depend
on their names. So we don't count a linknamed reference as a
builtin reference, and instead, count it as a named reference, so
it is checked by the linker.
Change-Id: Id3543295185c6bbd73a8cff82afb8f9cb4fd6f71
Reviewed-on: https://go-review.googlesource.com/c/go/+/635755
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This updates the new version API for the discussion on #63952.
This change reveals that in fact none of the tests set the
VERSYM_HIDDEN bit. The code before this CL set the hidden flag
for symbols that appear in DynamicVersionNeed, but that is not
an accurate representation of the ELF. The readelf program
does print undefined symbols that way (with a single '@'),
but that doesn't mean that the hidden flag is set.
Leaving tests with the hidden bit set for later.
For #63952
Change-Id: Ida60831e0c9922dfc10f10c7a64bc76a2b197537
Reviewed-on: https://go-review.googlesource.com/c/go/+/635079
Reviewed-by: Austin Clements <austin@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
This removes the difference in behavior between FIPS mode on and off.
Instead of the sentinel type we could have moved the Reader to the
drbg package and checked for equality, but then we would have locked the
crypto/rand.Reader implementation to the one in the FIPS module (which
we might have to support for years).
In internal/ed25519.GenerateKey we remove the random parameter entirely,
since that function is not actually used by crypto/ed25519.GenerateKey,
which instead commits to being deterministic.
Fixes#70772
Change-Id: Ic1c7ca2c1cd59eb9cd090a8b235c0ce218921ac5
Reviewed-on: https://go-review.googlesource.com/c/go/+/635195
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
InterfaceAddrs returns a list of the system's unicast interface addresses.
In order to do so, the function reuses the existing helpers and
list first all addresses with the netlink call RTM_GETADDR, then
all interfaces with RTM_GETLINK, and later it merge both lists
(each address references an interface).
However, the list of interfaces and addresses are obtained at
different times and there can be inconsistencies and, if an
address references an interface that is not present in the list
of interfaces, the function fails with an error.
Since the function InterfaceAddress is only about the system
addresses, there is no need to list all the interfaces, and we can
obtain the list of addresses directly from the netlink call RTM_GETADDR.
There is no need to correlate this list with the list of interfaces, as
the OS is the source of truth and should be the one providing the
consistency between addresses and interfaces.
Fixes#51934
Change-Id: I3b816e8146b1c07fdfe1bf6af338f001ef75734f
Reviewed-on: https://go-review.googlesource.com/c/go/+/635196
Reviewed-by: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Currently, cmd/internal/objfile provides dissassembly routines for
various architectures, which depend on dissassemblers from x/arch.
cmd/internal/objfile is imported in tools that need dissassembly
(objdump, pprof) and tools that don't need dissassembly (nm,
addr2line). Adding/improving disassembly support for more
architectures can cause binary size increase, and for some tools
(nm, addr2line) it is not necessary.
This CL breaks out dissassembly routines to a different package,
which is only imported in tools that need dissassembly. Other
tools can depend on cmd/internal/objfile without the disassembly
code from x/arch.
This reduces binary sizes for those tools. On darwin/arm64,
old new
cmd/addr2line 4554418 3648882 -20%
cmd/addr2line (-ldflags=-w) 3464626 2641650 -24%
cmd/nm 4503874 3616722 -20%
cmd/nm (-ldflags=-w) 3430594 2609490 -24%
For #70699.
Change-Id: Ie45d5d5c5500c5f3882e8b3c4e6eb81f0d815292
Reviewed-on: https://go-review.googlesource.com/c/go/+/634916
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
The comment being removed was added by commit ff3173849e
(which predates Gerrit and Rietveld, so no CL link), and
at the time it made sense.
Since CL 148370043 (and up to the current implementation of Clearenv)
the env map, which is populated by copyenv, is actually used, so the
comment is no longer valid.
It is also misleading, so it's best to remove it.
Change-Id: I8bd2e8bca6262759538e5bcbd396f0c71cca6a4c
Reviewed-on: https://go-review.googlesource.com/c/go/+/635078
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
When reading from time.Timer.C for an expired timer using
a fake clock (in a synctest bubble), the timer will not
be in a heap. Avoid a spurious panic claiming the timer
moved between synctest bubbles.
Drop the panic when a bubbled goroutine reads from a
non-bubbled timer channel: We allow bubbled goroutines
to access non-bubbled channels in general.
Fixes#70741
Change-Id: I27005e46f4d0067cc6846d234d22766d2e05d163
Reviewed-on: https://go-review.googlesource.com/c/go/+/634955
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
SP 800-90A Rev. 1 10.1.2.5 step 7 requires
reseed_counter = reseed_counter + 1
as the final step before returning SUCCESS.
This increment of reseedCounter was missing, meaning the reseed interval
check at the start of Generate wasn't actually functional.
Given how it's used, and that it has a reseed interval of 2^48, this
condition will never actually occur but the check is still required by
the standard.
For #69536
Change-Id: I314a7eee5852e6d0fa1a0a04842003553cd803e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/634775
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Currently specials try to save on space by only encoding the offset from
the base of the span in a uint16. This worked fine up until Go 1.24.
- Most specials have an offset of 0 (mem profile, finalizers, etc.)
- Cleanups do not care about the offset at all, so even if it's wrong,
it's OK.
- Weak pointers *do* care, but the unique package always makes a new
allocation, so the weak pointer handle offset it makes is always zero.
With Go 1.24 and general weak pointers now available, nothing is
stopping someone from just creating a weak pointer that is >64 KiB
offset from the start of an object, and this weak pointer must be
distinct from others.
Fix this problem by just increasing the size of a special and making the
offset a uintptr, to capture all possible offsets. Since we're in the
freeze, this is the safest thing to do. Specials aren't so common that I
expect a substantial memory increase from this change. In a future
release (or if there is a problem) we can almost certainly pack the
special's kind and offset together. There was already a bunch of wasted
space due to padding, so this would bring us back to the same memory
footprint before this change.
Also, add tests for equality of basic weak interior pointers. This
works, but we really should've had tests for it.
Fixes#70739.
Change-Id: Ib49a7f8f0f1ec3db4571a7afb0f4d94c8a93aa40
Reviewed-on: https://go-review.googlesource.com/c/go/+/634598
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Commit-Queue: Michael Knyszek <mknyszek@google.com>
In hindsight, I think the "advice" I wrote is a bit heavy-handed and
better suited for something like the GC guide. Listing the use-cases
seems good, and all the possible things that go wrong seems to do the
trick in terms of deterrence, like it does with finalizers.
Also, include some points I missed, like the tiny allocator warning and
the fact that weak pointers are not guaranteed to ever return nil.
Also, a lot of this actually shouldn't have been in the package docs.
Many of the warnings only apply to weak pointers, but not other data
structures that may live in this package in the future, like weak-keyed
maps.
Change-Id: Id245661540ffd93de4b727cd272284491d085c1e
Reviewed-on: https://go-review.googlesource.com/c/go/+/634376
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
The presence of a pc > entry check in CallersFrame implies we might
actually see pc == entry, when in reality Callers will never return such
a PC. This check is actually just a safety check for avoid reporting
completely nonsensical from bad input.
all.bash reports two violations to this invariant:
TestCallersFromWrapper, which explicitly constructs a CallersFrame input
with an entry PC.
runtime/pprof.printStackRecord, which passes pprof stacks to
CallersFrame (technically not a valid use of CallersFrames!).
runtime/pprof.(*Profile).Add can add the entry PC of
runtime/pprof.lostProfileEvent to samples.
(CPU profiles do lostProfileEvent + 1. I will send a second CL to fix
Add.)
Change-Id: Iac2a2f0c15117d4a383bd84cddf0413b2d7dd3ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/634315
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Before tools there was no way to directly import a package in another
module, and so missing packages were always marked as "all" due to being
dependencies of a package in a main module.
Tools break that assumption, and so to report errors in tool packages
correctly we need to mark packages as being in "all" even if they do not
exist.
Fixes#70582
Change-Id: I3273e0ec7910894565206de77986f5c249a5658c
Reviewed-on: https://go-review.googlesource.com/c/go/+/634155
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Before this change, when go tool wass used to start a tool defined in a
go.mod tool directive, it used the environment the go command was
running in. The issue with doing that is that the go command sets
various environment variables from the computed environment when
invoking a subcommand. That is used to standardise the environment for
the various tools invoked by the go command, but it is not the
expectatation of tools invoked by the go command, especially since those
environment variables may change the behavior of the tool run. Instead
use the same environment we use in go run to start the executable: the
original environment (with minor modifications) saved before we start
explicitly setting the envornment, with GOROOT/bin added to the path so
that sub commands that run the go tool use the proper go tool binary.
Fixes#70544
Change-Id: Ifbf0040a2543113638eec7232323eb9de1d61529
Reviewed-on: https://go-review.googlesource.com/c/go/+/631836
Reviewed-by: Conrad Irwin <conrad.irwin@gmail.com>
Reviewed-by: Sean Liao <sean@liao.dev>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The Go 1.24 RC is due for next week. This is a time to once again update
all golang.org/x/... module versions that contribute packages to the
std and cmd modules in the standard library to latest master versions.
For #36905.
[git-generate]
go install golang.org/x/build/cmd/updatestd@latest
go install golang.org/x/tools/cmd/bundle@latest
updatestd -goroot=$(pwd) -branch=master
Change-Id: If4fd03a18590ff3b6e701a9698370c57c69979c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/634041
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
go tool, go run, and the executable caching logic have all used
path.Base of a package's import path to set the name of the executable
produced. But the base name for a package name that's the same as a
module name ending in a major version is just that major version, which
is not very useful. For go build and go install, we use
load.DefaultExecName as the name of the binary which will select the
second to last element of the import path as the name of the executable
produced. This change changes go tool, go run, and the executable
caching logic to all use DefaultExecName consistently to pick the name
of the executable.
Change-Id: I8e615bbc6a4f9cc4549165c31954fab181d63318
Reviewed-on: https://go-review.googlesource.com/c/go/+/634039
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Conrad Irwin <conrad.irwin@gmail.com>
When adding support for module tools, we added the ability for `go tool`
to edit the module cache. For users with `GOFLAGS=-modcacherw` this
could have led to a situation where some of the files in the mod cache
were unexpectedly not deletable.
We also allow -modfile so that people can select which module they are
working in when looking for tools.
We still do not support arbitrary build flags for tools with `go tool`.
If you want those, use `go run` or `go build`, etc. instead.
Updates #48429
Change-Id: Ic3c56bb8b6ba46114196465ca6ee2dcb08b9dcc0
Reviewed-on: https://go-review.googlesource.com/c/go/+/632575
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
I did not added a test because `benchmark_test.go` is `package testing_test`
and I don't care to change that because calling predictN is not testing the
thing I would want to test.
Ideally we would run benchmark in a VM with a highjacked clocksource that never
marches forward, or using faketime but that looks fairly involved for a quickie
fix.
Fixes#70709
Change-Id: I8b4d697aff7cba33da388cb0ae8e2c2b550b9690
Reviewed-on: https://go-review.googlesource.com/c/go/+/633419
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Jorropo <jorropo.pgm@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
The TestScript/build_trimpath_cgo test for cmd/go has been accessing a
nil pointer when it tries to look up LineEntry.File.Name on a line entry
with EndSequence set to true. The doc for EndSequence specifies that if
EndSequence is set, only it and the Address field are meaningful. Skip
the entries with EndSequence set when building the set of files.
Fixes#70669
Change-Id: I421a2a9348e727e3ac4a3d42baa4d206cfbc047b
Reviewed-on: https://go-review.googlesource.com/c/go/+/633038
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@golang.org>
markUsed was not checking that the error from os.Stat was nil before
trying to access the FileInfo entry returned by it. Instead, always
check the error and return false if it's non-nil (usually because the
file does not exist). This can happen if an index entry exists in the
cache, but the output entry it points to does not. markUsed is called at
different points for the index entry and for the output entry, so it's
possible for the index entry to be marked used, and then for another go
process to trim the cache, deleting the output entry. I'm not sure how
likely that is, or if this is what has been triggering the user observed
instances of #70600, but it's enough for a test case.
Fixes#70600
Change-Id: Ia6be14b4a56736d06488ccf93c3596fff8159f22
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/633037
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Theses tests were forgot because when CL 462298 was originally written
And & Or atomics were not available in go.
Git were smart enough to rebase over And's & Or's addition.
After most reviews and before merging it were pointed I should
make theses new intrinsics noescape.
When doing this last minute addition I forgot to add tests.
Change-Id: I457f98315c0aee91d5743058ab76f256856cb782
Reviewed-on: https://go-review.googlesource.com/c/go/+/633416
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
On FreeBSD 14.1 we fail to link against C code with internal linking.
The symptom is apparently undefined symbols, but explicitly pointing the
linker at compiler-rt for -libgcc fixes the issue. This looks a lot like
the workaround on OpenBSD, but the symptom is different.
--print-libgcc-file-name produces libclang_rt.builtins-x86_64.a which
appears to be an insufficient subset of libcompiler_rt.a.
For #61095.
Change-Id: Iff5affbc923d69c89d671a69d8f4ecaadac42177
Cq-Include-Trybots: luci.golang.try:gotip-freebsd-amd64
Reviewed-on: https://go-review.googlesource.com/c/go/+/632975
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Currently, maphash.Comparable forces its argument to escape if it
contains a pointer, as we cannot hash stack pointers, which will
change when the stack moves. However, for a string, it is actually
okay if its data pointer points to the stack, as the hash depends
on only the content, not the pointer.
Currently there is no way to write this type-dependent escape
logic in Go code. So we implement it in the compiler as an
intrinsic. The compiler can also recognize not just the string
type, but types whose pointers are all string pointers, and make
them not escape.
Fixes#70560.
Change-Id: I3bf219ad71a238d2e35f0ea33de96487bc8cc231
Reviewed-on: https://go-review.googlesource.com/c/go/+/632715
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This is optimized to be cheap in terms of extra code and complexity,
rather than performance, so we reuse the GCD we have for inverting d.
Recovers most of the performance loss since CL 630516, although
benchmarking key generation is by nature extremely noisy.
goos: darwin
goarch: arm64
pkg: crypto/rsa
cpu: Apple M2
│ 3b42687c56 │ b3d018a1e8-dirty │
│ sec/op │ sec/op vs base │
GenerateKey/2048-8 104.1m ± 7% 139.7m ± 20% +34.10% (p=0.000 n=20)
Updates #69799
For #69536
Change-Id: I00347610935db8feb0597529a301ad7ace5b2f22
Reviewed-on: https://go-review.googlesource.com/c/go/+/632479
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Turns out that recomputing them (and qInv in particular) in constant
time is expensive, so let's not throw them away when they are available.
They are much faster to check, so we now do that on precompute.
Also, thanks to the opaque crypto/internal/fips140/rsa.PrivateKey type,
we now have some assurance that the values we use are always ones we
checked.
Recovers most of the performance loss since CL 630516 in the happy path.
Also, since now we always use the CRT, if necessary by running a
throwaway Precompute, which is now cheap if PrecomputedValues is filled
out, we effectively fixed the JSON round-trip slowdown (#59695).
goos: darwin
goarch: arm64
pkg: crypto/rsa
cpu: Apple M2
│ 3b42687c56 │ f017604bc6-dirty │
│ sec/op │ sec/op vs base │
ParsePKCS8PrivateKey/2048-8 26.76µ ± 1% 65.99µ ± 1% +146.64% (p=0.002 n=6)
Fixes#59695
Updates #69799
For #69536
Change-Id: I507f8c5a32e69ab28990a3bf78959836b9b08cc9
Reviewed-on: https://go-review.googlesource.com/c/go/+/632478
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
We are severely limited by the crypto/rsa API in a few ways:
- Precompute doesn't return an error, but is the only function allowed
to modify a PrivateKey.
- Clients presumably expect the PrecomputedValues big.Ints to be
populated after Precompute.
- MarshalPKCS1PrivateKey requires the precomputed values, and doesn't
have an error return.
- PrivateKeys with only N, e, and D have worked so far, so they might
have to keep working.
To move precomputation to the FIPS module, we focus on the happy path of
a PrivateKey with two primes where Precompute is called before anything
else, which match ParsePKCS1PrivateKey and GenerateKey.
There is a significant slowdown in the Parse benchmark due to the
constant-time inversion of qInv. This will be addressed in a follow-up
CL that will use (and check) the value in the ASN.1.
Note that the prime product check now moved to checkPrivateKey is broken
(Π should start at 1 not 0) and fixed in CL 632478.
Updates #69799
For #69536
Change-Id: I95a8bc1244755c6d15d7c4eb179135a15608ddd6
Reviewed-on: https://go-review.googlesource.com/c/go/+/632476
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
This is the result of running relnote todo today and reviewing its
output. Most of the remaining items that still need to be added to
Go 1.24 release notes are now tracked in release blocking issues.
For a few where it's less clear, I opted to comment on issues.
A good number of items were proposals that affect golang.org/x repos
and don't need to be mentioned in Go 1.24 release notes; they're now
annotated as such.
For #68545.
Change-Id: I4dc7f6d2cf5ab9e68bce83d01413224f80384e2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/631684
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Fix a regression that appeared in 1.23 when it comes to the stack traces
shown in the trace viewer. In 1.22 and earlier, the viewer was always
showing end stack traces. In 1.23 and later the viewer started to
exclusively show start stack traces.
Showing only the start stack traces made it impossible to see the last
stack trace produced by a goroutine. It also made it hard to understand
why a goroutine went off-cpu, as one had to hunt down the next running
slice of the same goroutine.
Emit end stack traces in addition to start stack traces to fix the
issue.
Fixes#70570
Change-Id: Ib22ea61388c1d94cdbc99fae2d207c4dce011a59
Reviewed-on: https://go-review.googlesource.com/c/go/+/631895
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
Reviewed-by: Nick Ripley <nick.ripley@datadoghq.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TestTransportRemovesH2ConnsAfterIdle is experiencing flaky
failures due to a bug in idle connection handling.
Upon inspection, TestTransportRemovesH2ConnsAfterIdle
is slow and (I think) not currently testing the condition
that it was added to test.
Using the new synctest package, this CL:
- Adds a test for the failure causing flakes in this test.
- Rewrites the existing test to use synctest to avoid sleeps.
- Adds a new test that covers the condition the test was
intended to examine.
The new TestTransportIdleConnRacesRequest exercises the
scenario where a never-used connection is closed by the
idle-conn timer at the same time as a new request attempts
to use it. In this race, the new request should either
successfully use the old connection (superseding the
idle timer) or should use a new connection; it should not
use the closing connection and fail.
TestTransportRemovesConnsAfterIdle verifies that
a connection is reused before the idle timer expires,
and not reused after.
TestTransportRemovesConnsAfterBroken verifies
that a connection is not reused after it encounters
an error. This exercises the bug fixed in CL 196665,
which introduced TestTransportRemovesH2ConnsAfterIdle.
For #70515
Change-Id: Id23026d2903fb15ef9a831b2df71177ea177b096
Reviewed-on: https://go-review.googlesource.com/c/go/+/631795
Reviewed-by: Jonathan Amsterdam <jba@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
Previously, we'd skip installing the bundled HTTP/2 support
if Transport.TLSNextProto is non-nil.
With the addition of the Transport.Protocols field, we'll
install HTTP/2 if Protocols contains HTTP2, even if TLSNextProto
is non-nil. However, we shouldn't do so if it already contains an
"h2" entry.
Change-Id: Ib086473bb52f1b76d83b1df961d41360c605832c
Reviewed-on: https://go-review.googlesource.com/c/go/+/631395
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
The Go 1.24 code freeze has recently started. This is a time to update
all golang.org/x/... module versions that contribute packages to the
std and cmd modules in the standard library to latest master versions.
For #36905.
[git-generate]
go install golang.org/x/build/cmd/updatestd@latest
go install golang.org/x/tools/cmd/bundle@latest
updatestd -goroot=$(pwd) -branch=master
Change-Id: I1b2e3b63ccc1137256d80c882b99ed26a66cbf6b
Reviewed-on: https://go-review.googlesource.com/c/go/+/631336
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org>
This change modifies the logic which searches for existing cleanups.
The existing search logic sets the next node to the current node
in certain conditions. This would cause future searches to loop
endlessly. The existing loop could convert non-cleanup specials into
cleanups and cause data corruption.
This also changes where we release the m while we are adding a
cleanup. We are currently holding onto an p-specific gcwork after
releasing the m.
Change-Id: I0ac0b304f40910549c8df114e523c89d9f0d7a75
Reviewed-on: https://go-review.googlesource.com/c/go/+/630278
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
1. In cmd/internal/obj, only apply the exclusion list to data symbols.
Text symbols are always fine since they can use PC-relative relocations.
2. In cmd/link, only skip trampolines for text symbols in the same package
with the same type. Before, all text symbols had type STEXT, but now that
there are different sections of STEXT, we can only rely on symbols in the
same package in the same section being close enough not to need
trampolines.
Fixes#70379.
Change-Id: Ifad2bdd6001ad3b5b23e641127554e9ec374715e
Reviewed-on: https://go-review.googlesource.com/c/go/+/631036
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This makes three related changes that work particularly well together
and would require significant extra work to do separately: it replaces
X25519Kyber768Draft00 with X25519MLKEM768, it makes CurvePreferences
ordering crypto/tls-selected, and applies a preference to PQ key
exchange methods over key shares (to mitigate downgrades).
TestHandshakeServerUnsupportedKeyShare was removed because we are not
rejecting unsupported key shares anymore (nor do we select them, and
rejecting them actively is a MAY). It would have been nice to keep the
test to check we still continue successfully, but testClientHelloFailure
is broken in the face of any server-side behavior which requires writing
any other messages back to the client, or reading them.
Updates #69985Fixes#69393
Change-Id: I58de76f5b8742a9bd4543fd7907c48e038507b19
Reviewed-on: https://go-review.googlesource.com/c/go/+/630775
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This is quite a bit slower (almost entirely in the e * d reductions,
which could be optimized), but the slowdown is only 12% of a signature
operation.
Also, call Validate at the end of GenerateKey as a backstop. Key
generation is so incredibly slow that the extra time is negligible.
goos: darwin
goarch: arm64
pkg: crypto/rsa
cpu: Apple M2
│ ec9643bbed │ ec9643bbed-dirty │
│ sec/op │ sec/op vs base │
SignPSS/2048-8 869.8µ ± 1% 870.2µ ± 0% ~ (p=0.937 n=6)
GenerateKey/2048-8 104.2m ± 17% 106.9m ± 10% ~ (p=0.589 n=6)
ParsePKCS8PrivateKey/2048-8 28.54µ ± 2% 136.78µ ± 8% +379.23% (p=0.002 n=6)
Fixes#57751
Co-authored-by: Derek Parker <parkerderek86@gmail.com>
Change-Id: Ifb476859207925a018b433c16dd62fb767afd2d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/630517
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Different Plan 9 file servers may return different error strings
on an attempt to open a directory for writing: EISDIR, EACCES or
EPERM. TestOpenError allows for the first two, but it needs to
allow for EPERM as well.
Fixes#70440
Change-Id: I705cc086e21630ca254499ca922ede78c9901b11
Reviewed-on: https://go-review.googlesource.com/c/go/+/629635
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org>
Unify how go/types, types2, and noder read in unified export data from
GC-created files.
This splits FindExportData into smaller pieces for improved code
sharing.
- FindPackageDefinition finds the package definition file in the ar
archive.
- ReadObjectHeaders reads the object headers.
- ReadExportDataHeader reads the export data format header.
There is a new convenience wrapper ReadUnified that combines all of
these. This documents the expected archive contents.
Updates noder and the importers to use these.
This also adjusts when end-of-section marker ("\n$$\n") checking happens.
Change-Id: Iec2179b0a1ae7f69eb12d077018f731116a77f13
Reviewed-on: https://go-review.googlesource.com/c/go/+/628155
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Tim King <taking@google.com>
For tests that are interested in testing the difference between TLS in
FIPS 140-3 required mode or otherwise two new helpers are introduced,
runWithFIPSEnabled and runWithFIPSDisabled. They take care of forcing
the correct TLS FIPS 140-3 state regardless of the overal GODEBUG=fips
state, and restoring it afterwards.
For the tests that use features or test data not appropriate for
TLS in FIPS 140-3 required mode we add skips. For some tests we can make
them appropriate for both TLS FIPS 140-3 required or not by tweaking some
parameters that weren't important to the subject under test, but would
otherwise preclude TLS FIPS 140-3 required mode (e.g. because they used
TLS 1.0 when the test could use TLS 1.2 instead). For others, switching
test certificates to a RSA 2048 hierarchy is sufficient. We avoid
regenerating the existing RSA 1024 certs as 2048 since it would
invalidate recorded static flow data.
Tests that rely on static message flows (primarily the client and server
handshake) tests are skipped due to FIPS mode being non-deterministic
and inappropriate for this style of testing.
Change-Id: I311f3828dac890bb3ff8ebda6ed73d50f0797110
Reviewed-on: https://go-review.googlesource.com/c/go/+/629736
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Per IG 10 3.A a module implementing PBKDF2 must perform a CAST
on the derivation of a master key. This commit adds the required CAST
test.
The salt length (16 bytes), and output length (14 bytes) for the test
are selected to meet FIPS requirements. The iteration count must be
at least 2 so we use that value exactly for the fastest self-test
allowable.
We test all underlying prerequisite algorithms (HMAC, digest algorithms)
separately.
For #69536
Change-Id: Iba9e87ab89eeec1c73adc7e56016674ac8065c39
Reviewed-on: https://go-review.googlesource.com/c/go/+/623195
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Add an UnencryptedHTTP2 protocol value.
Both Server and Transport implement "HTTP/2 with prior knowledge"
as described in RFC 9113, section 3.3. Neither supports the
deprecated HTTP/2 upgrade mechanism (RFC 7540, section 3.2 "h2c").
For Server, UnencryptedHTTP2 controls whether the server
will accept HTTP/2 connections on unencrypted ports.
When enabled, the server checks new connections for
the HTTP/2 preface and routes them appropriately.
For Transport, enabling UnencryptedHTTP2 and disabling HTTP1
causes http:// requests to be made over unencrypted HTTP/2
connections.
For #67816
Change-Id: I2763c4cdec1c2bc6bb8157edb93b94377de8a59b
Reviewed-on: https://go-review.googlesource.com/c/go/+/622976
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Setting GODEBUG=multipathtcp= [1] has no effects on apps using
ListenTCP or DialTCP directly.
According to the documentation, these functions are supposed to act like
Listen and Dial respectively:
ListenTCP acts like Listen for TCP networks.
DialTCP acts like Dial for TCP networks.
So when reading this, I think we should expect GODEBUG=multipathtcp= to
act on these functions as well.
Also, since #69016, MPTCP is used by default (if supported) with TCP
listeners. Similarly, when ListenTCP is used directly, MPTCP is
unexpectedly not used. It is strange to have a different behaviour.
So now, ListenTCP and DialTCP also check for MPTCP. Those are the exact
same checks that are done in dial.go, see Listen and dialSingle.
[1] https://pkg.go.dev/net#Dialer.SetMultipathTCPFixes#70500
Change-Id: I646431a74571668e505493fa8c1b2206bf30ed09
GitHub-Last-Rev: 69a31a1b03
GitHub-Pull-Request: golang/go#70501
Reviewed-on: https://go-review.googlesource.com/c/go/+/630715
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Adds support for server-side ECH.
We make a couple of implementation decisions that are not completely
in-line with the spec. In particular, we don't enforce that the SNI
matches the ECHConfig public_name, and we implement a hybrid
shared/backend mode (rather than shared or split mode, as described in
Section 7). Both of these match the behavior of BoringSSL.
The hybrid server mode will either act as a shared mode server, where-in
the server accepts "outer" client hellos and unwraps them before
processing the "inner" hello, or accepts bare "inner" hellos initially.
This lets the server operate either transparently as a shared mode
server, or a backend server, in Section 7 terminology. This seems like
the best implementation choice for a TLS library.
Fixes#68500
Change-Id: Ife69db7c1886610742e95e76b0ca92587e6d7ed4
Reviewed-on: https://go-review.googlesource.com/c/go/+/623576
Reviewed-by: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Auto-Submit: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Using filepath.SkipDir without confirming that d is a directory makes
it prone to taking unintended action if a file (not a directory) with
the same name gets added.
This isn't a problem today, but we shouldn't spend human code review
time checking that this doesn't somehow happen in the future, either.
Change-Id: I29bf203ddef175c3ad23c9ddc10fa934126ac853
Reviewed-on: https://go-review.googlesource.com/c/go/+/630635
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org>
Currently, instruction encoding is a slice of encoding types, which
is indexed by a masked version of the riscv64 opcode. Additional
information about some instructions (for example, if an instruction
has a ternary form and if there is an immediate form for an instruction)
is manually specified in other parts of the assembler code.
Rework the instruction encoding information so that we use a table
driven form, providing additional data for each instruction where
relevant. This means that we can simplify other parts of the code
by simply looking up the instruction data and reusing minimal logic.
Change-Id: I7b3b6c61a4868647edf28bd7dbae2150e043ae00
Cq-Include-Trybots: luci.golang.try:gotip-linux-riscv64
Reviewed-on: https://go-review.googlesource.com/c/go/+/622535
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
Reviewed-by: Mark Ryan <markdryan@rivosinc.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Adjust splitPathInRoot to match its documented behavior
of dropping . path components except at the end of the path.
This function takes a prefix, path, and suffix; previously
it would preserve a trailing . at the end of the path
even when joining to a suffix.
The practical effect of this change is that we we'll skip
a pointless open of . when following a symlink under some
circumstances:
- open "a/target"
- "a" is a symlink to "b/."
- previously: we rewrite our path to "b/./target"
- now: we rewrite our path to "b/target"
This is a fairly unimportant edge case, and our observable
behavior isn't changing. The main motivation for this change is
that the overall behavior is more comprehensible if splitPathInRoot
follows its documentation.
Change-Id: I96c6a5e3f489cdac991ba1bd702180d69625bc64
Reviewed-on: https://go-review.googlesource.com/c/go/+/630615
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
After tools CL 612038, the package astutil stops being vendored, but
_gen/rulegen.go needs to import this package.
In particular, after update golang.org/x/tools, the package astutil
is deleted from the vendor directory, and got error when run TestStdlib
in longtest. So in this CL, we make _gen an actual submodule and
skip it in TestStdlib.
Change-Id: I76f77b66427f6490b4746698711a6e307ad2ba79
Reviewed-on: https://go-review.googlesource.com/c/go/+/629015
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: abner chenc <chenguoqi@loongson.cn>
This commit imports the x/crypto/pbkdf2 package as described in the
linked proposal. The code is unchanged with the exception of a few
small updates to reflect feedback from the proposal comment period:
* the Key function is made generic over a hash.Hash
* the h function is moved to be the first argument
* keyLen is renamed to keyLength
* an error return is added
* the unit tests were moved to the pbkdf2_test package
Updates #69488
Change-Id: If72f854daeb65a5c7fbe45ebd341e63a33340624
Reviewed-on: https://go-review.googlesource.com/c/go/+/628135
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Russ Cox <rsc@golang.org>
While the cached name of an executable is set based on the base name of
the package path, the executable produced as the output of link doesn't
have ExeName set on it and is just called a.out (with a .exe suffix on
Windows). Set ExeName so that the first time the binary is run, from the
directory link is run in, it has the right name for ps.
For #48429
Change-Id: Ic049304ec6fd5b23c2f5aaaf91aa58d79fe5a7ba
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/630695
Reviewed-by: Conrad Irwin <conrad.irwin@gmail.com>
Reviewed-by: Hongxiang Jiang <hxjiang@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
1. Support for decimal arithmetic quad instructions of powerpc: DADDQ, DSUBQ, DMULQ
and DDIVQ.
2. Support for decimal compare ordered, unordered, quad instructions of powerpc:
DCMPU, DCMPO, DCMPUQ, and DCMPOQ.
Change-Id: I32a15a7f0a127b022b1f43d376e0ab0f7e9dd108
Cq-Include-Trybots: luci.golang.try:gotip-linux-ppc64_power10,gotip-linux-ppc64_power8,gotip-linux-ppc64le_power8,gotip-linux-ppc64le_power9,gotip-linux-ppc64le_power10
Reviewed-on: https://go-review.googlesource.com/c/go/+/623036
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Paul Murphy <murp@ibm.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Paul Murphy <murp@ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
The code takes care to print test results during "go test ./..."
in the package order, delaying prints until it's that package's
turn, even when tests run in parallel. For some reason, the
prints about the test not running were not included in that,
making them print out of order. Fix that, printing that result
with the usual result printer.
This is particularly noticeable during all.bash when we start
letting cmd/dist vet packages without tests.
Change-Id: If07f9fe5a6fac2b57b24d599126b451357a164e2
Reviewed-on: https://go-review.googlesource.com/c/go/+/630416
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Pass all packages to go test, even if they don't have test files,
so that go test can still run vet.
I just got burned by a vet error in a package without a test
showing up when I added an (unrelated) test.
There are not enough packages without tests to be worth
the "savings" of not letting the go command vet those packages.
For #60463.
Change-Id: Ib9258655151144dce6a51deeae73d651aa46cb2c
Reviewed-on: https://go-review.googlesource.com/c/go/+/630015
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
An earlier CL moved the actual test from crypto/internal/fips/check
to crypto/internal/fipstest (now crypto/internal/fips140test),
so this cmd/dist check has been doing nothing for a little while.
Fix it to do what it intends.
Also run the actual crypto package tests in FIPS mode in long mode.
Change-Id: Iea8113376b95ec068a459cb8f3d0e77d3e2340f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/630116
Reviewed-by: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
This CL rolls forward CL 630276, fixing the issues with the longtest builders that required the revert in CL 630317.
The change this CL makes compared to CL 630276 is adding the
shortPathErrorList function to rewrite the paths in the modfile.Errors
in a modfile.ErrorList using base.ShortPath and calling it on the error
returned from modfile.Parse.
The following is the commit message from the original change:
This CL first removes the base.ShortPathConservative function. It had
two classes of uses. The first was in opening files where the paths end
up in error messages. In all those cases, the non-shortened paths are
used to open the files, and ShortPath is only used for the error
messages. The second is in base.RelPaths. RelPaths will now call
ShortPath for each of the paths passed in instead of calling
RelConservative and then doing the same check as ShortPath to see if the
path is shorter.
To avoid the possibility of incorrect relative paths ending up in error
messages (that might have command lines suggested for users to run), and
to avoid the possibility of incorrect relative paths appearing in the
output of base.RelPaths, base.ShortPaths always does an os.SameFile
check to make sure that the relative path its providing is actually
correct. Since this makes ShortPath slower than just manipulating paths
(because we need to stat the files), we need to be continue to enforce
that ShortPath is only called for error messages (with the exception of
base.RelPaths and its callers).
This is a simpler way of solving the problem that base.ShortPaths
intended to solve.
For #68383
Change-Id: Ibcefb343535bd82999916b2282e9b512bb683433
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/630280
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
This change implements executable caching. It always caches the outputs of
link steps used by go run. To do so we need to make a few changes:
The first is that we want to cache binaries in a slightly different
location than we cache other outputs. The reason for doing so is so that
the name of the file could be the name of the program built. Instead of
placing the files in $GOCACHE/<two digit prefix>/<hash>-d, we place them
in $GOCACHE/<two digit prefix>/<hash>-d/<executable name>. This is done
by adding a new function called PutExecutable that works differently
from Put in two ways: first, it causes the binaries to written 0777
rather than 0666 so they can be executed. Second, PutExecutable also
writes its outputs to a new location in a directory with the output id
based name, with the file named based on p.Internal.ExeName or otherwise
the base name of the package (plus the .exe suffix on Windows).
The next changes are for writing and reading binaries from the cache. In
cmd/go/internal/work.updateBuildID, which updates build ids to the
content based id and then writes outputs to the cache, we first make the
change to always write the content based id into a binary. This is
because we won't be throwing the binaries away after running them. Then,
if the action is a link action, and we enabled excutable caching for the
action, we write the output to the binary cache.
When reading binaries, in the useCache function, we switch to using the
binary cache, and we also print the cached link outputs (which are
stored using the build action's action id).
Finally, we change go run to execute the built output from the cache.
The support for caching tools defined in a module that are run by go
tool will also use this functionality.
Fixes#69290
For #48429
Change-Id: Ic5f1d3b29d8e9786fd0d564460e3a5f53e951f41
Reviewed-on: https://go-review.googlesource.com/c/go/+/613095
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Use similar SIMD operations to the ones used in Abseil. We still
using 8-slot groups (even though the XMM registers could handle 16-slot
groups) to keep the implementation simpler (no changes to the memory
layout of maps).
Still, the implementations of matchH2 and matchEmpty are shorter than
the portable version using standard arithmetic operations. They also
return a packed bitset, which avoids the need to shift in bitset.first.
That said, the packed bitset is a downside in cognitive complexity, as
we have to think about two different possible representations. This
doesn't leak out of the API, but we do need to intrinsify bitset to
switch to a compatible implementation.
The compiler's intrinsics don't support intrinsifying methods, so the
implementations move to free functions.
This makes operations between 0-3% faster on my machine. e.g.,
MapGetHit/impl=runtimeMap/t=Int64/len=6-12 12.34n ± 1% 11.42n ± 1% -7.46% (p=0.000 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=12-12 15.14n ± 2% 14.88n ± 1% -1.72% (p=0.009 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=18-12 15.04n ± 6% 14.66n ± 2% -2.53% (p=0.000 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=24-12 15.80n ± 1% 15.48n ± 3% ~ (p=0.444 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=30-12 15.55n ± 4% 14.77n ± 3% -5.02% (p=0.004 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=64-12 15.26n ± 1% 15.05n ± 1% ~ (p=0.055 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=128-12 15.34n ± 1% 15.02n ± 2% -2.09% (p=0.000 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=256-12 15.42n ± 1% 15.15n ± 1% -1.75% (p=0.001 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=512-12 15.48n ± 1% 15.18n ± 1% -1.94% (p=0.000 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=1024-12 17.38n ± 1% 17.05n ± 1% -1.90% (p=0.000 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=2048-12 17.96n ± 0% 17.59n ± 1% -2.06% (p=0.000 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=4096-12 18.36n ± 1% 18.18n ± 1% -0.98% (p=0.013 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=8192-12 18.75n ± 0% 18.31n ± 1% -2.35% (p=0.000 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=65536-12 26.25n ± 0% 25.95n ± 1% -1.14% (p=0.000 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=262144-12 44.24n ± 1% 44.06n ± 1% ~ (p=0.181 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=1048576-12 85.02n ± 0% 85.35n ± 0% +0.39% (p=0.032 n=25)
MapGetHit/impl=runtimeMap/t=Int64/len=4194304-12 98.87n ± 1% 98.85n ± 1% ~ (p=0.799 n=25)
For #54766.
Cq-Include-Trybots: luci.golang.try:gotip-linux-ppc64_power10,gotip-linux-amd64-goamd64v3
Change-Id: Ic1b852f02744404122cb3672900fd95f4625905e
Reviewed-on: https://go-review.googlesource.com/c/go/+/626277
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Keith Randall <khr@google.com>
runtime.hmap never directly refers to the bucket type (it uses an
unsafe.Pointer), thus it shouldn't be possible to have infinite
recursion here.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-linux-amd64-longtest-race,gotip-linux-amd64-longtest-aliastypeparams
Change-Id: I457885e48bbc352acedae1c0c389078f39ed9d65
Reviewed-on: https://go-review.googlesource.com/c/go/+/619037
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
This CL first removes the base.ShortPathConservative function. It had
two classes of uses. The first was in opening files where the paths end
up in error messages. In all those cases, the non-shortened paths are
used to open the files, and ShortPath is only used for the error
messages. The second is in base.RelPaths. RelPaths will now call
ShortPath for each of the paths passed in instead of calling
RelConservative and then doing the same check as ShortPath to see if the
path is shorter.
To avoid the possibility of incorrect relative paths ending up in error
messages (that might have command lines suggested for users to run), and
to avoid the possibility of incorrect relative paths appearing in the
output of base.RelPaths, base.ShortPaths always does an os.SameFile
check to make sure that the relative path its providing is actually
correct. Since this makes ShortPath slower than just manipulating paths
(because we need to stat the files), we need to be continue to enforce
that ShortPath is only called for error messages (with the exception of
base.RelPaths and its callers).
This is a simpler way of solving the problem that base.ShortPaths
intended to solve.
For #68383
Change-Id: I474f464f51a9acb2250069dea3054b55d95a4ab4
Reviewed-on: https://go-review.googlesource.com/c/go/+/630276
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Sometimes we've used the 140 suffix (GOFIPS140, crypto/fips140)
and sometimes not (crypto/internal/fips, cmd/go/internal/fips).
Use it always, to avoid having to remember which is which.
Also, there are other FIPS standards, like AES (FIPS 197), SHA-2 (FIPS 180),
and so on, which have nothing to do with FIPS 140. Best to be clear.
For #70123.
Change-Id: I33b29dabd9e8b2703d2af25e428f88bc81c7c307
Reviewed-on: https://go-review.googlesource.com/c/go/+/630115
Reviewed-by: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
This makes methods on aliases of cgo-generated types a new compiler error.
That is ok because cgo-behavior is not covered by the G1 compatibility
guarantee.
Background: In 2023 we fixed a gopls issue related to this by actually
enabling methods on cgo-generated types in the first place (#59944).
See the discussion in #60725 and this CL for why we believe it is ok
to make this an error now.
Based on a variation of CL 503596 (by Xie Cui).
Fixes#60725.
For #59944.
Change-Id: I7e9e6e1a76447167483a282b268f5183214027c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/629715
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
If the receiver is an alias declaring type parameters, report
an error and ensure that the receiver type remains invalid.
Collect type parameters etc. as before but do not attempt to
find their constraints or instantiate the receiver type.
The constraints of the type parameters will be invalid by
default. The receiver type will not be (lazily) instantiated
which causes problems with existing invariants.
If a receiver denotes an instantiated (alias or defined) type,
report an error and ensure that the receiver type remains invalid.
While at it, add more comments and bring go/types and types2
closer together where there were differences.
Fixes#70417.
Change-Id: I87ef0b42d2f70464664cacc410f4b7eb1c994241
Reviewed-on: https://go-review.googlesource.com/c/go/+/629080
Reviewed-by: Robert Findley <rfindley@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
getOrAddWeakHandle is very careful about keeping its input alive across
the operation, but not very careful about keeping the heap-allocated
handle it creates alive. In fact, there's a window in this function
where it is *only* visible via the special. Specifically, the window of
time between when the handle is stored in the special and when the
special actually becomes visible to the GC.
(If we fail to add the special because it already exists, that case is
fine. We don't even use the same handle value, but the one we obtain
from the attached GC-visible special, *and* we return that value, so it
remains live.)
Fixes#70455.
Change-Id: Iadaff0cfb93bcaf61ba2b05be7fa0519c481de82
Reviewed-on: https://go-review.googlesource.com/c/go/+/630315
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
GenerateKey(nil) is documented to use crypto/rand.Reader, but we didn't
have a test.
While at it, since it's documented to be equivalent to NewKeyFromSeed,
actually implement it that way. This has the probably good side effect
of making it deterministic in FIPS mode. The other GenerateKey use
MaybeReadByte, so can change, but this one is probably worth keeping
deterministic. It's just slightly less compliant, but ok as long as
crypto/rand.Reader is the default one.
Intentionally leaving crypto/internal/fips/ed25519.GenerateKey in, in
case we need to switch to it during the life of the module.
Change-Id: Ic203436ff452bb9740291b9ca17f85aa6ae20b6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/630099
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
If cmd/compile is in an unhappy state, the testdir test can
fail with an unhelpful 'exit code 1' log message if
'go list' fails while gathering stdlib import config
When running individual files, such as:
go test cmd/internal/testdir -run='Test/escape.*.go'
This might also happen in other uses, or it might be
that a more expansive set of tests such as run.bash
might first trigger a more useful error.
This change prints stderr and states that it is 'go list'
that is having problems to help someone track down the
proper issue.
Change-Id: Iba658ea139bb9087ab8adb00c9f65080a1b6ee76
Reviewed-on: https://go-review.googlesource.com/c/go/+/524941
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Currently on Windows, commands like:
go test cmd/internal/testdir -run=foo -update_errors
will fail to update the errors because the parsing is
currently confused by the ':' in filepaths that
start with 'C:\', and wrongly thinks that ':' marks
the end of the Go filename.
Instead of finding the first ':', use a regexp
to find what looks to be the end of the Go filename.
Change-Id: I091106da55b8e9e9cf421814abf26a6f8b821af9
Reviewed-on: https://go-review.googlesource.com/c/go/+/524942
Reviewed-by: Russ Cox <rsc@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
GOFIPS140 does two things: (1) control whether to build binaries that
run in FIPS-140 mode by default, and (2) control which version of the
crypto/internal/fips source tree to use during a build.
This CL implements part (2). The older snapshot source trees are
stored in GOROOT/lib/fips140 in module-formatted zip files,
even though crypto/internal/fips is not technically a module.
(Reusing the module packing and unpacking code avoids reinventing it.)
See cmd/go/internal/fips/fips.go for an overview.
The documentation for GOFIPS140 is in a follow-up CL.
For #70200.
Change-Id: I73a610fd2c9ff66d0cced37d51acd8053497238e
Reviewed-on: https://go-review.googlesource.com/c/go/+/629201
Reviewed-by: Michael Matloob <matloob@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
In Loongson's new microstructure LA664 (Loongson-3A6000) and later, the atomic
instruction AMSWAP[DB]{B,H} [1] is supported. Therefore, the implementation of
the atomic operation exchange can be selected according to the CPUCFG flag LAM_BH:
AMSWAPDBB(full barrier) instruction is used on new microstructures, and traditional
LL-SC is used on LA464 (Loongson-3A5000) and older microstructures. This can
significantly improve the performance of Go programs on new microstructures.
Because Xchg8 implemented using traditional LL-SC uses too many temporary
registers, it is not suitable for intrinsics.
goos: linux
goarch: loong64
pkg: internal/runtime/atomic
cpu: Loongson-3A6000 @ 2500.00MHz
BenchmarkXchg8 100000000 10.41 ns/op
BenchmarkXchg8-2 100000000 10.41 ns/op
BenchmarkXchg8-4 100000000 10.41 ns/op
BenchmarkXchg8Parallel 96647592 12.41 ns/op
BenchmarkXchg8Parallel-2 58376136 20.60 ns/op
BenchmarkXchg8Parallel-4 78458899 17.97 ns/op
goos: linux
goarch: loong64
pkg: internal/runtime/atomic
cpu: Loongson-3A5000-HV @ 2500.00MHz
BenchmarkXchg8 38323825 31.23 ns/op
BenchmarkXchg8-2 38368219 31.23 ns/op
BenchmarkXchg8-4 37154156 31.26 ns/op
BenchmarkXchg8Parallel 37908301 31.63 ns/op
BenchmarkXchg8Parallel-2 30413440 39.42 ns/op
BenchmarkXchg8Parallel-4 30737626 39.03 ns/op
For #69735
[1]: https://loongson.github.io/LoongArch-Documentation/LoongArch-ELF-ABI-EN.html
Change-Id: I02ba68f66a2210b6902344fdc9975eb62de728ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/623058
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: sophie zhao <zhaoxiaolin@loongson.cn>
Reviewed-by: Meidan Li <limeidan@loongson.cn>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Left most of the tests in for now as they are almost all internal and
hard to externalize.
String initialization in the FIPS module has some issues, so switched
field.TestSqrtRatio to storing decoded byte slices instead.
For #69536
Change-Id: If9e4a2bb780a37a8d102a22ffd13c5293d11a8a6
Reviewed-on: https://go-review.googlesource.com/c/go/+/628776
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Bypass: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Since ECDSA private keys are irredeemably malleable, an application
could construct one where the public key doesn't match the private key.
They'd be very much on their own, but crashing the program feels a bit
harsh.
Add this one to the list of issues caused by exposing the ECDSA (and
RSA) key values as big.Ints.
For #69536
Change-Id: Iaa65c73d7145e74f860ca097fa9641448442fbf9
Reviewed-on: https://go-review.googlesource.com/c/go/+/628855
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This has the important advantage of using the system GOMODCACHE when it
exists, avoiding the download on every "go test".
While at it, also consistently use testenv.Command.
Change-Id: Ic999ffa281f6da73fe601b0feba29e60982cce3d
Reviewed-on: https://go-review.googlesource.com/c/go/+/628755
Reviewed-by: Russ Cox <rsc@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
TryBot-Bypass: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
For the future, some test vectors we should generate and then share
through Wycheproof or CCTV:
- A private key with a leading zero byte.
- A hash longer than the modulus.
- A hash longer than the P-521 modulus by a few bits.
- Reductions happening in hashToNat and bits2octets.
Fixes#64802
Change-Id: Ia0f89781b2c78eedd5103cf0e9720630711c37ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/628681
TryBot-Bypass: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
This intentionally gives up on the property of not computing the public
key until requested. It was nice, but it was making the code too
complex. The average use case is to call PublicKey immediately after
GenerateKey anyway.
Added support in the module for P-224, just in case we'd ever want to
support it in crypto/ecdh.
Tried various ways to fix test/fixedbugs/issue52193.go to be meaningful,
but crypto/ecdh is pretty complex and all the solutions would end up
locking in crypto/ecdh structure rather than compiler behavior. The rest
of that test is good enough on its own anyway. If we do the work in the
future of making crypto/ecdh zero-allocations using the affordances of
the compiler, we can add a more robust TestAllocations on our side.
For #69536
Change-Id: I68ac3955180cb31f6f96a0ef57604aaed88ab311
Reviewed-on: https://go-review.googlesource.com/c/go/+/628315
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Russ Cox <rsc@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The affine addition formula is significantly faster, and sets us up to
reuse the precomputed table from the assembly implementation.
This is an incremental step towards converging the purego and assembly
implementations, with the goal of eventually merging them.
Very proud of how the conditional AddAffine avoids the whole zero/sel
cmov dance, compared to the same logic in the assembly implementation.
Change-Id: Iab008e81869cf8c1565b938e4dd392dd4d5787fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/627938
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
As explained in fips_test.go, we generally want to minimize tests inside
the FIPS module. When there is a relevant calling package, the tests
should go there, otherwise in fipstest.
This required redoing a bit the CAST failure tests, but the new version
is actually more robust because it will fail if a _ import is missing.
Since TestCAST doesn't print a line for each passed CAST anymore, made
GODEBUG=fips140=debug do that, in case we need to show it to the lab.
For #69536
Change-Id: I0c1b82a4e9ee39e8df9bbe95bebb0527753f51c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/627955
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
GOFIPS140 does two things: (1) control whether to build binaries that
run in FIPS-140 mode by default, and (2) control which version of the
crypto/internal/fips source tree to use during a build.
This CL implements part (1). It recognizes the GOFIPS140 settings
"off" and "latest" and uses them to set the default GODEBUG=fips140
setting to "off" or "on" accordingly.
The documentation for GOFIPS140 is in a follow-up CL.
See cmd/go/internal/fips/fips.go for an overview.
For #70200.
Change-Id: I045f8ae0f19778a1e72a5cd2b6a7b0c88934fc30
Reviewed-on: https://go-review.googlesource.com/c/go/+/629198
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Hashing the key means we have to take the address of it. That inhibits
subsequent optimizations on the key variable.
By hashing a copy, we incur an extra store at the hash callsite, but
we no longer need a load of the key in the inner loop. It can live
in a register throughout. (Technically, it gets spilled around
the call to the hasher, but it gets restored outside the loop.)
Maybe one day we can have special hash functions that take
int64/int32/string instead of *int64/*int32/*string.
Change-Id: Iba3133f6e82328f53c0abcb5eec13ee47c4969d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/629419
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Instead of open-coding the waitid syscall wrapper add it to
internal/syscall/unix. As the syscall is currently only used on Linux,
switch the implementation in os.(*Process).blockUntilWaitable to use the
128-byte unix.SiginfoChild type instead of a plain 128-byte buffer.
Also use ignoringEINTR for the waitid calls instead of open-coding it.
Change-Id: I8dc47e361faa1f5e912d5de021f119c91c9f12f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/629655
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
Add an internal (for now) implementation of testing/synctest.
The synctest.Run function executes a tree of goroutines in an
isolated environment using a fake clock. The synctest.Wait function
allows a test to wait for all other goroutines within the test
to reach a blocking point.
For #67434
For #69687
Change-Id: Icb39e54c54cece96517e58ef9cfb18bf68506cfc
Reviewed-on: https://go-review.googlesource.com/c/go/+/591997
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
During the development of error wrapping (#29934),
the tests were modified to stop using reflect.DeepEqual
since the prototype for error wrapping at the time included
frame information of where the error was created.
However, that change diminished the fidelity of the test
so that it is no longer as strict, which affects the endeavor
to implement v1 in terms of the v2 prototype.
For locally declared error types, use reflect.DeepEqual
to check that the exact structure of the error value matches.
Change-Id: I443d418533866ab8d533bca3785fdc741e2c140e
Reviewed-on: https://go-review.googlesource.com/c/go/+/629517
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
GOFIPS140 will be used to control whether to build binaries that
run in FIPS-140 mode by default, as well as which version of
crypto/internal/fips is used during a given build.
It is a target configuration variable analogous to
GOOS, GOARCH, CGO_ENABLED, and the like, so the
default value is recorded in the toolchain during make.bash.
This CL adds the GOFIPS140 setting to the build process
and records the default for use by cmd/go.
For #70200.
Change-Id: Iafcb5a4207f00fae8bcd93e0184a63c72526abea
Reviewed-on: https://go-review.googlesource.com/c/go/+/629196
Reviewed-by: Michael Matloob <matloob@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
fsys.Bind(repl, dir) makes the virtual file system
redirect any references to dir to use repl instead.
In Plan 9 terms, it binds repl onto dir.
In Linux terms, it does a mount --bind of repl onto dir.
Or think of it as being like a symlink dir -> repl being
added to the virtual file system.
This is a separate layer from the overlay so that editors
working in the replacement directory can still apply
their own replacements within that tree, and also so
that editors working in the original dir do not have any
effect at all.
(If the binds and the overlay were in the same sorted list,
we'd have problems with keeping the relative priorities
of individual entries correct.)
Change-Id: Ibc88021cc95a3b8574efd5f37772ccb723aa8f7b
Reviewed-on: https://go-review.googlesource.com/c/go/+/628702
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Code like x := [12]byte{1,2,3,4,5,6,7,8,9,10,11,12} stores x in
a pair of registers and uses MOVD/MOVWU to load the values
from RODATA. The code generator needs to understand not
to use the aligned PC-relative relocation for that sequence.
In non-FIPS modes, more statictemp optimizations can be applied
and this problematic sequence doesn't happen.
Fix the decision about whether to assume alignment to match
the code used by the linker when deciding what to align.
Fixes the linker failure in CL 626437 patch set 5.
Change-Id: Iedad862c6faee758d4a2c5120cab2d329265b134
Reviewed-on: https://go-review.googlesource.com/c/go/+/628835
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Bypass: Russ Cox <rsc@golang.org>
Add a new function, WithDataIndependentTiming, which takes a function as
an argument, and encloses it with calls to set/unset the DIT PSTATE bit
on Arm64.
Since DIT is OS thread-local, for the duration of the execution of
WithDataIndependentTiming, we lock the goroutine to the OS thread, using
LockOSThread. For long running operations, this is likely to not be
performant, but we expect this to be tightly scoped around cryptographic
operations that have bounded execution times.
If locking to the OS thread turns out to be too slow, another option is
to add a bit to the g state indicating if a goroutine has DIT enabled,
and then have the scheduler enable/disable DIT when scheduling a g.
Additionally, we add a new GODEBUG, dataindependenttiming, which allows
setting DIT for an entire program. Running a program with
dataindependenttiming=1 enables DIT for the program during
initialization. In an ideal world PSTATE.DIT would be inherited from
the parent thread, so we'd only need to set it in the main thread and
then all subsequent threads would inherit the value. While this does
happen in the Linux kernel [0], it is not the case for darwin [1].
Rather than add complex logic to only set it on darwin for each new
thread, we just unconditionally set it in mstart1 and cgocallbackg1
regardless of the OS. DIT will already impose some overhead, and the
cost of setting the bit is only ~two instructions (CALL, MSR), so it
should be cheap enough.
Fixes#66450
Updates #49702
[0] e8bdb3c8be/arch/arm64/kernel/process.c (L373)
[1] 8d741a5de7/osfmk/arm64/status.c (L1666)
Change-Id: I78eda691ff9254b0415f2b54770e5850a0179749
Reviewed-on: https://go-review.googlesource.com/c/go/+/598336
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Replace the tree of nodes with a sorted list of file replacements.
The most important property of this representation is that it
allows replacing directories: a replacement x -> y where y is
a directory could not be implemented before, because it would
require making a node for every file in the tree rooted at y,
or else it would require unsuccessful lookups for files like
x/a/b/c/d/e/f/g/h/i/j/k to try every possible parent in order
to discover the x -> y mapping.
The sorted list makes it easy to find the x -> y mapping:
when you do the binary search for x/a/b/c/d/e/f/g/h/i/j/k,
you end up immediately after the x -> y mapping, so stepping
backward one entry provides the mapping we need, if it exists.
This CL does not allow overlay files to include directories,
but now it is possible. This is at least useful for other kinds
of experiments (like FIPS).
Change-Id: Ief0afaee82e644dab8ae4eafeec20440afee2e36
Reviewed-on: https://go-review.googlesource.com/c/go/+/628701
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
These are most likely redundant, but cmd/compile/internal/amd64's
TestGoAMD64v1 turns them off when clobbering those instructions, so we
need to know to skip the assembly in those cases.
Thankfully we have Avo now that adds a helpful comment with the list of
features used by each generated function!
Also improve the error output of TestGoAMD64v1. It had broken before in
#49402 and had required the exact same patch.
Change-Id: I7fab8f36042cdff630f806723aa1d8124c294f60
Reviewed-on: https://go-review.googlesource.com/c/go/+/626876
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Refactor vfs lookup into 'func stat', which knows the internal
data structures for the vfs and returns information about a
given path. The callers can then all use stat and avoid direct
knowledge of the internal data structures.
This is setting up for a different internal data structure.
Change-Id: I496b7b3fb686cdde81b14687f65eb0bf51ec62be
Reviewed-on: https://go-review.googlesource.com/c/go/+/628699
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Many releases ago we migrated
from ioutil.ReadDir, which returned []os.FileInfo,
to os.ReadDir, which returns []fs.DirEntry.
The latter is faster, but the former is expected by go/build.Context.
Convert fsys to use the new ReadDir signature.
This should make the go command faster when scanning
source trees, and it brings cmd/go up to date with the rest
of the tree.
Similarly, convert Walk to WalkDir.
Change-Id: I767a8548d7ca7cc3c05f2ff073d18070a4e8a0da
Reviewed-on: https://go-review.googlesource.com/c/go/+/628698
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Bypass: Russ Cox <rsc@golang.org>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Rename canonicalize to abs.
Rename IsDirWithGoFiles to IsGoDir.
Remove Init argument.
Split OverlayPath into Actual and Renamed.
Clean up doc comments.
Other minor cleanups.
Preparation for larger changes.
Change-Id: Ida022588149a1618a63acc91e3800b09df873b6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/628697
TryBot-Bypass: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
The Glob and Walk code does not depend on any of the fsys internals;
it simply uses ReadDir as an opaque abstraction.
Move it to separate files so that when working on the
actual overlay abstraction, it is out of sight, out of mind.
Change-Id: Ifa98feaaaafe5c1d8d8edce82de4fd0c78f599c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/628696
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
TryBot-Bypass: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
This re-enables the behavior of CL 536399 (by effectively reverting CL
628955), so now go test -json again includes build output and failures
as JSON rather than text.
However, since this behavior is clearly enough to trip up some build
systems, this CL includes a GODEBUG=gotestjsonbuildtext that can be
set to 1 to revert to the old behavior.
Fixes#70402.
Updates #62067.
Cq-Include-Trybots: luci.golang.try:gotip-darwin-arm64_13,gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Change-Id: I84e778cd844783dacfc83433e391b5ccb5925127
Reviewed-on: https://go-review.googlesource.com/c/go/+/629335
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Austin Clements <austin@google.com>
This is thought to be the cause of certain recent longtest failures.
Let's try it out.
This appears to fix the longtests fuzz failures. I suspect that the
sync.Map in internal/godebug is at fault with the implementation
changing. I'm not sure yet exactly why this is a problem, maybe inlining
that didn't happen before? I don't know exactly when coverage
instrumentation happens in the compiler, but this is definitely the
problem.
For good measure, let's add internal/sync. If sync is on the list,
internal/sync should be, too.
Fixes#70429.
Fixes#70430.
Fixes#70431.
Change-Id: Ic9f49daa0956e3a50192bcc7778983682b5d12b8
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/629475
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
[ ]
[ It has been [ 0 ] days since Filippo broke a TestAllocations. ]
[ ]
Concentrate all the skips in one place, so we don't have to re-discover
always the same ones via trial and error.
This might over-skip fixable allocations, but all these targets are not
fast anyway, so they are not worth going back for.
Removed the sysrand TestAllocations because it causes an import loop
with cryptotest and it's covered by TestAllocations in crypto/rand.
Change-Id: Icd40e97f9128e037f567147f8c9dafa758a47fac
Reviewed-on: https://go-review.googlesource.com/c/go/+/626438
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
The crypto/aes <-> crypto/cipher interfaces and the hardware support
upgrades were layered over the years, and had grown unwieldily.
Before: conditionally wrap the private crypto/aes type in private types
that implement an interface that's interface-upgraded by crypto/cipher
to replace the generic implementation in crypto/cipher.
crypto/aes depended on crypto/cipher, which is backwards.
After: provide concrete exported implementations of modes in
crypto/internal/fips/aes that crypto/cipher returns if the input Block
is the crypto/internal/fips/aes concrete implementation.
crypto/aes and crypto/cipher both depend on crypto/internal/fips/aes.
Also, made everything follow go.dev/wiki/TargetSpecific by only putting
the minimal code necessary and no exported functions in build-tagged
files.
The GCM integration still uses an interface upgrade, because the generic
implementation is complex enough that it was not trivial to duplicate.
This will be fixed in a future CL to make review easier.
For #69536
Change-Id: I21c2b93a498edb31c562b1aca824e21e8457fdff
Reviewed-on: https://go-review.googlesource.com/c/go/+/624395
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
The implementation runs up to 8 AES instructions in different registers
one after another in ASM code. Because CPU has instruction pipelining
and the instructions do not depend on each other, they can run in
parallel with this layout of code. This results in significant speedup
compared to the regular implementation in which blocks are processed in
the same registers so AES instructions do not run in parallel.
GCM mode already utilizes the approach.
The ASM implementation of ctrAble has most of its code in XORKeyStreamAt
method which has an additional argument, offset. It allows to use it
in a stateless way and to jump to any location in the stream. The method
does not exist in pure Go and boringcrypto implementations.
[ Mailed as CL 413594, then edited by filippo@ to manage the counter
with bits.Add64, remove bounds checks, make the assembly interface more
explicit, and to port the amd64 to Avo. Squeezed another -6.38% out. ]
goos: linux
goarch: amd64
pkg: crypto/cipher
cpu: AMD Ryzen 7 PRO 8700GE w/ Radeon 780M Graphics
│ 19df80d792 │ c8b0409d40 │
│ sec/op │ sec/op vs base │
AESCTR/50-8 64.68n ± 0% 26.89n ± 0% -58.42% (p=0.000 n=10)
AESCTR/1K-8 1145.0n ± 0% 135.8n ± 0% -88.14% (p=0.000 n=10)
AESCTR/8K-8 9145.0n ± 0% 917.5n ± 0% -89.97% (p=0.000 n=10)
geomean 878.2n 149.6n -82.96%
│ 19df80d792 │ c8b0409d40 │
│ B/s │ B/s vs base │
AESCTR/50-8 737.2Mi ± 0% 1773.3Mi ± 0% +140.54% (p=0.000 n=10)
AESCTR/1K-8 848.5Mi ± 0% 7156.6Mi ± 0% +743.40% (p=0.000 n=10)
AESCTR/8K-8 853.8Mi ± 0% 8509.9Mi ± 0% +896.70% (p=0.000 n=10)
geomean 811.4Mi 4.651Gi +486.94%
Fixes#20967
Updates #39365
Updates #26673
Co-authored-by: Filippo Valsorda <filippo@golang.org>
Change-Id: Iaeea29fb93a56456f2e54507bc25196edb31b84b
Reviewed-on: https://go-review.googlesource.com/c/go/+/621958
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
This tweaks the inlining cost knob for closures
specifically, they receive a doubled budget. The
rationale for this is that closures have a lot of
"crud" in their IR that will disappear after inlining,
so the standard budget penalizes them unnecessarily.
This is also the cause of these bugs -- looking at the
code involved, these closures "should" be inlineable,
therefore tweak the parameters until behavior matches
expectations. It's not costly in binary size, because
the only-called-from-one-site case is common (especially
for rangefunc iterators).
I can imagine better fixes and I am going to try to
get that done, but this one is small and makes things
better.
Fixes#69411, #69539.
Change-Id: I8a892c40323173a723799e0ddad69dcc2724a8f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/629195
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The linker knows the types of the global variables. We can use those
types to build the GC programs that describe the data and bss pointer masks.
That way we don't use the GC programs of the constituent types.
This is part of an effort to remove GC programs from the runtime.
There's a major complication in that when we're linking against a
shared library (typically, libstd.so), the relocations we need to
break apart arrays and structs into constituent types are difficult to
find. Load that additional data when linking against shared libraries.
Change-Id: I8516b24a0604479895c7b8a8a358d3cd8d421530
Reviewed-on: https://go-review.googlesource.com/c/go/+/546216
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
This change adds a GOEXPERIMENT, synchashtriemap, which replaces the
internals of a sync.Map with internal/sync.HashTrieMap[any, any]. The
main purpose behind this change is improved performance. Across almost
every benchmark, HashTrieMap[any, any] performs better than Map.
Also, relax TestMapClearNoAllocations to allow for one allocation.
Currently, the HashTrieMap allocates a new empty root node and stores
it: that's the whole clear operation. At the cost of some complexity, we
could allow Clear to have zero allocations by clearing the root node.
The complexity comes down to allowing threads to race to install a new
root node *or* creating a top-level mutex for installing a root node.
But I'm not sure this is worth it. Whether Clear or some other operation
takes the hit for allocating a single node almost certainly doesn't
matter. And Clear is still much, much faster in the new implementation
than the old, so I don't consider this a regression.
Change-Id: I939aa70a0edf2e850cedbea239aaf29a11a77b79
Reviewed-on: https://go-review.googlesource.com/c/go/+/608335
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
We observe the CompareAndSwap and Swap can both be substantially faster
if the value in each entry node is mutable. This change modifies the
map entry node to store the value indirectly, allowing us to perform
swaps for existing nodes and compare-and-swaps without taking the
parent node's lock.
Change-Id: I371343aa81a843d3a7e6bc5ac87b8a96c12ca3a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/606462
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently the HashTrieMap requires both keys and values to be
comparable, but it's actually OK if the value is not comparable. Some
operations may fail, but others will not, and we can check comparability
dynamically on map initialization. This makes the implementation
substantially more flexible.
Change-Id: Idc9c30dfa273d80ae4d46a9eefb5c155294408aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/594061
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
This CL refactors sync.Mutex such that its implementation lives in the
new internal/sync package. The purpose of this change is to eventually
reverse the dependency edge between internal/concurrent and sync, such
that sync can depend on internal/concurrent (or really, its contents,
which will likely end up in internal/sync).
The only change made to the sync.Mutex code is the frame skip count for
mutex profiling, so that the internal/sync frames are omitted in the
profile.
Change-Id: Ib3603d30e8e71508c4ea883a584ae2e51ce40c3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/594056
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
FreeBSD and Dragonfly have used the sysctl method for years, while
NetBSD has read the name of the executable from /proc. Unfortunately,
some folks are hitting errors when building Go software in a sandbox
that lacks a mounted /proc filesystem.
Switch NetBSD to use the same implementation as FreeBSD and Dragonfly.
Unfortunately, the order of the arguments in the MIB is also
OS-dependent.
Change-Id: I6fd774904af417ccd127e3779af45a20dc8696ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/629035
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Bypass: Ian Lance Taylor <iant@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Following CLs will refactor Mutex and change the internals of Map. This
ends up breaking tests in x/tools for the copylock vet check, because
the error message changes. Let's insulate ourselves from such things
permanently by adding an explicit noCopy field. We'll update the vet
check to accept that as the problem, rather than depend on less explicit
internals.
We capture Once here too to clean up the error message as well.
Change-Id: Iead985fc8ec9ef3ea5ff615f26dde17bb03aeadb
Reviewed-on: https://go-review.googlesource.com/c/go/+/627777
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Tim King <taking@google.com>
This is a two-pronged approach. First, try to keep large objects
off the stack frame. Second, if they do manage to appear anyway,
use straight bitmasks instead of gc programs.
Generally probably a good idea to keep large objects out of stack frames.
But particularly keeping gc programs off the stack simplifies
runtime code a bit.
This CL sets the limit of most stack objects to 131072 bytes (on 64-bit archs).
There can still be large objects if allocated by a late pass, like order, or
they are required to be on the stack, like function arguments.
But the size for the bitmasks for these objects isn't a huge deal,
as we have already have (probably several) bitmasks for the frame
liveness map itself.
Change-Id: I6d2bed0e9aa9ac7499955562c6154f9264061359
Reviewed-on: https://go-review.googlesource.com/c/go/+/542815
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
It wasn't actually testing what it says it was testing.
A random permutation isn't cyclic. It only probably hits a few
elements before entering a cycle.
Use an algorithm that generates a random cyclic permutation instead.
Fixing the test makes the previous CL look less good. But it still helps.
(Theory: Fixing the test makes it less cache friendly, so there are
more misses all around. That makes the benchmark slower, suppressing
the differences seen. Also fixing the benchmark makes the loop
iteration count less predictable, which hurts the raw loop
implementation somewhat.)
(baseline = tip, experiment = tip+previous CL, noswiss = GOEXPERIMENT=noswissmap)
goos: darwin
goarch: arm64
pkg: runtime
cpu: Apple M2 Ultra
│ baseline │ experiment │
│ sec/op │ sec/op vs base │
MapCycle-24 20.59n ± 4% 18.99n ± 3% -7.77% (p=0.000 n=10)
khr@Mac-Studio src % benchstat noswiss experiment
goos: darwin
goarch: arm64
pkg: runtime
cpu: Apple M2 Ultra
│ noswiss │ experiment │
│ sec/op │ sec/op vs base │
MapCycle-24 16.12n ± 1% 18.99n ± 3% +17.83% (p=0.000 n=10)
Change-Id: I3a4edb814ba97fec020a6698c535ce3a87a9fc67
Reviewed-on: https://go-review.googlesource.com/c/go/+/625900
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
XORBytes doesn't say anything about how it deals with destination and
source overlaps. Current implementations as written do work if the
destination overlaps perfectly with a source, but will unavoidably
return nonsensical results if the destination is ahead of the source.
Lock in the current behavior with tests, docs, and panics.
Note that this introduces a new panic, but if any applications run into
it we are potentially catching a security issue.
Also, expand the tests and move them outside the FIPS module per #69536
convention. (We want to minimize changes within the module boundary.)
Updates #53021
Change-Id: Ibb0875fd38da3818079e31b83b1a227b53755930
Reviewed-on: https://go-review.googlesource.com/c/go/+/622276
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently, the Printer interface has `Output`, which acts like Print
and `Errorf`, which acts like Printf. It's confusing that the
formatting style is tied to whether it's regular output or an error.
Fix this by replacing Output with Printf, so both use Printf-style
formatting.
Change-Id: I4c76f941e956f2599c5620b455bf41e21636b44e
Reviewed-on: https://go-review.googlesource.com/c/go/+/627795
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently, if a test or imported package fails to build during "go
test -json", the build error text will be interleaved with the JSON
output of tests. Furthermore, there’s currently no way to reliably
associate a build error with the test package or packages it affected.
This creates unnecessary friction and complexity in tools that consume
the "go test -json" output.
This CL makes "go test -json" enable JSON reporting of build errors.
It also adds a "FailedBuild" field to the "fail" TestEvent, which
gives the package ID of the package that failed to build and caused
the test to fail.
Using this, CI systems should be able to consume the entire output
stream from "go test -json" in a structured way and easily associate
build failures with test failures during reporting.
Fixes#62067.
Updates #35169.
Updates #37486.
Change-Id: I49091dcc7aa52db01fc9fa6042771633e97b8407
Reviewed-on: https://go-review.googlesource.com/c/go/+/536399
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently, each Action tracks whether it failed, which is propagated
up from dependencies. Shortly, we'll need to know the root cause if a
test fails because of a build failure. To support this, replace the
Failed boolean with a Failed *Action that tracks the root Action that
failed and caused other Actions to fail.
For #62067.
Change-Id: I8f84a51067354043ae9531a4368c6f8b11d688d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/536398
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently, under *most* circumstances, if there's a package loading
error during "go test", that will get reported as a "FAIL p [setup
failed]" or "FAIL p [build failed] message and won't prevent running
unaffected test packages.
However, if there's a loading error from a non-test file in a package
listed directly on the "go test" command line, that gets reported as
an immediate fatal error, without any "FAIL" line, and without
attempting to run other tests listed on the command line. Likewise,
certain early build errors (like a package containing no Go files) are
currently immediately fatal rather than reporting a test failure.
Fix this by eliminating the check that causes that immediate failure.
This causes one minor follow-up problem: since
load.TestPackagesAndErrors was never passed a top-level package with
an error before, it doesn't currently propagate such an error to the
packages it synthesizes (even though it will propagate errors in
imported packages). Fix this by copying the error from the top-level
package into the synthesized test package while we're copying
everything else.
For #62067.
Change-Id: Icd563a3d9912256b53afd998050995e5260ebe5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/558637
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
This replaces the existing Shell print function callback. The
interface also gives us a way to report build failures, which is the
other type of event that will appear in the build -json output.
This CL hooks up error reporting in two places:
- In Builder.Do, where all builder errors are reported.
- In load.CheckPackageErrors, where most loading errors are reported.
For #62067.
Change-Id: Id66a31b0d2c3786559c7d2bb376fffeffc9a66ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/536396
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Switch labelMap from map[string]string to use LabelSet as a data
structure. Optimize Labels() for the case where the keys are given in
sorted order without duplicates.
This is primarily motivated by reducing the overhead of distributed
tracing systems that use pprof labels. We have encountered cases where
users complained about the overhead relative to the rest of our
distributed tracing library code. Additionally, we see this as an
opportunity to free up hundreds of CPU cores across our fleet.
A secondary motivation is eBPF profilers that try to access pprof
labels. The current map[string]string requires them to implement Go map
access in eBPF, which is non-trivial. With the enablement of swiss maps,
this complexity is only increasing. The slice data structure introduced
in this CL will greatly lower the implementation complexity for eBPF
profilers in the future. But to be clear: This change does not imply
that the pprof label mechanism is now a stable ABI. They are still an
implementation detail and may change again in the future.
goos: darwin
goarch: arm64
pkg: runtime/pprof
cpu: Apple M1 Max
│ baseline.txt │ patch1.txt │
│ sec/op │ sec/op vs base │
Labels/set-one-10 153.50n ± 3% 75.00n ± 1% -51.14% (p=0.000 n=10)
Labels/merge-one-10 187.8n ± 1% 128.8n ± 1% -31.42% (p=0.000 n=10)
Labels/overwrite-one-10 193.1n ± 2% 102.0n ± 1% -47.18% (p=0.000 n=10)
Labels/ordered/set-many-10 502.6n ± 4% 146.1n ± 2% -70.94% (p=0.000 n=10)
Labels/ordered/merge-many-10 516.3n ± 2% 238.1n ± 1% -53.89% (p=0.000 n=10)
Labels/ordered/overwrite-many-10 569.3n ± 4% 247.6n ± 2% -56.51% (p=0.000 n=10)
Labels/unordered/set-many-10 488.9n ± 2% 308.3n ± 3% -36.94% (p=0.000 n=10)
Labels/unordered/merge-many-10 523.6n ± 1% 258.5n ± 1% -50.64% (p=0.000 n=10)
Labels/unordered/overwrite-many-10 571.4n ± 1% 412.1n ± 2% -27.89% (p=0.000 n=10)
geomean 366.8n 186.9n -49.05%
│ baseline.txt │ patch1b.txt │
│ B/op │ B/op vs base │
Labels/set-one-10 424.0 ± 0% 104.0 ± 0% -75.47% (p=0.000 n=10)
Labels/merge-one-10 424.0 ± 0% 200.0 ± 0% -52.83% (p=0.000 n=10)
Labels/overwrite-one-10 424.0 ± 0% 136.0 ± 0% -67.92% (p=0.000 n=10)
Labels/ordered/set-many-10 1344.0 ± 0% 392.0 ± 0% -70.83% (p=0.000 n=10)
Labels/ordered/merge-many-10 1184.0 ± 0% 712.0 ± 0% -39.86% (p=0.000 n=10)
Labels/ordered/overwrite-many-10 1056.0 ± 0% 712.0 ± 0% -32.58% (p=0.000 n=10)
Labels/unordered/set-many-10 1344.0 ± 0% 712.0 ± 0% -47.02% (p=0.000 n=10)
Labels/unordered/merge-many-10 1184.0 ± 0% 712.0 ± 0% -39.86% (p=0.000 n=10)
Labels/unordered/overwrite-many-10 1.031Ki ± 0% 1.008Ki ± 0% -2.27% (p=0.000 n=10)
geomean 843.1 405.1 -51.95%
│ baseline.txt │ patch1b.txt │
│ allocs/op │ allocs/op vs base │
Labels/set-one-10 5.000 ± 0% 3.000 ± 0% -40.00% (p=0.000 n=10)
Labels/merge-one-10 5.000 ± 0% 5.000 ± 0% ~ (p=1.000 n=10) ¹
Labels/overwrite-one-10 5.000 ± 0% 4.000 ± 0% -20.00% (p=0.000 n=10)
Labels/ordered/set-many-10 8.000 ± 0% 3.000 ± 0% -62.50% (p=0.000 n=10)
Labels/ordered/merge-many-10 8.000 ± 0% 5.000 ± 0% -37.50% (p=0.000 n=10)
Labels/ordered/overwrite-many-10 7.000 ± 0% 4.000 ± 0% -42.86% (p=0.000 n=10)
Labels/unordered/set-many-10 8.000 ± 0% 4.000 ± 0% -50.00% (p=0.000 n=10)
Labels/unordered/merge-many-10 8.000 ± 0% 5.000 ± 0% -37.50% (p=0.000 n=10)
Labels/unordered/overwrite-many-10 7.000 ± 0% 5.000 ± 0% -28.57% (p=0.000 n=10)
geomean 6.640 4.143 -37.60%
¹ all samples are equal
Change-Id: Ie68e960a25c2d97bcfb6239dc481832fa8a39754
Reviewed-on: https://go-review.googlesource.com/c/go/+/574516
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
This change introduces AddCleanup to the runtime package. AddCleanup attaches
a cleanup function to an pointer to an object.
The Stop method on Cleanups will be implemented in a followup CL.
AddCleanup is intended to be an incremental improvement over
SetFinalizer and will result in SetFinalizer being deprecated.
For #67535
Change-Id: I99645152e3fdcee85fcf42a4f312c6917e8aecb1
Reviewed-on: https://go-review.googlesource.com/c/go/+/627695
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
The tri-state mutex implementation (unlocked, locked, sleeping) avoids
sleep/wake syscalls when contention is low or absent, but its
performance degrades when many threads are contending for a mutex to
execute a fast critical section.
A fast critical section means frequent unlock2 calls. Each of those
finds the mutex in the "sleeping" state and so wakes a sleeping thread,
even if many other threads are already awake and in the spin loop of
lock2 attempting to acquire the mutex for themselves. Many spinning
threads means wasting energy and CPU time that could be used by other
processes on the machine. Many threads all spinning on the same cache
line leads to performance collapse.
Merge the futex- and semaphore-based mutex implementations by using a
semaphore abstraction for futex platforms. Then, add a bit to the mutex
state word that communicates whether one of the waiting threads is awake
and spinning. When threads in lock2 see the new "spinning" bit, they can
sleep immediately. In unlock2, the "spinning" bit means we can save a
syscall and not wake a sleeping thread.
This brings up the real possibility of starvation: waiting threads are
able to enter a deeper sleep than before, since one of their peers can
volunteer to be the sole "spinning" thread and thus cause unlock2 to
skip the semawakeup call. Additionally, the waiting threads form a LIFO
stack so any wakeups that do occur will target threads that have gone to
sleep most recently. Counteract those effects by periodically waking the
thread at the bottom of the stack and allowing it to spin.
Exempt sched.lock from most of the new behaviors; it's often used by
several threads in sequence to do thread-specific work, so low-latency
handoff is a priority over improved throughput.
Gate use of this implementation behind GOEXPERIMENT=spinbitmutex, so
it's easy to disable. Enable it by default on supported platforms (the
most efficient implementation requires atomic.Xchg8).
Fixes#68578
goos: linux
goarch: amd64
pkg: runtime
cpu: 13th Gen Intel(R) Core(TM) i7-13700H
│ old │ new │
│ sec/op │ sec/op vs base │
MutexContention 17.82n ± 0% 17.74n ± 0% -0.42% (p=0.000 n=10)
MutexContention-2 22.17n ± 9% 19.85n ± 12% ~ (p=0.089 n=10)
MutexContention-3 26.14n ± 14% 20.81n ± 13% -20.41% (p=0.000 n=10)
MutexContention-4 29.28n ± 8% 21.19n ± 10% -27.62% (p=0.000 n=10)
MutexContention-5 31.79n ± 2% 21.98n ± 10% -30.83% (p=0.000 n=10)
MutexContention-6 34.63n ± 1% 22.58n ± 5% -34.79% (p=0.000 n=10)
MutexContention-7 44.16n ± 2% 23.14n ± 7% -47.59% (p=0.000 n=10)
MutexContention-8 53.81n ± 3% 23.66n ± 6% -56.04% (p=0.000 n=10)
MutexContention-9 65.58n ± 4% 23.91n ± 9% -63.54% (p=0.000 n=10)
MutexContention-10 77.35n ± 3% 26.06n ± 9% -66.31% (p=0.000 n=10)
MutexContention-11 89.62n ± 1% 25.56n ± 9% -71.47% (p=0.000 n=10)
MutexContention-12 102.45n ± 2% 25.57n ± 7% -75.04% (p=0.000 n=10)
MutexContention-13 111.95n ± 1% 24.59n ± 8% -78.04% (p=0.000 n=10)
MutexContention-14 123.95n ± 3% 24.42n ± 6% -80.30% (p=0.000 n=10)
MutexContention-15 120.80n ± 10% 25.54n ± 6% -78.86% (p=0.000 n=10)
MutexContention-16 128.10n ± 25% 26.95n ± 4% -78.96% (p=0.000 n=10)
MutexContention-17 139.80n ± 18% 24.96n ± 5% -82.14% (p=0.000 n=10)
MutexContention-18 141.35n ± 7% 25.05n ± 8% -82.27% (p=0.000 n=10)
MutexContention-19 151.35n ± 18% 25.72n ± 6% -83.00% (p=0.000 n=10)
MutexContention-20 153.30n ± 20% 24.75n ± 6% -83.85% (p=0.000 n=10)
MutexHandoff/Solo-20 13.54n ± 1% 13.61n ± 4% ~ (p=0.206 n=10)
MutexHandoff/FastPingPong-20 141.3n ± 209% 164.8n ± 49% ~ (p=0.436 n=10)
MutexHandoff/SlowPingPong-20 1.572µ ± 16% 1.804µ ± 19% +14.76% (p=0.015 n=10)
geomean 74.34n 30.26n -59.30%
goos: darwin
goarch: arm64
pkg: runtime
cpu: Apple M1
│ old │ new │
│ sec/op │ sec/op vs base │
MutexContention 13.86n ± 3% 12.09n ± 3% -12.73% (p=0.000 n=10)
MutexContention-2 15.88n ± 1% 16.50n ± 2% +3.94% (p=0.001 n=10)
MutexContention-3 18.45n ± 2% 16.88n ± 2% -8.54% (p=0.000 n=10)
MutexContention-4 20.01n ± 2% 18.94n ± 18% ~ (p=0.469 n=10)
MutexContention-5 22.60n ± 1% 17.51n ± 9% -22.50% (p=0.000 n=10)
MutexContention-6 23.93n ± 2% 17.35n ± 2% -27.48% (p=0.000 n=10)
MutexContention-7 24.69n ± 1% 17.15n ± 3% -30.54% (p=0.000 n=10)
MutexContention-8 25.01n ± 1% 17.33n ± 2% -30.69% (p=0.000 n=10)
MutexHandoff/Solo-8 13.96n ± 4% 12.04n ± 4% -13.78% (p=0.000 n=10)
MutexHandoff/FastPingPong-8 68.89n ± 4% 64.62n ± 2% -6.20% (p=0.000 n=10)
MutexHandoff/SlowPingPong-8 9.698µ ± 22% 9.646µ ± 35% ~ (p=0.912 n=10)
geomean 38.20n 32.53n -14.84%
Change-Id: I0058c75eadf282d08eea7fce0d426f0518039f7c
Reviewed-on: https://go-review.googlesource.com/c/go/+/620435
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com>
Implement sema{create,sleep,wakeup} in terms of the futex syscall when
available. Split the lock2/unlock2 implementations out of lock_sema.go
and lock_futex.go (which they shared with runtime.note) to allow
swapping in new implementations of those.
Let futex-based platforms use the semaphore-based mutex implementation.
Control that via the new "spinbitmutex" GOEXPERMENT value, disabled by
default.
This lays the groundwork for a "spinbit" mutex implementation; it does
not include the new mutex implementation.
For #68578.
Change-Id: I091289c85124212a87abec7079ecbd9e610b4270
Reviewed-on: https://go-review.googlesource.com/c/go/+/622996
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This is panicking on the darwin-amd64-longtest builders.
Not sure why, but it was added only to get a stack trace
during debugging. If there's still a problem, we should let
it proceed and find the real problem.
The test that was failing - internal/coverage/cfile - passes
with this CL, even when I set GODEBUG=fips140=on,
so there's hope that it will fix the longtest builders.
Change-Id: I9b3e743effdddcc0a76895922f87631527781dff
Reviewed-on: https://go-review.googlesource.com/c/go/+/628375
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
For now, FIPS does not work with ASAN: ASAN detects reads
it doesn't like during the scans of memory done by verification.
It could be made to work if there was a way to disable ASAN
during verification, but that doesn't appear to be possible.
Instead of a cryptic ASAN message, panic with a clear error.
And disable the test during ASAN.
Fixes#70321.
Change-Id: Ibc3876836abb83248a23c18c3b44c4cbb4a0c600
Reviewed-on: https://go-review.googlesource.com/c/go/+/627603
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
The test "if(! ~ $#GOROOT_BOOTSTRAP 1)", to check for the environment
variable GOROOT_BOOTSTRAP being undefined, will not succeed if the
variable is set to the empty string (as the coordinator was doing).
A better test is "if(~ $"GOROOT_BOOTSTRAP '')", which succeeds if
the variable is undefined, or set to an empty list or an empty string.
For #69038
Change-Id: Ic6e6944e0c76461daea206ba9575b863f92f6228
Reviewed-on: https://go-review.googlesource.com/c/go/+/627944
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David du Colombier <0intro@gmail.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Go has never supported Cygwin as a C compiler, but users get the
following cryptic error message when they try to use it:
implicit declaration of function '_beginthread'
This is because Cygwin doesn't implement _beginthread. Note that
this is not the only problem with Cygwin, but it's the one that
users are most likely to run into first.
This CL improves the error message to make it clear that Cygwin
is not supported, and suggests using MinGW instead.
Fixes#59490Fixes#36691
Change-Id: Ifeec7a2cb38d7c5f50d6362c95504f72818c6a76
Reviewed-on: https://go-review.googlesource.com/c/go/+/627935
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
For a wasmexport wrapper, we generate a call to the actual
exported Go function, and use the wrapper function's PC 1 as the
(fake) return address. This address is not used for returning,
which is handled by the Wasm call stack. It is used for stack
unwinding, and PC 1 makes it past the prologue and therefore has
the right SP delta. But if the function has no arguments and
results, the wrapper is frameless, with no prologue, and PC 1
doesn't exist. This causes the unwinder to fail. In this case, we
put PC 0, which also has the correct SP delta (0).
Fixes#69584.
Change-Id: Ic047a6e62100db540b5099cc5a56a1d0f16d58b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/624000
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently it's possible for weak->strong conversions to create more GC
work during mark termination. When a weak->strong conversion happens
during the mark phase, we need to mark the newly-strong pointer, since
it may now be the only pointer to that object. In other words, the
object could be white.
But queueing new white objects creates GC work, and if this happens
during mark termination, we could end up violating mark termination
invariants. In the parlance of the mark termination algorithm, the
weak->strong conversion is a non-monotonic source of GC work, unlike the
write barriers (which will eventually only see black objects).
This change fixes the problem by forcing weak->strong conversions to
block during mark termination. We can do this efficiently by setting a
global flag before the ragged barrier that is checked at each
weak->strong conversion. If the flag is set, then the conversions block.
The ragged barrier ensures that all Ps have observed the flag and that
any weak->strong conversions which completed before the ragged barrier
have their newly-minted strong pointers visible in GC work queues if
necessary. We later unset the flag and wake all the blocked goroutines
during the mark termination STW.
There are a few subtleties that we need to account for. For one, it's
possible that a goroutine which blocked in a weak->strong conversion
wakes up only to find it's mark termination time again, so we need to
recheck the global flag on wake. We should also stay non-preemptible
while performing the check, so that if the check *does* appear as true,
it cannot switch back to false while we're actively trying to block. If
it switches to false while we try to block, then we'll be stuck in the
queue until the following GC.
All-in-all, this CL is more complicated than I would have liked, but
it's the only idea so far that is clearly correct to me at a high level.
This change adds a test which is somewhat invasive as it manipulates
mark termination, but hopefully that infrastructure will be useful for
debugging, fixing, and regression testing mark termination whenever we
do fix it.
Fixes#69803.
Change-Id: Ie314e6fd357c9e2a07a9be21f217f75f7aba8c4a
Reviewed-on: https://go-review.googlesource.com/c/go/+/623615
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Execution of the loop body previously either terminated
the iteration (returned false because of a break, goto, or
return) or actually panicked. The check against abi.RF_READY
ensures that the body can no longer run and also panics.
This CL in addition transitions the loop state to abi.RF_PANIC
so that if this already badly-behaved iterator defer-recovers
this panic, then the exit check at the loop context will
catch the problem and panic there.
Previously, panics triggered by attempted execution of a
no-longer active loop would not trigger a panic at the loop
context if they were defer-recovered.
Change-Id: Ieeed2fafd0d65edb66098dc27dc9ae8c1e6bcc8c
Reviewed-on: https://go-review.googlesource.com/c/go/+/625455
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Tim King <taking@google.com>
Iteration over swissmaps with low load (think map with large hint but
only one entry) is signicantly regressed vs old maps. See noswiss vs
swiss-tip below (+60%).
Currently we visit every single slot and individually check if the slot
is full or not.
We can do much better by using the control word to find all full slots
in a group in a single operation. This lets us skip completely empty
groups for instance.
Always using the control match approach is great for maps with low load,
but is a regression for mostly full maps. Mostly full maps have the
majority of slots full, so most calls to mapiternext will return the
next slot. In that case, doing the full group match on every call is
more expensive than checking the individual slot.
Thus we take a hybrid approach: on each call, we first check an
individual slot. If that slot is full, we're done. If that slot is
non-full, then we fall back to doing full group matches.
This trade-off works well. Both mostly empty and mostly full maps
perform nearly as well as doing all matching and all individual,
respectively.
The fast path is placed above the slow path loop rather than combined
(with some sort of `useMatch` variable) into a single loop to help the
compiler's code generation. The compiler really struggles with code
generation on a combined loop for some reason, yielding ~15% additional
instructions/op.
Comparison with old maps prior to this CL:
│ noswiss │ swiss-tip │
│ sec/op │ sec/op vs base │
MapIter/Key=int64/Elem=int64/len=6-12 11.53n ± 2% 10.64n ± 2% -7.72% (p=0.002 n=6)
MapIter/Key=int64/Elem=int64/len=64-12 10.180n ± 2% 9.670n ± 5% -5.01% (p=0.004 n=6)
MapIter/Key=int64/Elem=int64/len=65536-12 10.78n ± 1% 10.15n ± 2% -5.84% (p=0.002 n=6)
MapIterLowLoad/Key=int64/Elem=int64/len=6-12 6.116n ± 2% 6.840n ± 2% +11.84% (p=0.002 n=6)
MapIterLowLoad/Key=int64/Elem=int64/len=64-12 2.403n ± 2% 3.892n ± 0% +61.95% (p=0.002 n=6)
MapIterLowLoad/Key=int64/Elem=int64/len=65536-12 1.940n ± 3% 3.237n ± 1% +66.81% (p=0.002 n=6)
MapPop/Key=int64/Elem=int64/len=6-12 66.20n ± 2% 60.14n ± 3% -9.15% (p=0.002 n=6)
MapPop/Key=int64/Elem=int64/len=64-12 97.24n ± 1% 171.35n ± 1% +76.21% (p=0.002 n=6)
MapPop/Key=int64/Elem=int64/len=65536-12 826.1n ± 12% 842.5n ± 10% ~ (p=0.937 n=6)
geomean 17.93n 20.96n +16.88%
After this CL:
│ noswiss │ swiss-cl │
│ sec/op │ sec/op vs base │
MapIter/Key=int64/Elem=int64/len=6-12 11.53n ± 2% 10.90n ± 3% -5.42% (p=0.002 n=6)
MapIter/Key=int64/Elem=int64/len=64-12 10.180n ± 2% 9.719n ± 9% -4.53% (p=0.043 n=6)
MapIter/Key=int64/Elem=int64/len=65536-12 10.78n ± 1% 10.07n ± 2% -6.63% (p=0.002 n=6)
MapIterLowLoad/Key=int64/Elem=int64/len=6-12 6.116n ± 2% 7.022n ± 1% +14.82% (p=0.002 n=6)
MapIterLowLoad/Key=int64/Elem=int64/len=64-12 2.403n ± 2% 1.475n ± 1% -38.63% (p=0.002 n=6)
MapIterLowLoad/Key=int64/Elem=int64/len=65536-12 1.940n ± 3% 1.210n ± 6% -37.67% (p=0.002 n=6)
MapPop/Key=int64/Elem=int64/len=6-12 66.20n ± 2% 61.54n ± 2% -7.02% (p=0.002 n=6)
MapPop/Key=int64/Elem=int64/len=64-12 97.24n ± 1% 110.10n ± 1% +13.23% (p=0.002 n=6)
MapPop/Key=int64/Elem=int64/len=65536-12 826.1n ± 12% 504.7n ± 6% -38.91% (p=0.002 n=6)
geomean 17.93n 15.29n -14.74%
For #54766.
Cq-Include-Trybots: luci.golang.try:gotip-linux-ppc64_power10
Change-Id: Ic07f9df763239e85be57873103df5007144fdaef
Reviewed-on: https://go-review.googlesource.com/c/go/+/627156
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
This package is in charge of the FIPS init-time code+data verification.
If GODEBUG=fips140=off or the empty string, then no verification
happens. Otherwise, the setting must be "on", "debug", or "only",
all of which enable verification. If the setting is "debug", successful
verification prints a message to that effect. Otherwise successful
verification is quiet.
The linker leaves special information for this package to use.
See cmd/internal/obj/fips.go and cmd/link/internal/ld/fips.go,
both submitted in earlier CLs, for details.
For #69536.
Change-Id: Ie1fe29f316db290e0bd7df0a5a09108be4779d63
Reviewed-on: https://go-review.googlesource.com/c/go/+/625998
Reviewed-by: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
For FIPS init-time code+data verification, we need to arrange to
put the FIPS symbols into contiguous regions of the executable
and then record those sections along with the expected checksum.
The cmd/internal/obj changes identify the FIPS symbols and give
them distinguished types, which the linker then places in contiguous
regions. The linker also writes out information to use at run time
to find the FIPS sections, along with the expected hash.
See cmd/internal/obj/fips.go and cmd/link/internal/ld/fips.go
for more details.
The code is disabled in this commit.
CL 625998 and 625999 adds tests.
CL 626000 enables the code.
For #69536.
Change-Id: I48da6db94bc0bea7428c43d4abcf999527bccfcd
Reviewed-on: https://go-review.googlesource.com/c/go/+/625997
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The failures in #70288 are consistent with and strongly imply
stack corruption during fault handling, and debug prints show
that the Go code run during fault handling is running about
300 bytes above the bottom of the goroutine stack.
That should be okay, but that implies the DLL code that called
Go's handler was running near the bottom of the stack too,
and maybe it called other deeper things before or after the
Go handler and smashed the stack that way.
stackSystem is already 4096 bytes on amd64;
making it match that on 386 makes the flaky failures go away.
It's a little unsatisfying not to be able to say exactly what is
overflowing the stack, but the circumstantial evidence is
very strong that it's Windows.
Fixes#70288.
Change-Id: Ife89385873d5e5062a71629dbfee40825edefa49
Reviewed-on: https://go-review.googlesource.com/c/go/+/627375
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL refers to the implementation of ARM64 and adds support for the following
types of SIMD instructions:
1. Move general-purpose register to a vector element, e.g.:
VMOVQ Rj, <Vd>.<T>[index]
<T> can have the following values:
B, H, W, V
2. Move vector element to general-purpose register, e.g.:
VMOVQ <Vj>.<T>[index], Rd
<T> can have the following values:
B, BU, H, HU, W, WU, VU
3. Duplicate general-purpose register to vector, e.g.:
VMOVQ Rj, <Vd>.<T>
<T> can have the following values:
B16, H8, W4, V2, B32, H16, W8, V4
4. Move vector, e.g.:
XVMOVQ Xj, <Xd>.<T>
<T> can have the following values:
B16, H8, W4, V2, Q1
5. Move vector element to scalar, e.g.:
XVMOVQ Xj, <Xd>.<T>[index]
XVMOVQ Xj.<T>[index], Xd
<T> can have the following values:
W, V
6. Move vector element to vector register, e.g.:
VMOVQ <Vn>.<T>[index], Vn.<T>
<T> can have the following values:
B, H, W, V
This CL only adds syntax and doesn't break any assembly that already exists.
Change-Id: I7656efac6def54da6c5ae182f39c2a21bfdf92bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/616258
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Meidan Li <limeidan@loongson.cn>
Reviewed-by: sophie zhao <zhaoxiaolin@loongson.cn>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This fixes a bug in the test only function Import where it looked for
the first instance of the string "\n$$\n" as the end of the exportdata
section. This should look for the last instance of "\n$$\n" within
the ar file.
Adds unit tests that demonstrate the error.
Added comments to tests that can correctly use the first instance.
Change-Id: I7a85afa41cf1c2902119516b757b7c6625d46d13
Reviewed-on: https://go-review.googlesource.com/c/go/+/626775
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Now that we support pointer types on wasmimport functions, use
them, instead of unsafe.Pointer. This removes unsafe conversions.
There is still one unsafe.Pointer argument left. It is actually a
*Stat_t, which is an exported type with an int field, which is not
allowed as a wasmimport field type. We probably cannot change it
at this point.
Updates #66984.
Change-Id: I445c70b356c3877a5604bee67d19d99a538c682e
Reviewed-on: https://go-review.googlesource.com/c/go/+/627059
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
As proposed on #66984, this CL allows more types to be used as
wasmimport/wasmexport function parameters and results.
Specifically, bool, string, and uintptr are now allowed, and also
pointer types that point to allowed element types. Allowed element
types includes sized integer and floating point types (including
small integer types like uint8 which are not directly allowed as
a parameter type), bool, array whose element type is allowed, and
struct whose fields are allowed element type and also include a
struct.HostLayout field.
For #66984.
Change-Id: Ie5452a1eda21c089780dfb4d4246de6008655c84
Reviewed-on: https://go-review.googlesource.com/c/go/+/626615
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Add FIPS symbol kinds that will be needed for FIPS support.
This is a separate CL to keep the re-generated changes in
the string methods separate from hand-written changes.
The separate symbol kinds will let us group the FIPS-related
code and data together, so that it can be checksummed at
startup, as required by FIPS.
It's also separate because it breaks buildall, by changing the
on-disk symbol kind enumeration. We want non-buildall
changes to be as simple as possible.
For #69536.
Change-Id: I2d5a238498929fff8b24736ee54330c17323bd86
Reviewed-on: https://go-review.googlesource.com/c/go/+/625995
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
As the comment notes, all calls to Errorf now pass nil,
so remove that argument entirely.
There is a TODO to remove uses of Errorf entirely, but
that seems wrong: sometimes there is no symbol on
which to report the error, and in that situation, Errorf is
appropriate. So clarify that in the docs.
Change-Id: I92b3b6e8e3f61ba8356ace8cd09573d0b55d7869
Reviewed-on: https://go-review.googlesource.com/c/go/+/625617
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
The old API was to do
r := obj.AddRel(sym)
r.Type = this
r.Off = that
etc
The new API is:
sym.AddRel(ctxt, obj.Reloc{Type: this: Off: that, etc})
This new API is more idiomatic and avoids ever having relocations
that are only partially constructed. Most importantly, it sets up
for sym.AddRel being able to check relocation validity in the future.
(Passing ctxt is for use in validity checking.)
Passes golang.org/x/tools/cmd/toolstash/buildall.
Change-Id: I042ea76e61bb3bf6402f98ca11291a13f4799972
Reviewed-on: https://go-review.googlesource.com/c/go/+/625616
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
On a 301 redirect, the HTTP client changes the request to be
a GET with no body.
On a 308 redirect, the client leaves the request method and
body unchanged.
A 308 following a 301 should preserve the rewritten request
from the first redirect: GET with no body. We were preserving
the method, but sending the original body. Fix this.
Fixes#70180
Change-Id: Ie20027a6058a82bfdffc7197d07ac6c7f98099e2
Reviewed-on: https://go-review.googlesource.com/c/go/+/626055
Reviewed-by: Jonathan Amsterdam <jba@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Strictly speaking, the sort comparison was inconsistent
(and therefore invalid) for the sort-by-name case, if you had
a size 0
b size 1
c size 0
zerobase
That would result in the inconsistent comparison ordering:
a < b (by name)
b < c (by name)
c < zerobase (by zerobase rule)
zerobase < b (by zerobase rule)
This can't happen today because we only disable size-based
sort in a segment that has no zerobase symbol, but it's
confusing to reason through that, so clean up the code anyway.
Passes golang.org/x/tools/cmd/toolstash/buildall.
Change-Id: I21e4159cdedd2053952ba960530d1b0f28c6fb24
Reviewed-on: https://go-review.googlesource.com/c/go/+/625615
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
syscall.SyscallN is implemented by runtime.syscall_syscalln, which makes
sure that the variadic argument doesn't escape.
There is no need to worry about the lifetime of the elements of the
variadic argument, as the compiler will keep them live until the
function returns.
Fixes#70197.
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-amd64-race
Change-Id: I12991f0be12062eea68f2b103fa0a794c1b527eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/625297
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
For the SubFromLen64 codegen test case to work as intended, we need
to fold c-(-(x-d)) into x+(c-d).
Still, some instances of LeadingZeros are not optimized into single
CLZ instructions right now (actually, the LeadingZeros micro-benchmarks
are currently still compiled with redundant adds/subs of 64, due to
interference of loop optimizations before lowering), but perf numbers
indicate it's not that bad after all.
Micro-benchmark results on Loongson 3A5000 and 3A6000:
goos: linux
goarch: loong64
pkg: math/bits
cpu: Loongson-3A5000 @ 2500.00MHz
| bench.old | bench.new |
| sec/op | sec/op vs base |
LeadingZeros 3.660n ± 0% 1.348n ± 0% -63.17% (p=0.000 n=20)
LeadingZeros8 1.777n ± 0% 1.767n ± 0% -0.56% (p=0.000 n=20)
LeadingZeros16 2.816n ± 0% 1.770n ± 0% -37.14% (p=0.000 n=20)
LeadingZeros32 5.293n ± 1% 1.683n ± 0% -68.21% (p=0.000 n=20)
LeadingZeros64 3.622n ± 0% 1.349n ± 0% -62.76% (p=0.000 n=20)
geomean 3.229n 1.571n -51.35%
goos: linux
goarch: loong64
pkg: math/bits
cpu: Loongson-3A6000 @ 2500.00MHz
| bench.old | bench.new |
| sec/op | sec/op vs base |
LeadingZeros 2.410n ± 0% 1.103n ± 1% -54.23% (p=0.000 n=20)
LeadingZeros8 1.236n ± 0% 1.501n ± 0% +21.44% (p=0.000 n=20)
LeadingZeros16 2.106n ± 0% 1.501n ± 0% -28.73% (p=0.000 n=20)
LeadingZeros32 2.860n ± 0% 1.324n ± 0% -53.72% (p=0.000 n=20)
LeadingZeros64 2.6135n ± 0% 0.9509n ± 0% -63.62% (p=0.000 n=20)
geomean 2.159n 1.256n -41.81%
Updates #59120
This patch is a copy of CL 483356.
Co-authored-by: WANG Xuerui <git@xen0n.name>
Change-Id: Iee81a17f7da06d77a427e73dfcc016f2b15ae556
Reviewed-on: https://go-review.googlesource.com/c/go/+/624575
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: abner chenc <chenguoqi@loongson.cn>
A part of the keeping Go's vendored dependencies and generated code
up to date.
For #36905.
[git-generate]
cd src
go get golang.org/x/sys@v0.26.1-0.20241105152852-e0753d469443
go mod tidy
go mod vendor
cd cmd
go get golang.org/x/sys@v0.26.1-0.20241105152852-e0753d469443
go mod tidy
go mod vendor
go generate syscall internal/syscall/...
Change-Id: Ia84505f8934399f7c4518c6218892b81d30e5c17
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/623821
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Compared with the version generated by dec64.rules based on Ctz32,
the number of assembly instructions is reduced by half.
SwissMap uses TrailingZeros64 to find the first match in its control
group and may benefit from this CL on 386 architectures.
goos: linux
goarch: 386
cpu: 13th Gen Intel(R) Core(TM) i7-13700H
│ old.txt │ new.txt │
│ sec/op │ sec/op vs base │
TrailingZeros64-20 0.8828n ± 1% 0.6299n ± 1% -28.65% (p=0.000 n=20)
Change-Id: Iba08a3f4e13efd3349715dfb7fcd5fd470286cd3
Reviewed-on: https://go-review.googlesource.com/c/go/+/624376
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Keith Randall <khr@golang.org>
The TestProfilerStackDepth/heap test can spuriously fail if the profiler
happens to capture a stack with an allocation several frames deep into
runtime code. The pprof API hides runtime frames at the leaf-end of
stacks, but those frames still count against the profiler's stack depth
limit. The test checks only the first stack it finds with the desired
prefix and fails if it's not deep enough or doesn't have the right root
frame. So it can fail in that scenario, even though the implementation
isn't really broken.
Relax the test to check that there is at least one stack with desired
prefix, depth, and root frame.
Fixes#70112
Change-Id: I337fb3cccd1ddde76530b03aa1ec0f9608aa4112
Reviewed-on: https://go-review.googlesource.com/c/go/+/623998
Reviewed-by: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Add support for assembling the FMA instructions present in the LoongArch
base ISA v1.00. This requires adding a new instruction format and making
use of a third source operand, which is put in RestArgs[0].
The single-precision instructions have the `.s` prefix in their official
mnemonics, and similar Go asm instructions all have `S` prefix for the
other architectures having FMA support, but in this change they instead
have `F` prefix in Go asm because loong64 currently follows the mips
backends in the naming convention. This could be changed later because
FMA is fully expressible in pure Go, making it unlikely to have to hand-
write such assembly in the wild.
Example mapping between actual encoding and Go asm syntax:
fmadd.s fd, fj, fk, fa -> FMADDF fa, fk, fj, fd
(prog.From = fa, prog.Reg = fk, prog.RestArgs[0] = fj and prog.To = fd)
fmadd.s fd, fd, fk, fa -> FMADDF fa, fk, fd
(prog.From = fa, prog.Reg = fk and prog.To = fd)
This patch is a copy of CL 477716.
Co-authored-by: WANG Xuerui <git@xen0n.name>
Change-Id: I9b4e4c601d6c5a854ee238f085849666e4faf090
Reviewed-on: https://go-review.googlesource.com/c/go/+/623877
Reviewed-by: abner chenc <chenguoqi@loongson.cn>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Seems simple, but putting the return after fatal ensures that at the
point of the small group loop, no call has happened so the key is
still in a register. This ensures that we don't have to restore the
key from the stack before the comparison on each iteration. That gets
rid of a load from the inner loop.
name old time/op new time/op delta
MapAccessHit/Key=int64/Elem=int64/len=6-8 4.01ns ± 6% 3.85ns ± 3% -3.92% (p=0.001 n=10+10)
Change-Id: Ia23ac48e6c5522be88f7d9be0ff3489b2dfc52fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/624255
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Ever since we had to upgrade from our COS image, we've been experiencing
TSAN test failures. My best guess is that the ASLR randomization entropy
increased, causing TSAN to fail. TSAN already re-execs itself in Clang
18+ with ASLR disabled, so just execute the tests with ASLR disabled on
Linux.
Fixes#59418.
Change-Id: Icb4536ddf0f2f5e7850734564d40f5a208ab8d01
Cq-Include-Trybots: luci.golang.try:gotip-linux-386,gotip-linux-386-clang15,gotip-linux-amd64-clang15,gotip-linux-amd64-boringcrypto,gotip-linux-amd64-aliastypeparams,gotip-linux-amd64-asan-clang15,gotip-linux-amd64-msan-clang15,gotip-linux-amd64-goamd64v3
Reviewed-on: https://go-review.googlesource.com/c/go/+/623956
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
All remaining unary bitop instructions in the LoongArch v1.00 base ISA
are added with this change.
While at it, add the missing W suffix to the current CLO/CLZ names. They
are not used anywhere as far as we know, so no breakage is expected.
Also, stop reusing SLL's instruction format for simplicity, in favor of
a new but trivial instruction format case.
This patch is a copy of CL 477717.
Co-authored-by: WANG Xuerui <git@xen0n.name>
Change-Id: Idbcaca25dda1ed313674ef8b26da722e8d7151c0
Reviewed-on: https://go-review.googlesource.com/c/go/+/623876
Reviewed-by: abner chenc <chenguoqi@loongson.cn>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
This tests fails sporadically on the aix-ppc64 CI. I suspect this is
an aix performance related issue. Skip the test.
AIX seems slow to perform a non-blocking reading on a pipe, and this
results in too many threads being created. This happens as far back
as go1.22, where I stopped looking.
On the GCC farm machine gcc119, The failure rate seemed coupled to
GOMAXPROCS; about 1% for <=8, up to 40%+ for >=30 for all releases
tested.
For #70131
Change-Id: If002b55e5a4586d10cc7876d7c25259e61b17163
Reviewed-on: https://go-review.googlesource.com/c/go/+/623817
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Avoid integer overflow when passing a number of bytes to sendfile.
Also, Solaris might not support passing a 0 length to read to
the end of a file, but it does support passing a very large length.
So just do that instead of looking up the source file size.
Change-Id: Ibf750892938d9e2bafb1256c6e380c88899495f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/623315
TryBot-Bypass: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Currently, at a cgo callback where there is already a Go frame on
the stack (i.e. C->Go->C->Go), we require that at the inner Go
callback the SP is within the g0's stack bounds set by a previous
callback. This is to prevent that the C code switches stack while
having a Go frame on the stack, which we don't really support. But
this could also happen when we cannot get accurate stack bounds,
e.g. when pthread_getattr_np is not available. Since the stack
bounds are just estimates based on the current SP, if there are
multiple C->Go callbacks with various stack depth, it is possible
that the SP of a later callback falls out of a previous call's
estimate. This leads to runtime throw in a seemingly reasonable
program.
This CL changes it to save the old g0 stack bounds at cgocallback,
update the bounds, and restore the old bounds at return. So each
callback will get its own stack bounds based on the current SP,
and when it returns, the outer callback has the its old stack
bounds restored.
Also, at a cgo callback when there is no Go frame on the stack,
we currently always get new stack bounds. We do this because if
we can only get estimated bounds based on the SP, and the stack
depth varies a lot between two C->Go calls, the previous
estimates may be off and we fall out or nearly fall out of the
previous bounds. But this causes a performance problem: the
pthread API to get accurate stack bounds (pthread_getattr_np) is
very slow when called on the main thread. Getting the stack bounds
every time significantly slows down repeated C->Go calls on the
main thread.
This CL fixes it by "caching" the stack bounds if they are
accurate. I.e. at the second time Go calls into C, if the previous
stack bounds are accurate, and the current SP is in bounds, we can
be sure it is the same stack and we don't need to update the bounds.
This avoids the repeated calls to pthread_getattr_np. If we cannot
get the accurate bounds, we continue to update the stack bounds
based on the SP, and that operation is very cheap.
On a Linux/AMD64 machine with glibc:
name old time/op new time/op delta
CgoCallbackMainThread-8 96.4µs ± 3% 0.1µs ± 2% -99.92% (p=0.000 n=10+9)
Fixes#68285.
Fixes#68587.
Change-Id: I3422badd5ad8ff63e1a733152d05fb7a44d5d435
Reviewed-on: https://go-review.googlesource.com/c/go/+/600296
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
The compiler will stack allocate the Map struct and initial group if
possible.
Stack maps are initialized inline without calling into the runtime.
Small heap allocated maps use makemap_small.
These are the same heuristics as existing maps.
For #54766.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest-swissmap
Change-Id: I6c371d1309716fd1c38a3212d417b3c76db5c9b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/622042
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Load the field we need from the type once outside the search loop.
Get rid of the multiply to compute the slot position. Instead compute
the slot position incrementally using addition.
Move the hashing later in access2.
Based on khr@'s CL 618959.
For #54766.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest-swissmap
Change-Id: Id11b5479fa5bc0130a1d8d9e664d0206d24942ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/620217
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Add all the specialized variants that exist for the existing maps.
Like the existing maps, the fast variants do not support indirect
key/elem.
Note that as of this CL, the Get and Put methods on Map/table are
effectively dead. They are only reachable from the internal/runtime/maps
unit tests.
For #54766.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest-swissmap
Change-Id: I95297750be6200f34ec483e4cfc897f048c26db7
Reviewed-on: https://go-review.googlesource.com/c/go/+/616463
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Regenerate RISC-V instruction table from the riscv-opcodes repository,
due to various changes and shuffling upstream.
This has been changed to remove pseudo-instructions, since Go only
needs the instruction encodings and including the pseudo-instructions
is creating unnecessary complications (for example, the inclusion
of ANOP and ARET, as well as strangely named aliases such as
AJALPSEUDO/AJALRPSEUDO). Remove pseudo-instructions that are not
currently supported by the assembler and add specific handling for
RDCYCLE, RDTIME and RDINSTRET, which were previously implemented
via the instruction encodings.
Change-Id: I78be4506ba6b627eba1f321406081a63bab5b2e6
Cq-Include-Trybots: luci.golang.try:gotip-linux-riscv64
Reviewed-on: https://go-review.googlesource.com/c/go/+/616116
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
On a whim I decided to investigate the possibility of whether the
flakiness on the asan builder was due to a concurrently executing test.
Of the most recent failures there were a few candidates, and this test
was one of them. After disabling each candidate one by one, we had a
winner: this test causes other concurrently executing tests, running
pure Go code, to spuriously fail.
I do not know why yet, but this test doesn't seem like it would have
incredibly high value for ASAN, and does funky things like MAP_FIXED in
recently unmapped regions, so I think it's fine.
For #70054.
For #64257.
Change-Id: Ib9a84d9b69812e76c390d99b00698710ee1ece1a
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-asan-clang15
Reviewed-on: https://go-review.googlesource.com/c/go/+/623336
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
If there is a phi that is computing the minimum of its two inputs,
then we know the result of the phi is smaller than or equal to both
of its inputs. Similarly for maxiumum (although max seems less useful).
This pattern happens for the case
n := copy(a, b)
n is the minimum of len(a) and len(b), so with this optimization we
know both n <= len(a) and n <= len(b). That extra information is
helpful for subsequent slicing of a or b.
Fixes#16833
Change-Id: Ib4238fd1edae0f2940f62a5516a6b363bbe7928c
Reviewed-on: https://go-review.googlesource.com/c/go/+/622240
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
These fail for the same reason as for the race detector, and is the most
frequently failing test in both.
For #70054.
For #64257.
For #64256.
Change-Id: I3649e58069190b4450f9d4deae6eb8eca5f827a3
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-asan-clang15,gotip-linux-amd64-msan-clang15
Reviewed-on: https://go-review.googlesource.com/c/go/+/623176
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
BGT, BLT, BLE, BGE, BNE, BVS, BVC, and BEQ support by assembler. This will simplify the usage of BC constructs like
BC 12, 30, LR <=> BEQ CR7, LR
BC 12, 2, LR <=> BEQ CR0, LR
BC 12, 0, target <=> BLT CR0, target
BC 12, 2, target <=> BEQ CR0, target
BC 12, 5, target <=> BGT CR1, target
BC 12, 30, target <=> BEQ CR7, target
BC 4, 6, target <=> BNE CR1, target
BC 4, 5, target <=> BLE CR1, target
code cleanup based on the above additions.
Change-Id: I02fdb212b6fe3f85ce447e05f4d42118c9ce63b5
Cq-Include-Trybots: luci.golang.try:gotip-linux-ppc64_power10,gotip-linux-ppc64_power8,gotip-linux-ppc64le_power8,gotip-linux-ppc64le_power9,gotip-linux-ppc64le_power10
Reviewed-on: https://go-review.googlesource.com/c/go/+/612395
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Paul Murphy <murp@ibm.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
The internal/poll/sendfile_{bsd,linux,solaris}.go implementations
have more in common than not. Combine into a single sendfile_unix.go.
The net and os packages have redundant code dealing with sendfile
quirks on non-Linux Unix systems, such as the need to determine the
size of the source file before sending. Move the common code into
internal/poll.
Remove some obsolete or incorrect behaviors:
Drop the maximum sendfile chunk size. If we ask the kernel
to copy more data than it is willing to send, it'll copy up to
its limit.
There was a comment in net/sendfile_unix_alt.go indicating that
copying more bytes than a file contains results in the kernel
looping back to the start of the file. I am unable to replicate
this behavior anywhere. Dropped the comment, the workarounds,
and added a test covering this case.
Darwin, Dragonfly, and FreeBSD all support copying the entire
contents of a file by passing 0 for the copy limit.
Take advantage of this.
Change-Id: I9f707ac7a27c165020ae02a6b5bb8f6f16f3c530
Reviewed-on: https://go-review.googlesource.com/c/go/+/621416
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
First, skip all the allocation count tests.
In some cases this aligns with existing skips for -race, but in others
we've got new issues. These are debug modes, so some performance loss is
expected, and this is clearly no worse than today where the tests fail.
Next, skip internal linking and static linking tests for msan and asan.
With asan we get an explicit failure that neither are supported by the C
and/or Go compilers. With msan, we only get the Go compiler telling us
internal linking is unavailable. With static linking, we segfault
instead. Filed #70080 to track that.
Next, skip some malloc tests with asan that don't quite work because of
the redzone.
This is because of some sizeclass assumptions that get broken with the
redzone and the fact that the tiny allocator is effectively disabled
(again, due to the redzone).
Next, skip some runtime/pprof tests with asan, because of extra
allocations.
Next, skip some malloc tests with asan that also fail because of extra
allocations.
Next, fix up memstats accounting for arenas when asan is enabled. There
is a bug where more is added to the stats than subtracted. This also
simplifies the accounting a little.
Next, skip race tests with msan or asan enabled; they're mutually
incompatible.
Fixes#70054.
Fixes#64256.
Fixes#64257.
For #70079.
For #70080.
Change-Id: I99c02a0b9d621e44f1f918b307aa4a4944c3ec60
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-asan-clang15,gotip-linux-amd64-msan-clang15
Reviewed-on: https://go-review.googlesource.com/c/go/+/622855
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
We still use the hash and control word, but loop over all 8 bytes
instead of doing the match operation, which ends up being slightly
faster when there is only one group.
Note that specialized variants added later will avoid hashing at all.
For #54766.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest-swissmap
Change-Id: I3bb353b023dd6120b6585e87d3efe2f18ac9e1ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/611189
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
If the map contains 8 or fewer entries, it is wasteful to have a
directory that points to a table that points to a group.
Add a special case that replaces the directory with a direct pointer to
a group.
We could theoretically do similar for single table maps (no directory,
just point directly to a table), but that is left for later.
For #54766.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest-swissmap
Change-Id: I6fc04dfc11c31dadfe5b5d6481b4c4abd43d48ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/611188
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Keith Randall <khr@google.com>
While walking the probe sequence, Put keeps track of the first deleted
slot it encountered. If it reaches the end of the probe sequence without
finding a match, then it will prefer to use the deleted slot rather than
a new empty slot.
For #54766.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest-swissmap
Change-Id: I19356ef6780176506f57b42990ac15dc426f1b14
Reviewed-on: https://go-review.googlesource.com/c/go/+/618016
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
This change amends the long-form help output for 'go help build' and
'go help testflag' to specify that the '-coverpkg' flag operates
explicitly on import paths as well as package names. Import paths are
fundamental for precise specification of packages versus unqualified
package names, and the naming of the flag '-coverpkg' and its original
documentation leads a user to assume that it only operates on the
simple, unqualified package name form. The situation warrants
clarification.
Fixes#69653
Change-Id: Ifde6a974405ce1614e28898fc2b92ed5bad94e57
GitHub-Last-Rev: 466c662a70
GitHub-Pull-Request: golang/go#69655
Reviewed-on: https://go-review.googlesource.com/c/go/+/616257
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Adds a new crypto/internal/fips test binary that operates as both a unit
test fetching/driving the BoringSSL acvptool, and an acvptool module
wraper when invoked by the unit test. Initial support for testing the
SHA2 and SHA3 family of digests, and the HMAC family of MACs is
included.
Test vectors and expected answers are maintained in a separate repo,
`github.com/cpu/go-acvp` and fetched through the module proxy as part of
the test process.
The BSSL acvptool "lowers" the NIST ACVP server JSON test vectors into
a simpler stdin/stdout protocol that can be implemented by a module
wrapper. The tool will fork our acvpwrapper binary, request the
supported configuration, and then provide test cases over stdin,
expecting results to be returned on stdout.
See "Testing other FIPS modules" from the BoringSSL ACVP.md
documentation for a more detailed description of the protocol used
between the acvptool and module wrappers.
Updates #69642
Updates #69536
Change-Id: I6b568c67f2a71144fbf31db467c6fd25710457f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/615816
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
It's a little annoying, but we can fit the IBM instructions on top of
the regular state, avoiding more intrusive interventions.
Going forward we should not accept assembly that replaces the whole
implementation, because it doubles the work to do any refactoring like
the one in this chain.
Also, it took me a while to find the specification of these
instructions, which should have been linked from the source for the next
person who'd have to touch this.
Finally, it's really painful to test this without a LUCI TryBot, per #67307.
For #69536
Change-Id: I90632a90f06b2aa2e863967de972b12dbaa5b2ae
Reviewed-on: https://go-review.googlesource.com/c/go/+/617359
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Main changes are
- return concrete *Digest and *SHAKE instead of interfaces
- make tests external (sha3_test) so they will be easy to move to
the public package
- drop most of the developer guidance docs (to be updated and
reintroduced in the public package)
- consolidate the _noasm.go files (matching the single _s390x.go)
- move TestAllocations from build tags to testenv
- temporarily disable s390x code, to refactor in a following CL
For #69536
Change-Id: Ie5fd3e2b589b9eb835b9e3174b7a79c2ac728ab1
Reviewed-on: https://go-review.googlesource.com/c/go/+/617357
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
A GODEBUG is actually a security risk here: most programs will start to
ignore errors from Read because they can't happen (which is the intended
behavior), but then if a program is run with GODEBUG=randcrash=0 it will
use a partial buffer in case an error occurs, which may be catastrophic.
Note that the proposal was accepted without the GODEBUG, which was only
added later.
This (partially) reverts CL 608435. I kept the tests.
Updates #66821
Change-Id: I3fd20f9cae0d34115133fe935f0cfc7a741a2662
Reviewed-on: https://go-review.googlesource.com/c/go/+/622115
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
With LDREXB/STREXB now available for the arm assembler we can implement these operations natively. The instructions are armv6k+ but for simplicity I only use them on armv7.
Benchmark results for a raspberry Pi 3 model B+:
goos: linux
goarch: arm
pkg: internal/runtime/atomic
cpu: ARMv7 Processor rev 4 (v7l)
│ old.txt │ new.txt │
│ sec/op │ sec/op vs base │
And8-4 127.65n ± 0% 68.74n ± 0% -46.15% (p=0.000 n=10)
Change-Id: Ic87f307c35f7d7f56010980302f253056f6d54dc
GitHub-Last-Rev: a7351802fd
GitHub-Pull-Request: golang/go#70002
Cq-Include-Trybots: luci.golang.try:gotip-linux-arm
Reviewed-on: https://go-review.googlesource.com/c/go/+/622075
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This change finally fully fixes mallocgc for asan after the recent
refactoring. Here is everything that changed:
Fix the accounting for the alloc header; large objects don't have them.
Mask out extra bits set from unrolling the bitmap for slice backing
stores in writeHeapBitsSmall. The redzone in asan mode makes it so that
dataSize is no longer an exact multiple of typ.Size_ in this case (a
new assumption I have recently discovered) but we didn't mask out any
extra bits, so we'd accidentally set bits in other allocations. Oops.
Move the initHeapBits optimization for the 8-byte scan sizeclass on
64-bit platforms up to mallocgc, out from writeHeapBitsSmall. So, this
actually caused a problem with asan when the optimization first landed,
but we missed it. The issue was then masked once we started passing the
redzone down into writeHeapBitsSmall, since the optimization would no
longer erroneously fire on asan. What happened was that dataSize would
be 8 (because that was the user-provided alloc size) so we'd skip
writing heap bits, but it would turn out the redzone bumped the size
class, so we'd actually *have* to write the heap bits for that size
class. This is not really a problem now *but* it caused problems for me
when debugging, since I would try to remove the red zone from dataSize
and this would trigger this bug again. Ultimately, this whole situation
is confusing because the check in writeHeapBitsSmall is *not* the same
as the check in initHeapBits. By moving this check up to mallocgc, we
can make the checks align better by matching on the sizeclass, so this
should be less error-prone in the future.
Change-Id: I1e9819223be23f722f3bf21e63e812f5fb557194
Reviewed-on: https://go-review.googlesource.com/c/go/+/622041
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
CL 622235 would fix#70000 while resulting in one extra sendfile(2) system
call when sendfile(2) returns (>0, EAGAIN).
That's also why I left sendfile_bsd.go behind, and didn't make it line up
with other two implementations: sendfile_linux.go and sendfile_solaris.go.
Unlike sendfile(2)'s on Linux and Solaris that always return (0, EAGAIN),
sendfile(2)'s on *BSD and macOS may return (>0, EAGAIN) when using a socket
marked for non-blocking I/O. In that case, the current code will try to re-call
sendfile(2) immediately, which will most likely get us a (0, EAGAIN).
After that, it goes to `dstFD.pd.waitWrite(dstFD.isFile)` below,
which should have been done in the first place.
Thus, the real problem that leads to #70000 is that the old code doesn't handle
the special case of sendfile(2) sending the exact number of bytes the caller requested.
Fixes#70000
Change-Id: I6073d6b9feb58b3d7e114ec21e4e80d9727bca66
Reviewed-on: https://go-review.googlesource.com/c/go/+/622255
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Andy Pan <panjf2000@gmail.com>
Sometimes the runtime needs to reserve some memory with a large
alignment, which the OS usually won't directly satisfy. So, it
asks size+align bytes instead, and frees the unaligned portions.
On sbrk systems, this doesn't work that well, as freeing the tail
portion doesn't really free the memory to the OS. Instead, we
could simply round the current break up, then reserve the given
size, without wasting the tail portion.
Also, don't create heap arena hints on sbrk systems. We can only
grow the break sequentially, and reserving specific addresses
would not succeed anyway.
For #69018.
Change-Id: Iadc2c54d62b00ad7befa5bbf71146523483a8c47
Reviewed-on: https://go-review.googlesource.com/c/go/+/621715
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Goroutine profiles require checking in with the profiler before any
goroutine starts running. coroswitch is a place where a goroutine may
start running, but where we do not check in with the profiler, which
leads to crashes. Fix this by checking in with the profiler the same way
execute does.
Fixes#69998.
Change-Id: Idef6dd31b70a73dd1c967b56c307c7a46a26ba73
Reviewed-on: https://go-review.googlesource.com/c/go/+/622016
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
A previous CL broke the ASAN poisoning calculation in mallocgc by not
taking into account a possible allocation header, so the beginning of
the following allocation could have been poisoned.
This mostly isn't a problem, actually, since the following slot would
usually just have an allocation header in it that programs shouldn't be
touching anyway, but if we're going a word-past-the-end at the end of a
span, we could be poisoning a valid heap allocation.
Change-Id: I76a4f59bcef01af513a1640c4c212c0eb6be85b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/622295
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Replace Transport's limit of 5 1xx responses with a limit based
on MaxResponseHeaderBytes: The total number of responses
(including 1xx reponses and the final response) must not exceed
this value.
When the user is reading 1xx responses using a Got1xxResponse
client trace hook, disable the limit: Each 1xx response is
individually limited by MaxResponseHeaderBytes, but there
is no limit on the total number of responses. The user is
responsible for imposing a limit if they want one.
For #65035
Change-Id: If4bbbbb0b808cb5016701d50963c89f0ce1229f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/615255
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The BSD implementation of poll.SendFile incorrectly halted
copying after succesfully writing one full chunk of data.
Adjust the copy loop to match the Linux and Solaris
implementations.
In testing, empirically macOS appears to sometimes return
EAGAIN from sendfile after successfully copying a full
chunk. Add a check to all implementations to return nil
after successfully copying all data if the last sendfile
call returns EAGAIN.
For #70000
Change-Id: I57ba649491fc078c7330310b23e1cfd85135c8ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/622235
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
This will be required for #69536 but is also good hygiene and required
by go.dev/wiki/AssemblyPolicy.
> The code must be tested in our CI. This means there need to be
> builders that support the instructions, and if there are multiple (or
> fallback) paths they must be tested separately.
The new crypto/internal/impl registry lets us select alternative
implementations from both the same package and importers (such as
crypto/sha256 tests once we have crypto/internal/fips/sha256, or
crypto/hmac).
Updates #69592
Updates #69593
Change-Id: Ifea22a9fc9ccffcaf4924ff6bd08da7c9bd39e99
Cq-Include-Trybots: luci.golang.try:gotip-linux-arm64-longtest,gotip-linux-amd64-longtest,gotip-linux-ppc64le_power8,gotip-linux-ppc64_power8
Reviewed-on: https://go-review.googlesource.com/c/go/+/614656
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Change the description of an operand x that has a named type of sorts
by providing a description of the type structure (array, struct, slice,
pointer, etc).
For instance, given a (variable) operand x of a struct type T, the
operand is mentioned as (new):
x (variable of struct type T)
instead of (old):
x (variable of type T)
This approach is also used when a basic type is renamed, for instance
as in:
x (value of uint type big.Word)
which makes it clear that big.Word is a uint.
This change is expected to produce more informative error messages.
Fixes#69955.
Change-Id: I544b0698f753a522c3b6e1800a492a94974fbab7
Reviewed-on: https://go-review.googlesource.com/c/go/+/621458
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This change improves error message for recursive types.
Currently, compilation of the [following program](https://go.dev/play/p/3ef84ObpzfG):
package main
type T1[T T2] struct{}
type T2[T T1] struct{}
returns an error:
./prog.go:3:6: invalid recursive type T1
./prog.go:3:6: T1 refers to
./prog.go:4:6: T2 refers to
./prog.go:3:6: T1
With the patch applied the error message looks like:
./prog.go:3:6: invalid recursive type T1
./prog.go:3:6: T1 refers to T2
./prog.go:4:6: T2 refers to T1
Change-Id: Ic07cdffcffb1483c672b241fede4e694269b5b79
GitHub-Last-Rev: cd042fdc38
GitHub-Pull-Request: golang/go#69574
Reviewed-on: https://go-review.googlesource.com/c/go/+/614084
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Tim King <taking@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
If a function f being considered for inlining calls
one of its parameters, reduce the normal cost of that
call (57) to 17 to increase the chance that f will
be inlined and (with luck) that parameter will be
revealed as a constant function (which unblocks
escape analysis) or perhaps even be inlined.
The least-change value for that was still effective for
iter_test benchmarks was 32; however tests showed no
particular harm even when reduced as low as 7, and there
have been reports of other performance problems with
rangefunc overheads and so I picked a middling number
in hopes of warding off such reports.
Updates #69015
Change-Id: I2a525c1beffb9f88daa14caa8a622864b023675c
Reviewed-on: https://go-review.googlesource.com/c/go/+/609095
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Tim King <taking@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
The spill/restore code around morestack is almost never exectued, so
we should make it as small as possible. Using 2-register loads/stores
makes sense here. Also, the offsets from SP are pretty small so the
offset almost always fits in the (smaller than a normal load/store)
offset field of the instruction.
Makes cmd/go 0.6% smaller.
Change-Id: I8845283c1b269a259498153924428f6173bda293
Reviewed-on: https://go-review.googlesource.com/c/go/+/621556
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
As described in issue #69912, type checking dot-imported identifiers can
result in a call to objDecl on an imported object, which leads to a data
race to the color_ field.
There are multiple potential fixes for this race. Opt for avoiding the
call to objDecl altogether, rather than setting color_ during import.
The color_ field is an internal property of objects that should only be
valid during the type checking of their package. We should not be
calling objDecl on imported objects.
Fixes#69912
Change-Id: I55eb652479715f2a7ac84104db2f448091c4e7ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/621637
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Robert Findley <rfindley@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Currently, for Wasm, the linker sets the initial memory size to
the size of global data plus 16 MB. The intention is that it
covers the global data and runtime initialization without growing
the linear memory. However, the code accounts only the data
"section", not the bss "section", therefore the extra 16 MB is
actually used to cover bss variables. Also, as seen on the
previous CL, the runtime actually didn't use the extra space,
which means the program can start without that space.
This CL corrects the global data size calculation, and reduces the
extra to 1 MB. Currently the runtime's allocation pattern at
startup is that it allocates a few pages for the page allocator's
metadata, the an 8 MB reservation for the first 4 MB size, 4 MB
aligned heap arena (it may be possible to reduce that, but we'll
leave that for later). Here we use 1 MB extra space to cover the
small allocations, but let the runtime allocate the heap arena, so
the linker code and the runtime's allocator are not tightly
coupled.
For #69018.
Change-Id: I39fe1172382ecc03f4b537e43ec710af8075eab3
Reviewed-on: https://go-review.googlesource.com/c/go/+/621636
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
CL 476717 adopted the memory management mechanism on Plan 9 to
manage Wasm's linear memory. But the Plan 9 code uses global
variable bloc and blocMax to keep track of the runtime's and the
OS's sense of break, whereas the Wasm sbrk function doesn't use
those global variables, and directly goes to grow the linear
memory instead. This causes that if there is any unused portion at
the end of the linear memory, the runtime doesn't use it. This CL
fixes it, adopts the same mechanism as the Plan 9 code.
In particular, the runtime is not aware of any unused initial
memory at startup. Therefore, (most of) the extra initial memory
set by the linker are not actually used. This CL fixes this as
well.
For #69018.
Change-Id: I2ea6a138310627eda5f19a1c76b1e1327362e5f2
Reviewed-on: https://go-review.googlesource.com/c/go/+/621635
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This change switches isSending to be an atomic.Int32 instead of an
atomic.Uint8. The Int32 version is managed as a counter, which is
something that we couldn't do with Uint8 without adding a new intrinsic
which may not be available on all architectures.
That is, instead of only being able to support 8 concurrent timer
firings on the same timer because we only have 8 independent bits to set
for each concurrent timer firing, we can now have 2^31-1 concurrent
timer firings before running into any issues. Like the fact that each
bit-set was matched with a clear, here we match increments with
decrements to indicate that we're in the "sending on a channel" critical
section in the timer code, so we can report the correct result back on
Stop or Reset.
We choose an Int32 instead of a Uint32 because it's easier to check for
obviously bad values (negative values are always bad) and 2^31-1
concurrent timer firings should be enough for anyone.
Previously, we avoided anything bigger than a Uint8 because we could
pack it into some padding in the runtime.timer struct. But it turns out
that the type that actually matters, runtime.timeTimer, is exactly 96
bytes in size. This means its in the next size class up in the 112 byte
size class because of an allocation header. We thus have some free space
to work with. This change increases the size of this struct from 96
bytes to 104 bytes.
(I'm not sure if runtime.timer is often allocated directly, but if it
is, we get lucky in the same way too. It's exactly 80 bytes in size,
which means its in the 96-byte size class, leaving us with some space to
work with.)
Fixes#69969.
Related to #69880 and #69312.
Change-Id: I9fd59cb6a69365c62971d1f225490a65c58f3e77
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/621616
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently, on Mach-O, the -B UUID setting is only applied in
internal linking mode, whereas in external linking mode the UUID
is always rewritten to a hash of Go build ID. This CL makes it
apply to external linking as well. This makes the behavior
consistent on both linkmodes, and also consistent with the -B
flag's behavior for GNU build ID on ELF.
Add tests.
Updates #68678.
Cq-Include-Trybots: luci.golang.try:gotip-darwin-amd64_14,gotip-darwin-arm64_13
Change-Id: I276a5930e231141440cdba16e8812df28ac4237b
Reviewed-on: https://go-review.googlesource.com/c/go/+/618599
Reviewed-by: Than McIntosh <thanm@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
With the "-B gobuildid" linker option (which will be the default
on some platforms), the host build ID (GNU build ID, Mach-O UUID)
depends on the Go buildid. If the host build ID is included in the
Go buildid computation, it will lead to convergence problem for
the toolchain binaries. So ignore the host build ID in the buildid
computation.
This CL only handles Mach-O UUID. ELF GNU build ID will be handled
later.
For #68678.
For #63934.
Cq-Include-Trybots: luci.golang.try:gotip-darwin-amd64_14,gotip-darwin-arm64_13
Change-Id: Ie8ff20402a1c6083246d25dea391140c75be40d0
Reviewed-on: https://go-review.googlesource.com/c/go/+/618597
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@golang.org>
Currently the linker has some code handling and manipulating
Mach-O files. Specifically, it augments the debug/macho package
with file offset and length, so the content can be handled or
updated easily with the file.
Move this code to an internal package, so it can be used by other
part of the toolchain, e.g. buildid computation.
For #68678.
Cq-Include-Trybots: luci.golang.try:gotip-darwin-amd64_14,gotip-darwin-arm64_13
Change-Id: I2311af0a06441b7fd887ca5c6ed9e6fc44670a16
Reviewed-on: https://go-review.googlesource.com/c/go/+/618596
Reviewed-by: Than McIntosh <thanm@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently, on Mach-O, the Go linker doesn't generate LC_UUID in
internal linking mode. This causes some macOS system tools unable
to track the binary, as well as in some cases the binary unable
to access local network on macOS 15.
This CL makes the linker start generate LC_UUID. Currently, the
UUID is generated if the -B flag is specified. And we'll make it
generate UUID by default in a later CL. The -B flag is currently
for generating GNU build ID on ELF, which is a similar concept to
Mach-O's UUID. Instead of introducing another flag, we just use
the same flag and the same setting. Specifically, "-B gobuildid"
will generate a UUID based on the Go build ID.
For #68678.
Cq-Include-Trybots: luci.golang.try:gotip-darwin-amd64_14,gotip-darwin-arm64_13
Change-Id: I90089a78ba144110bf06c1c6836daf2d737ff10a
Reviewed-on: https://go-review.googlesource.com/c/go/+/618595
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ingo Oeser <nightlyone@googlemail.com>
Reviewed-by: Than McIntosh <thanm@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This is a peace-of-mind change to make sure that delayed-zeroed memory
(in the large alloc case) is globally visible from the moment the
allocation is published back to the caller.
The way it's written right now is good enough for the garbage collector
(we already have a publication barrier for a nil span.largeType, so the
GC will ignore the noscan span) but this might matter for user code on
weak memory architectures.
Change-Id: I06ac9b95863074e5f09382629083b19bfa87fdb8
Reviewed-on: https://go-review.googlesource.com/c/go/+/619036
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Last CL we separated mallocgc into several specialized paths. Let's
split up heapSetType too. This will make the specialized heapSetType
functions inlineable and cut out some branches as well as a function
call.
Microbenchmark results at this point in the stack:
│ before.out │ after-5.out │
│ sec/op │ sec/op vs base │
Malloc8-4 13.52n ± 3% 12.15n ± 2% -10.13% (p=0.002 n=6)
Malloc16-4 21.49n ± 2% 18.32n ± 4% -14.75% (p=0.002 n=6)
MallocTypeInfo8-4 27.12n ± 1% 18.64n ± 2% -31.30% (p=0.002 n=6)
MallocTypeInfo16-4 28.71n ± 3% 21.63n ± 5% -24.65% (p=0.002 n=6)
geomean 21.81n 17.31n -20.64%
Change-Id: I5de9ac5089b9eb49bf563af2a74e6dc564420e05
Reviewed-on: https://go-review.googlesource.com/c/go/+/614795
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Right now mallocgc is a monster of a function. In real programs, we see
that a substantial amount of time in mallocgc is spent in mallocgc
itself. It's very branch-y, holds a lot of state, and handles quite a few
disparate cases, trying to merge them together.
This change breaks apart mallocgc into separate, leaner functions.
There's some duplication now, but there are a lot of branches that can
be pruned as a result.
There's definitely still more we can do here. heapSetType can be inlined
and broken down for each case, since its internals roughly map to each
case anyway (done in a follow-up CL). We can probably also do more with
the size class lookups, since we know more about the size of the object
in each case than before.
Below are the savings for the full stack up until now.
│ after-3.out │ after-4.out │
│ sec/op │ sec/op vs base │
Malloc8-4 13.32n ± 2% 12.17n ± 1% -8.63% (p=0.002 n=6)
Malloc16-4 21.64n ± 3% 19.38n ± 10% -10.47% (p=0.002 n=6)
MallocTypeInfo8-4 23.15n ± 2% 19.91n ± 2% -14.00% (p=0.002 n=6)
MallocTypeInfo16-4 25.86n ± 4% 22.48n ± 5% -13.11% (p=0.002 n=6)
MallocLargeStruct-4 270.0n ± ∞ ¹
geomean 20.38n 30.97n -11.58%
Change-Id: I681029c0b442f9221c4429950626f06299a5cfe4
Reviewed-on: https://go-review.googlesource.com/c/go/+/614257
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
This change breaks out the debug.malloc codepaths into dedicated
functions, both for making mallocgc easier to read, and to reduce the
function's size (currently all that code is inlined and really doesn't
need to be).
This is a microoptimization that on its own changes very little, but
together with other optimizations and a breaking up of the various
malloc paths will matter all together ("death by a thousand cuts").
Change-Id: I30b3ab4a1f349ba85b4a1b5b2c399abcdfe4844f
Reviewed-on: https://go-review.googlesource.com/c/go/+/617879
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
These debug checks are very occasionally helpful, but they do cost real
time. The biggest issue seems to be the bloat of mallocgc due to the
"throw" paths. Overall, after some follow-ups, this change cuts about
1ns off of the mallocgc fast path.
This is a microoptimization that on its own changes very little, but
together with other optimizations and a breaking up of the various
malloc paths will matter all together ("death by a thousand cuts").
Change-Id: I07c4547ad724b9f94281320846677fb558957721
Reviewed-on: https://go-review.googlesource.com/c/go/+/617878
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
This change stops tracking assistG across malloc to reduce number of
slots the compiler must keep track of in mallocgc, which adds to
register pressure. It also makes the call to deductAssistCredit only
happen if the GC is running.
This is a microoptimization that on its own changes very little, but
together with other optimizations and a breaking up of the various
malloc paths will matter all together ("death by a thousand cuts").
Change-Id: I4cfac7f3e8e873ba66ff3b553072737a4707e2c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/617876
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
This is an allocator microoptimization. There's no reason to check
gcphase in general, since it's mostly for debugging anyway.
writeBarrier.enabled is set in all the same cases here, and we force one
fewer cache line (probably) to be touched during malloc.
Conceptually, it also makes a bit more sense. The allocate-black policy
is partly informed by the write barrier design.
Change-Id: Ia5ff593d64c29cf7f4d1bced3204056566444a98
Reviewed-on: https://go-review.googlesource.com/c/go/+/617875
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Checking whether the current allocation needs to be profiled is
currently branch-y and weirdly a lot of code. The branches are
generally predictable, but it's a surprising number of instructions.
Part of the problem is that MemProfileRate is just a global that can be
set at any time, so we need to load it and check certain settings
explicitly. In an ideal world, we would just always subtract from
nextSample and have a single branch to take the slow path if we
subtract below zero.
If MemProfileRate were a function, we could trash all the nextSample
values intentionally in each mcache. This would be slow, but
MemProfileRate changes rarely while the malloc hot path is well, hot.
Unfortunate...
Although this ideal world is, AFAICT, impossible, we can still get
close. If we cache the value of MemProfileRate in each mcache, then we
can force malloc to take the slow path whenever MemProfileRate changes.
This does require two additional loads, but crucially, these loads are
independent of everything else in mallocgc. Furthermore, the branch
dependent on those loads is incredibly predictable in practice.
This CL on its own has little-to-no impact on mallocgc. But this
codepath is going to be duplicated in several places in the next CL, so
it'll pay to simplify it. Also, we're very much trying to remedy a
death-by-a-thousand-cuts situation, and malloc is currently still kind
of a monster -- it will not help if mallocgc isn't really streamlined
itself.
Lastly, there's a nice property now that all nextSample values get
immediately re-sampled when MemProfileRate changes.
Change-Id: I6443d0cf9bd7861595584442b675ac1be8ea3455
Reviewed-on: https://go-review.googlesource.com/c/go/+/615815
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
This change brings back a minor optimization lost in the Go 1.22 cycle
wherein the 8-byte pointer-ful span class spans would have the pointer
bitmap written ahead of time in bulk, because there's only one possible
pattern.
│ before │ after │
│ sec/op │ sec/op vs base │
MallocTypeInfo8-4 25.13n ± 1% 23.59n ± 2% -6.15% (p=0.002 n=6)
Change-Id: I135b84bb1d5b7e678b841b56430930bc73c0a038
Reviewed-on: https://go-review.googlesource.com/c/go/+/614256
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
For whatever reason, span.heapBits is kind of slow. It accounts for
about a quarter of the cost of writeHeapBitsSmall, which is absurd. We
get a nice speed improvement for small allocations by eliminating this
call.
│ before │ after │
│ sec/op │ sec/op vs base │
MallocTypeInfo16-4 29.47n ± 1% 27.02n ± 1% -8.31% (p=0.002 n=6)
Change-Id: I6270e26902e5a9254cf1503fac81c3c799c59d6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/614255
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
src/runtime/testdata/testprogcgo/threadprof.go contains C code with a
variable called nullptr. This conflicts with the nullptr keyword in
the C23 revision of the C standard (showing up as gccgo test build
failures when updating GCC to use C23 by default when building C
code).
Rename that variable to nullpointer to avoid the clash with the
keyword (any other name that's not a keyword would work just as well).
Change-Id: Ida5ef371a3f856c611409884e185c3d5ded8e86c
GitHub-Last-Rev: 2ec464703b
GitHub-Pull-Request: golang/go#69927
Reviewed-on: https://go-review.googlesource.com/c/go/+/620955
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
This commit fixes the issue where tls testdata recordings made with the
newer version of the prerecorded tls conversation test harness, doesn't
end up capturing the final close notify message. The fix simply ensures
that the tls.Client closes before the recording of the conversation is
closed. The closing of the client connection directly is no longer
needed when updating the recording since it will be closed when the
tls.Client is closed.
Fixesgolang/go#69846
Change-Id: I93898de32abd89659a32ed240df6daea5aeaa7fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/620395
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Refactor TestOpenError to use relative paths in test cases,
in preparation for extending it to test os.Root.
Use a test temporary directory instead of system directory
with presumed-known contents.
Move the testcase type and case definitions inline with the test.
For #67002
Change-Id: Idc53dd9fcecf763d3e4eb3b4643032e3003d7ef4
Reviewed-on: https://go-review.googlesource.com/c/go/+/620157
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Use the new SwissTable-based map in internal/runtime/maps as the basis
for the runtime map when GOEXPERIMENT=swissmap.
Integration is complete enough to pass all.bash. Notable missing
features:
* Race integration / concurrent write detection
* Stack-allocated maps
* Specialized "fast" map variants
* Indirect key / elem
For #54766.
Cq-Include-Trybots: luci.golang.try:gotip-linux-ppc64_power10,gotip-linux-amd64-longtest-swissmap
Change-Id: Ie97b656b6d8e05c0403311ae08fef9f51756a639
Reviewed-on: https://go-review.googlesource.com/c/go/+/594596
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The PR is to add more details for the error, so that it would be easier to troubleshoot the cyclic imports error.
The change for the error looks like the following:
package cyclic-import-example
imports cyclic-import-example/packageA from /Users/personal/cyclic-import-example/main.go:4:5
imports cyclic-import-example/packageB from /Users/personal/cyclic-import-example/packageA/a.go:5:2
imports cyclic-import-example/packageA from /Users/personal/cyclic-import-example/packageB/bb.go:5:2: import cycle not allowed
Fixes#66078
Change-Id: I162cd348004bf4e4774b195f8355151c1bf0a652
GitHub-Last-Rev: c5a16256d1
GitHub-Pull-Request: golang/go#68337
Reviewed-on: https://go-review.googlesource.com/c/go/+/597035
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This is similar to CL 478196 and CL 477296,
but this is for -buildmode=shared.
When using "go install -buildmode=shared std",
because the gold linker is used by default on Linux arm64,
it will cause temporary paths to be included in libstd.so.
Based on the changes of CL 478196,
I speculate that this may also have issues on other platforms.
So, this change is for all platform.
Fixes#69464
Change-Id: I4493c82be030186e61aef597ea0e6f43bcf95a32
GitHub-Last-Rev: ee40cf81ac
GitHub-Pull-Request: golang/go#69394
Reviewed-on: https://go-review.googlesource.com/c/go/+/612396
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
syscall.Open param names are confusing, mainly because what should be
named flag is named mode and what should be named mode is named perm.
The name perm is used as synonym for mode in other places, so keep
it as is. Rename mode to flag to match the real meaning of the
parameter. Also, rename path to name for consistency with other
usage of the same parameter.
Change-Id: Ideed09839d80c0383584c2268afbb6cc09ffda8c
Reviewed-on: https://go-review.googlesource.com/c/go/+/619276
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
syscall.Open is the functions that maps Unix/Go flags into Windows
concepts. Part of the flag validation logic was still implemented
in os.OpenFile, move it to syscall.Open for consistency.
A nice side effect is that we don't have to translate the file name
twice in case of an access denied error.
Change-Id: I32c647a9a2a066277c78f53bacb45fb3036f6353
Reviewed-on: https://go-review.googlesource.com/c/go/+/619275
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
The current implementation of O_TRUNC in syscall.Open on Windows is
prone to TOCTOU issues, as it opens the file twice if the first open
detects that the file doesn't exist. The file could
be created in between the two open calls, leading to the creation
of a new file with the undesired readonly attribute.
This CL implements O_TRUNC by just calling CreateFile once without
taking O_TRUNCATE into account, and then using Ftruncate if O_TRUNC is
set to truncate the file.
Updates #38225.
Change-Id: Ic3ad1bab75c9a1c16f99c8c5bed867c5dbc3a23b
Reviewed-on: https://go-review.googlesource.com/c/go/+/618836
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
The syscall package is mostly frozen, but wasip1 file syscall
support was added to syscall and the Open and Openat
implementations overlap. Implement Openat in syscall for
overall simplicity.
We already have syscall.Openat for some platforms, so this
doesn't add any new functions to syscall.
For #67002
Change-Id: Ia34b12ef11fc7a3b7832e07b3546a760c23efe5b
Reviewed-on: https://go-review.googlesource.com/c/go/+/617378
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
The TestMkdirStickyUmask modifies the umask for testing purpose.
When run in parallel with TestCopyFS, this temporary umask change can cause TestCopyFS to create files with unintended permissions, leading to test failures.
This change removes the t.Parallel call in TestMkdirStickyUmask to prevent interference with TestCopyFS, ensuring it doesn't run concurrently with the other tests that require umask.
Fixes#69788
Change-Id: I9cf1da9f92283340ff85d2721781760a750d124c
Reviewed-on: https://go-review.googlesource.com/c/go/+/618055
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
This change replaces the MD5 hash used to identify coverage files with a
128-bit FNV-1a hash. This change is motivated by the fact that MD5
should only be used for legacy cryptographic purposes.
The 128-bit FNV-1a hash is sufficient for the purpose of identifying
coverage files, it having the same theoretical collision resistance as
MD5, but with the added benefit of being faster to compute.
Change-Id: I7b547ce2ea784f8f4071599a10fcb512b87ee469
Reviewed-on: https://go-review.googlesource.com/c/go/+/617360
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
OpenBSD system calls are mediated by libc anyway, and arc4random_buf()
is the preferred mechanism to obtain random bytes.
Also, rename NetBSD's function to reflect it's not actually calling
getentropy(3).
Cq-Include-Trybots: luci.golang.try:gotip-openbsd-amd64
Change-Id: Id1f3f7af16750537e2420bcf44b086de5854198c
Reviewed-on: https://go-review.googlesource.com/c/go/+/608395
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Bypass: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reintroduce the urandom fallback, but this time with a robust set of
tests all pointing guns at each other, including a seccomp'd respawn
simulating the lack of getrandom, to make sure the fallback both works
and is never hit unexpectedly.
Unlike the Go 1.23 fallback, the new one only triggers on ENOSYS (which
is cached by unix.GetRandom) and doesn't handle the EAGAIN errors we
never got an explanation for.
We still crash the program from Read if we have to go to /dev/urandom
and we fail to open it.
For #67001
Updates #66821
Tested on legacy SlowBots (without plan9 and illumos, which don't work):
TRY=aix-ppc64,dragonfly-amd64,freebsd-amd64,freebsd-386,netbsd-amd64
Cq-Include-Trybots: luci.golang.try:gotip-darwin-amd64_14,gotip-solaris-amd64,gotip-js-wasm,gotip-wasip1-wasm_wasmtime,gotip-wasip1-wasm_wazero,gotip-windows-amd64,gotip-windows-386,gotip-linux-386,gotip-linux-amd64-longtest-race,gotip-linux-amd64-boringcrypto
Change-Id: Idecc96a18cd6363087f5b2a4671c6fd1c41a3b0e
Reviewed-on: https://go-review.googlesource.com/c/go/+/608175
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The fallback was reachable on
- Linux, where starting in Go 1.24 we require a kernel with
getrandom(2), see #67001.
- FreeBSD, which added getrandom(2) in FreeBSD 12.0, which we
require since Go 1.19.
- OpenBSD, which added getentropy(2) in OpenBSD 5.6, and we only
support the latest version.
- DragonFly BSD, which has getrandom(2) and where we support only
the latest version.
- NetBSD, where we switched to kern.arandom in CL 511036, available
since NetBSD 4.0.
- illumos, which has getrandom(2). (Supported versions unclear.)
- Solaris, which had getrandom(2) at least since Oracle
Solaris 11.4.
- AIX, which... ugh, fine, but that code is now in rand_aix.go.
At the end of the day the platform-specific code is just a global
func(b []byte) error, so simplified the package around that assumption.
This also includes the following change, which used to be a separate CL.
crypto/rand: improve getrandom batching and retry logic
The previous logic assumed getrandom never returned short, and then
applied stricter-than-necessary batch size limits, presumably to
avoid short returns.
This was still not sufficient because above 256 bytes getrandom(2)
can be interrupted by a signal and return short *or* it can simply
return EINTR if the pool is not initialized (regardless of buffer
size).
https://man.archlinux.org/man/getrandom.2#Interruption_by_a_signal_handler
Whether this ever failed in practice is unknown: it would have been
masked by the /dev/urandom fallback before.
Instead, we apply buffer size limits only where necessary (really,
only Solaris in practice and FreeBSD in theory) and then handle
gracefully short returns and EINTR.
Change-Id: I8677b457aab68a8fb6137a3b43538efc62eb7c93
It turns out that we now know that large getrandom calls *did* fail in
practice, falling back on /dev/urandom, because when we removed the
fallback TestBidiStreamReverseProxy with its 4KiB read started failing.
https://cr-buildbucket.appspot.com/build/8740779846954406033
For #66821
Change-Id: Iaca62997604f326501a51401cdc2659c2790ff22
Reviewed-on: https://go-review.googlesource.com/c/go/+/602495
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
During type inference, when comparing type parameters against their
constraints, if a type argument is completely known it must implement
its constraint. In this case, always unify the type argument's methods
against the constraint methods, if any.
Before this CL, this step was only attempted if the constraint had no
core type. That left information unused which led to type inference
failures where it should have succeeded.
Fixes#66751.
Change-Id: I71e96b71258624212186cf17ec47e67a589817b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/617896
Reviewed-by: Tim King <taking@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Inside Google we have seen issues with QEMU user mode failing to wake a
parent waitid when this child exits with SYS_EXIT. This bug appears to
not affect SYS_EXIT_GROUP.
It is currently unclear if this is a general QEMU or specific to
Google's configuration, but SYS_EXIT and SYS_EXIT_GROUP are semantically
equivalent here, so we can use the latter here in case this is a general
QEMU bug.
For #68976.
Change-Id: I34e51088c9a6b7493a060e2a719a3cc4a3d54aa0
Reviewed-on: https://go-review.googlesource.com/c/go/+/617417
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
I've done some more testing of the new isSending field.
I'm not able to get more than 2 bits set. That said,
with this change it's significantly less likely to have even
2 bits set. The idea here is to clear the bit before possibly
locking the channel we are sending the value on, thus avoiding
some delay and some serialization.
For #69312
Change-Id: I8b5f167f162bbcbcbf7ea47305967f349b62b0f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/617497
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
While working on CL 611241 and CL 616375, I introduced a bug that wasn't
caught by any test. CL 611241 added more inline expansion at sample time
for block/mutex profile stacks collected via frame pointer unwinding.
CL 616375 then changed how inline expansion for those stacks is done at
reporting time. So some frames passed through multiple rounds of inline
expansion, and this lead to duplicate stack frames in some cases. The
stacks from TestBlockMutexProfileInlineExpansion looked like
sync.(*Mutex).Unlock
runtime/pprof.inlineF
runtime/pprof.inlineE
runtime/pprof.inlineD
runtime/pprof.inlineD
runtime.goexit
after those two CLs, and in particular after CL 616375. Note the extra
inlineD frame. The test didn't catch that since it was only looking for
a few frames in the stacks rather than checking the entire stacks.
This CL makes that test stricter by checking the entire expected stacks
rather than just a portion of the stacks.
Change-Id: I0acc739d826586e9a63a081bb98ef512d72cdc9a
Reviewed-on: https://go-review.googlesource.com/c/go/+/617235
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
I noticed in pprof that acquirem() was a bit of a hotspot. It turns out
that we can use the same trick that runtime.rand() does, and only
acquirem if we're doing something non-nosplit -- in this case, getting a
new state -- but otherwise just do getg().m, which is safe because we're
inside runtime and don't call split functions.
cpu: 11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz
│ sec/op │ sec/op vs base │
ParallelGetRandom-16 2.651n ± 4% 2.416n ± 7% -8.87% (p=0.001 n=10)
│ B/s │ B/s vs base │
ParallelGetRandom-16 1.406Gi ± 4% 1.542Gi ± 6% +9.72% (p=0.001 n=10)
Change-Id: Iae075f4e298b923e499cd01adfabacab725a8684
Reviewed-on: https://go-review.googlesource.com/c/go/+/616738
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The existing prose for struct identity did only require that two structs
"have the same sequence of fields, and if corresponding fields have the
same names, and identical types, and identical tags" for the structs to
be identical.
The implementation (forever) has also required that two corresponding
fields are either both embedded or not embedded. This is arguably part
of a struct's structure but is not explicitly specified.
This CL makes a minor change to the prose to address that.
Fixes#69472.
Change-Id: Ifa4ca69717986675642a09d03ce683ba8235efcb
Reviewed-on: https://go-review.googlesource.com/c/go/+/616697
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Bypass: Robert Griesemer <gri@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Before this patch, the documentation of Dialer.Control and
ListenConfig.Control did not specify what networks would be
passed to the Control function other than the "tcp" case.
It was thus challenging to use the Control function to filter
out certain networks. This patch documents all known networks.
Fixes#69693
Change-Id: I2ab10d68c4e4fac66d51d2cc232f02cf3b305e89
Reviewed-on: https://go-review.googlesource.com/c/go/+/617055
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
CL 615915 simplified test for issue 69434, using gcflags maymorestack to
force stack moving, making program failed with invalid stack pointer.
However, it seems that this maymorestack is broken on riscv64. At least
gotip-linux-riscv64 is currently broken.
This CL fixes this problem by using the initial approach, growing stack
size big enough to force stack moving.
Updates #69434Fixes#69714
Cq-Include-Trybots: luci.golang.try:gotip-linux-riscv64
Change-Id: I95255fba884a200f75bcda34d58e9717e4a952ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/616698
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Over the years we've had various bugs in pprof stack handling resulting
in appendLocsForStack crashing because stk is too short for a cached
location. i.e., the cached location claims several inlined frames. Those
should always appear together in stk. If some frames are missing from
stk, appendLocsForStack.
If we find this case, replace the slice out of bounds panic with an
explicit panic that contains more context.
Change-Id: I52725a689baf42b8db627ce3e1bc6c654ef245d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/617135
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This change adds a new environment variable GOAUTH which takes a semicolon-separated list of commands to run for authentication during go-import resolution and HTTPS module mirror protocol interactions.
This CL only supports netrc and off. Future CLs to follow will extend support to git and a custom authenticator command.
For #26232
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Change-Id: I6cfa4c89fd27a7a4e7d25c8713d191dc82b7e28a
Reviewed-on: https://go-review.googlesource.com/c/go/+/605256
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
Using a different build of Go (specifically, a different GOROOT) to
maintain the vendor directory doesn't always reproduce the same results.
This can result in unknowingly creating a vendor directory that isn't
able to build Go.
Add a note to README.vendor to point this out. Specifically, mention
that a mismatched GOROOT is an issue, and recommend using a fresh build
of Go to maintain the vendor directory.
Updates #69235
Change-Id: Id80c7607bf28bd76e43e1fdc672811c50f2bffb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/616815
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Sam Thanawalla <samthanawalla@google.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
This commit amends package errors' documentation to include a reference
to the https://go.dev/blog/go1.13-errors blog article. The motivation
is multi-fold, but chiefly the article includes good information about
error philosophy (e.g., when to wrap), and developers who have come to
Go in the intervening five years are likely not have seen this article
at all given the nature of blog publishing and post fanfare. The
material deserves a promotion in visibility.
Change-Id: Ia6f8307784521dd59de3a3d638dbc0a7fcd445e6
GitHub-Last-Rev: 20980dd507
GitHub-Pull-Request: golang/go#69698
Reviewed-on: https://go-review.googlesource.com/c/go/+/616341
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
The source code quoted tailscale's development fork, which is only a
development fork. The canonical github url is actually
github.com/wireguard/wireguard-go, but that's really just a mirror of
git.zx2c4.com/wireguard-go, and in any case, the proper go package name
is golang.zx2c4.com/wireguard, so just use that.
Change-Id: Ifa63c1c538989b3fcebcf06d1c238469bc73724d
Reviewed-on: https://go-review.googlesource.com/c/go/+/616736
Auto-Submit: Jason Donenfeld <Jason@zx2c4.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
This prevents false sharing, which makes a large difference on machines
with several NUMA nodes, such as this dual socket server:
cpu: Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz
│ sec/op │ sec/op vs base │
ParallelGetRandom-128 0.7944n ± 5% 0.4503n ± 0% -43.31% (p=0.000 n=10)
│ B/s │ B/s vs base │
ParallelGetRandom-128 4.690Gi ± 5% 8.272Gi ± 0% +76.38% (p=0.000 n=10)
Change-Id: Id4421e9a4c190b38aff0be4c59e9067b0a38ccd7
Reviewed-on: https://go-review.googlesource.com/c/go/+/616535
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Jason Donenfeld <Jason@zx2c4.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
This CL adds a local only VCS lookup for Mercurial.
It fixes a bug in pkg.go by passing in the repo directory to
the LookupLocal function instead of the module directory. It could be
the case that a binary is built in a subdirectory of the repo.
For: #50603
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Change-Id: Ic36b5a361a8ba3b0ba1a6968cde5f5263c9c8dd0
Reviewed-on: https://go-review.googlesource.com/c/go/+/609155
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Linux 6.11 supports calling getrandom() from the vDSO. It operates on a
thread-local opaque state allocated with mmap using flags specified by
the vDSO.
Opaque states are allocated in chunks, ideally ncpu at a time as a hint,
rounding up to as many fit in a complete page. On first use, a state is
assigned to an m, which owns that state, until the m exits, at which
point it is given back to the pool.
Performance appears to be quite good:
│ sec/op │ sec/op vs base │
Read/4-16 222.45n ± 3% 27.13n ± 6% -87.80% (p=0.000 n=10)
│ B/s │ B/s vs base │
Read/4-16 17.15Mi ± 3% 140.61Mi ± 6% +719.82% (p=0.000 n=10)
Fixes#69577.
Change-Id: Ib6f44e8f2f3940c94d970eaada0eb566ec297dc7
Reviewed-on: https://go-review.googlesource.com/c/go/+/614835
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Auto-Submit: Jason Donenfeld <Jason@zx2c4.com>
Reviewed-by: Paul Murphy <murp@ibm.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
There were a few Mercurial command line uses that could cause the wrong
data to be used:
* The log command needs '-r.' to specify the currently checked out commit
* HGPLAIN is needed to disable optional output on commands
* '-S' is needed to for the 'status' command to recurse into any subrepos
The most likely issue to be seen here was the use of '-l1' instead of
'-r.', which prints the most recent commit instead of the current checkout.
Since tagging in Mercurial creates a new commit, this basically means the
data was wrong for every tagged build.
This also adds an hgrc config file to the test, with config options to
keep the time and author values fixed. It's what's used in the Mercurial
test harness to keep the commit hashes stable, and allows the tests here to
also match the time and the revision ID, to prevent regressing.
Fixes#63532
Change-Id: I5b9971ce87c83431ec77e4a002bdc33fcf393856
GitHub-Last-Rev: 62c9db0a28
GitHub-Pull-Request: golang/go#63557
Reviewed-on: https://go-review.googlesource.com/c/go/+/535377
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Auto-Submit: Sam Thanawalla <samthanawalla@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
This moves the implementation of Scope.LookupParent into
environment.lookupScope where it encapsulates the use of
the current environment's position. At least in types2,
that position can be removed, because it is never set.
With this, the type checker doesn't rely on position
information anymore for looking up objects during type
checking.
LookupParent is still called from tests and some go/types
code.
Updates #69673.
Change-Id: I7159ba95b71cf33cc3b16058aa19327e166224b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/616337
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
In extremely rare cases of receiver base types of the form
C.foo where C refers to an `import "C"`, we needed Scope.Contains
to lookup the file scope containing the "C" import.
Replace the position-dependent Scope.Contains with an explicit
scope search that doesn't require a position.
Also, make the surrounding code match more closely between
go/types and types2.
Change-Id: Ic007108928dd8b382a06e2bbf09ef8bd6bd0ff36
Reviewed-on: https://go-review.googlesource.com/c/go/+/616256
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
On Arch Linux with gdb version 15.1, the test for TestGdbAutotmpTypes print
the following output,
----
~/src/go/src/runtime
$ go test -run=TestGdbAutotmpTypes -v
=== RUN TestGdbAutotmpTypes
=== PAUSE TestGdbAutotmpTypes
=== CONT TestGdbAutotmpTypes
runtime-gdb_test.go:78: gdb version 15.1
runtime-gdb_test.go:570: gdb output:
Loading Go Runtime support.
Target 'exec' cannot support this command.
Breakpoint 1 at 0x46e416: file /tmp/TestGdbAutotmpTypes750485513/001/main.go, line 8.
This GDB supports auto-downloading debuginfo from the following URLs:
<https://debuginfod.archlinux.org>
Enable debuginfod for this session? (y or [n]) [answered N; input not from terminal]
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
[New LWP 355373]
[New LWP 355374]
[New LWP 355375]
[New LWP 355376]
Thread 1 "a.exe" hit Breakpoint 1, main.main () at /tmp/TestGdbAutotmpTypes750485513/001/main.go:8
8 func main() {
9 var iface interface{} = map[string]astruct{}
All types matching regular expression "astruct":
File runtime:
[]main.astruct
bucket<string,main.astruct>
hash<string,main.astruct>
main.astruct
typedef hash<string,main.astruct> * map[string]main.astruct;
typedef noalg.[8]main.astruct noalg.[8]main.astruct;
noalg.map.bucket[string]main.astruct
runtime-gdb_test.go:587: could not find []main.astruct; in 'info typrs astruct' output
!!! FAIL
exit status 1
FAIL runtime 0.273s
$
----
In the back trace for "File runtime", each output lines does not end with
";" anymore, while in test we check the string with it.
While at it, print the expected string with "%q" instead of "%s" for
better error message.
Fixes#67089
Change-Id: If6019ee68c0d8e495c920f98568741462c7d0fd0
Reviewed-on: https://go-review.googlesource.com/c/go/+/598135
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
The timer code is careful to ensure that if stop/reset is called
while a timer is being run, we cancel the run. However, the code
failed to ensure that in that case stop/reset returned true,
meaning that the timer had been stopped. In the racing case
stop/reset could see that t.when had been set to zero,
and return false, even though the timer had not and never would fire.
Fix this by tracking whether a timer run is in progress,
and using that to reliably detect that the run was cancelled,
meaning that stop/reset should return true.
Fixes#69312
Change-Id: I78e870063eb96650638f12c056e32c931417c84a
Reviewed-on: https://go-review.googlesource.com/c/go/+/611496
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
When using the -x or -n option, we were printing the external
linker error messages from producing the dynimport file.
This was confusing because those linker errors are unimportant and
ignored; only the linker exit status matters, and failure doesn't
drop the build.
Change cmd/go -x to not print the error messages, and to instead
print the linker command line with a notation of whether the
link succeeded or failed.
Fixes#68743
Change-Id: Ie3cc58d2d6a7d33d7baa6f1273b4fb5a7deee7e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/615916
Reviewed-by: Michael Matloob <matloob@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The old calculation just looked whether PC was within a page of a vDSO
symbol. This doesn't work because the vDSO .text might span two whole
pages, with trampolines and such redirecting PC around between them.
This manifests itself with the new vDSO getrandom() function, where on
PowerPC, the trampoline is quite far away from the actual C function it
jumps into. The effect is that the signal handler doesn't know it's
interrupting a vDSO call and forgets to restore g to R30, resulting in a
crash.
Fix this by storing the start and end of the LOAD section from the
program headers. We could be more specific and parse out the .text
section, but PT_LOAD is good enough and appears to work well.
Change-Id: I3cf16955177eedb51e28b3b1a0191b32c3327a42
Reviewed-on: https://go-review.googlesource.com/c/go/+/616015
Auto-Submit: Jason Donenfeld <Jason@zx2c4.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Previously, the Checker.allowVersion method would use a token.Pos
to try to infer which file of the current package the checker
was "in". This proved fragile when type-checking syntax that
had been modified or synthesized and whose positions were invalid.
This change records the effective version in the checker state
(checker.environment.version). Just like other aspects of the
environment, the version changes from one file to the next
and must be saved and restored with each check.later closure.
Similarly, declInfo captures and temporarily reinstates
the effective version when checking each object.
+ Test of position independence in go/types and types2
+ Test of panic avoidance in go/types
Fixesgolang/go#69477Fixesgolang/go#69338
Change-Id: Ic06f9d88151c64a4f7848f8942d08e3c312cdd6f
Reviewed-on: https://go-review.googlesource.com/c/go/+/613735
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The SSA backend currently only handle struct with up to 4 fields. Thus,
there are different operations corresponding to number fields of the
struct.
This CL generalizes these with just one OpStructMake, allow struct types
with arbitrary number of fields.
However, the ssa.MaxStruct is still kept as-is, and future CL will
increase this value to optimize large structs.
Updates #24416
Change-Id: I192ffbea881186693584476b5639394e79be45c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/611075
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
The SignPSS hash override happened after the boringcrypto block, meaning
if a boringcrypto user passed a hash in the PSSOptions which did not
match the hash argument, it wouldn't be overriden. This change moves the
check above the boring block to make sure the override is honored.
Thanks to Quim Muntal of Microsoft for spotting this issue.
Change-Id: I05082a84ccb1863798ac6eae7a15cf4d1e59f12d
Reviewed-on: https://go-review.googlesource.com/c/go/+/614276
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: David Chase <drchase@google.com>
Fix a regression introduced in CL 572396 causing goroutine stacks not
getting null terminated.
This bug impacts callers that reuse the []StackRecord slice for multiple
calls to GoroutineProfile. See https://github.com/felixge/fgprof/issues/33
for an example of the problem.
Add a test case to prevent similar regressions in the future. Use null
padding instead of null termination to be consistent with other profile
types and because it's less code to implement. Also fix the
ThreadCreateProfile code path.
Fixes#69243
Change-Id: I0b9414f6c694c304bc03a5682586f619e9bf0588
Reviewed-on: https://go-review.googlesource.com/c/go/+/609815
Reviewed-by: Tim King <taking@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Fix a regression introduced in CL 598515 causing runtime.MutexProfile
stack traces to omit their root frames.
In most cases this was merely causing the `runtime.goexit` frame to go
missing. But in the case of runtime._LostContendedRuntimeLock, an empty
stack trace was being produced.
Add a test that catches this regression by checking for a stack trace
with the `runtime.goexit` frame.
Also fix a separate problem in expandFrame that could cause
out-of-bounds panics when profstackdepth is set to a value below 32.
There is no test for this fix because profstackdepth can't be changed at
runtime right now.
Fixes#69335
Change-Id: I1600fe62548ea84981df0916d25072c3ddf1ea1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/611615
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Nick Ripley <nick.ripley@datadoghq.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Preparation for removing the existing non-standard iterators
(is, underIs). Note that we cannot use typeset iterators in
range-over-func because the bootstrap compiler doesn't have
access to it yet.
While at it, move underIs from expr.go to under.go
and adjust some doc strings in typset.go to match
prevailing style in that file.
Change-Id: Iecd014eeb5b3fca56a807381c148c5f7a29bfb78
Reviewed-on: https://go-review.googlesource.com/c/go/+/614239
Reviewed-by: Tim King <taking@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Commit-Queue: Robert Griesemer <gri@google.com>
Like for Named types, print type parameters for Alias types.
Add test case for Alias object string to existing test.
To make the test work, factor out the mechanism to set
GOEXPERIMENT=aliastypeparams at test time and use it
for this test as well.
No test case for un-instantiated generic type Alias type
string: there's no existing test framework, the code is
identical as for Named types, and these strings only appear
in tracing output. Tested manually.
Change-Id: I476d04d0b6a7c18b79be1d34a9e3e072941df83f
Reviewed-on: https://go-review.googlesource.com/c/go/+/615195
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Tim King <taking@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Since we added a local context to git lookups, we need to be more
careful about fetching from remote.
We should not fetch when we are stamping a binary because that could
slow down builds.
For #50603
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Change-Id: I81a719b7609e8d30b32ffb3c12a05074c5fd0c22
Reviewed-on: https://go-review.googlesource.com/c/go/+/611916
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
They are constant time, but some constants were incorrect. This
resulting in reading beyond the tables.
I've added linux specific tests which verify these functions are not
reading beyond the limits of their table.
Thank you Sun Yimin, @emmansun for catching this bug and suggesting
corrected constants.
Fixes#69080
Cq-Include-Trybots: luci.golang.try:gotip-linux-ppc64_power10,gotip-linux-ppc64_power8,gotip-linux-ppc64le_power10,gotip-linux-ppc64le_power8,gotip-linux-ppc64le_power9
Change-Id: Id37e0e22b2278ea20adaa1c84cbb32c3f20d4cf7
Reviewed-on: https://go-review.googlesource.com/c/go/+/608816
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Archana Ravindar <aravinda@redhat.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The Frames function is almost an iter.Seq, except for its bool return
value.
Since none of the callers in the Go tree rely on the bool, we can remove
it. However, doing so might still obscure the intended usage as an iterator.
This refactor changes the API to return iter.Seq, making the intended
usage explicit. Refactoring the existing callers to take advantage of
the new interface will be done in a follow-up CL.
Change-Id: I03e4d6d762910e418cc37d59a6c519eb7f39b3b0
Reviewed-on: https://go-review.googlesource.com/c/go/+/608855
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TestScript is very slow on Plan 9 because this test
is particularly i/o intensive.
This is leading the plan9/386 and plan9/amd64 builders
to time out. This test was already skipped on plan9/arm
because arm is part of the "slow architectures" list.
This change skips TestScript on Plan 9 on short mode.
Change-Id: I3e68046dac825cd14fa8daca601c492cf11c6fff
Reviewed-on: https://go-review.googlesource.com/c/go/+/614855
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: David du Colombier <0intro@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Bypass: Ian Lance Taylor <iant@golang.org>
With b.Loop() in place, the time measurement of loop scaling could be improved to be tighter. By identifying the first call to b.Loop(), we can avoid measuring the expensive ramp-up time by reset the timer tightly before the loop starts. The remaining loop scaling logic of b.N style loop is largely reused.
For #61515.
Change-Id: Ia7b8f0a8838f57c00ac6c5ef779d86f8d713c9b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/612835
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
It makes use of the hiter structure which matches runtime.hiter's.
This change mainly improves the performance of Next method of MapIter.
goos: darwin
goarch: arm64
pkg: reflect
cpu: Apple M2
│ ./old.txt │ ./new.txt │
│ sec/op │ sec/op vs base │
MapIterNext-8 61.95n ± 0% 54.95n ± 0% -11.28% (p=0.000 n=10)
for the change of `test/escape_reflect.go`:
removing mapiterkey, mapiterelem would cause leaking MapIter content
when calling SetIterKey and SetIterValue,
and this may cause map bucket to be allocated on heap instead of stack.
Reproduce:
```
{
m := map[int]int{1: 2} // escapes to heap after this change
it := reflect.ValueOf(m).MapRange()
it.Next()
var k, v int
reflect.ValueOf(&k).Elem().SetIterKey(it)
reflect.ValueOf(&v).Elem().SetIterValue(it)
println(k, v)
}
```
This CL would not introduce abi.NoEscape to fix this. It may need futher
optimization and tests on hiter field usage and its escape analysis.
Fixes#69416
Change-Id: Ibaa33bcf86228070b4a505b9512680791aa59f04
Reviewed-on: https://go-review.googlesource.com/c/go/+/612616
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL introduces the ability to print information about the toolchain switch used in the
go command, controlled by the `toolchaintrace` setting. This setting defaults to `toolchaintrace=0`,
meaning no information is printed. Setting it to `toolchaintrace=1` will cause the go command
to print a message indicating the toolchain used and where it was found.
Fixes: #63939
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Change-Id: Idc58e3d5bc76573aa48e1f7df352caa13004c25e
Reviewed-on: https://go-review.googlesource.com/c/go/+/610235
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL special-case User.GroupIds to get the group IDs from the user's
token when the user is the current user.
This approach is more efficient than calling NetUserGetLocalGroups.
It is also more reliable for users joined to an Active Directory domain,
where NetUserGetLocalGroups is likely to fail.
Updates #26041.
Fixes#62712.
Cq-Include-Trybots: luci.golang.try:gotip-windows-arm64
Change-Id: If7c30287192872077b98a514bd6346dbd1a64fb4
Reviewed-on: https://go-review.googlesource.com/c/go/+/611116
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Tim King <taking@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Before this change, test binaries didn't have build info populated on them
unless they were tests for package main. Now we generate them for all
test binaries so that they can be inspected like other binaries.
We don't need to add the default GODEBUG in printLinkerConfig because it
will now always be present on the build info, and when build info is
present we use it to generate the hash.
Fixes#33976
Change-Id: Ib4f51c04f87df3c7f2f21c400ab446e70d66a101
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/613096
Auto-Submit: Michael Matloob <matloob@golang.org>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
CL 28490 speeded up non-ASCII rune decoding, and ASCII rune is also
decoded faster now.
Benchmark using:
perflock -governor 70% go test -run=NONE -bench=BenchmarkRuneCountInString -count=10
Result:
name old time/op new time/op delta
RuneCountInStringTenASCIIChars-8 10.2ns ± 0% 7.1ns ± 1% -30.53% (p=0.000 n=8+9)
RuneCountInStringTenJapaneseChars-8 49.3ns ± 2% 38.5ns ± 2% -21.84% (p=0.000 n=8+8)
Fixes#13162
Change-Id: Ifb01f3799c5c93e7f7c7af13a95becfde85ae807
Reviewed-on: https://go-review.googlesource.com/c/go/+/612617
Reviewed-by: Tim King <taking@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Tim King <taking@google.com>
clone(CLONE_PIDFD) was added in Linux 5.2 and pidfd_open was added in
Linux 5.3. Thus our feature check for pidfd_open should be sufficient to
ensure that clone(CLONE_PIDFD) works.
Unfortuantely, some alternative Linux implementations may not follow
this strict ordering. For example, QEMU 7.2 (Dec 2022) added pidfd_open,
but clone(CLONE_PIDFD) was only added in QEMU 8.0 (Apr 2023).
Debian bookworm provides QEMU 7.2 by default.
Fixes#69259.
Change-Id: Ie3f3dc51f0cd76944871bf98690abf59f68fd7bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/592078
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
The ANDN, ORN and XNOR RISC-V Zbb extension instructions are easily
synthesised. Make them always available by adding support to the
riscv64 assembler so that we either emit two instruction sequences,
or a single instruction, when permitted by the GORISCV64 profile.
This means that these instructions can be used unconditionally,
simplifying compiler rewrite rules, codegen tests and manually
written assembly.
Around 180 instructions are removed from the Go binary on riscv64
when built with rva22u64.
Change-Id: Ib2d90f2593a306530dc0ed08a981acde4d01be20
Reviewed-on: https://go-review.googlesource.com/c/go/+/611895
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
Reviewed-by: Tim King <taking@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Previously we expected the default GODEBUG that's embedded in the binary
to be taken into account for build actionIDs through the build info. The
build info contains the default GODEBUG for a package main, and then
that build info is used to generate the action id. But tests of packages
other than main do not have buildinfo set on them. So the default
GODEBUG isn't taken into account in the action id for those tests.
Explicitly include GODEBUG when generating all link actions' action ids
to make sure it's always present.
Fixes#69203
Change-Id: Ifbc58482454ecfb51ba09cfcff02972cac3270c1
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/610875
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
The function resultsToWasmFields was originally for only
wasmimport. I adopted it for wasmexport as well, but forgot to
update a few places that were wasmimport-specific. This leads to
compiler panic if an invalid result type is passed, and also
unsafe.Pointer not actually supported. This CL fixes it.
Updates #65199.
Change-Id: I9bbd7154b70422504994840ff541c39ee596ee8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/611315
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Achille Roussel <achille.roussel@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
asm_riscv64.h will be used to define macros for each riscv64
extension that is not part of the rva20u64 base profile but that the
_riscv64.s assembly files are allowed to use because the user has
specified a more capable profile in the GORISCV64 variable. This will
allow us, for example, to test for the hasZba macro in those assembly
files instead of the GORISCV64_rva22u64 macro before using a Zba
instruction. This is important as it means that in the future when
we add support for new profiles that support Zba, e.g., rva23u64,
we only need to update asm_riscv64.h to indicate rva23u64 supports
Zba. We will not need to update every assembly language file that
already uses Zba instructions.
Updates #61476
Change-Id: I83abfeb20d08a87ac8ea88f4d8a93437f0631353
Reviewed-on: https://go-review.googlesource.com/c/go/+/608255
Auto-Submit: Tim King <taking@google.com>
Reviewed-by: Tim King <taking@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This field is present during the initial development of generic support
inside compiler, and indicating whether a type is fully instantiated is
the solely purpose at this moment. Further, its name is also confused,
and there have been a TODO to chose a better name for it.
Instead, just using a bit to track whether a type is fully instantiated,
then this rparams field can be removed to simplify the code.
Change-Id: Ia29c6dd5792487c440b83b0f3b77bd60917c2019
Reviewed-on: https://go-review.googlesource.com/c/go/+/611255
Reviewed-by: Tim King <taking@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
Use OTAILCALL in wrapper if the receiver and method are both pointers and it is
not going to be inlined, similar to how it is done in reflectdata.methodWrapper.
Currently tail call may be used for functions with identical argument types.
This change updates wrappers where both wrapper and the wrapped method's
receiver are pointers. In this case, we have the same signature for the
wrapper and the wrapped method (modulo the receiver's pointed-to types),
and do not need any local variables in the generated wrapper (on stack)
because the arguments are immediately passed to the wrapped method in place
(without need to move some value passed to other register or to change any
argument/return passed through stack). Thus, the wrapper does not need its
own stack frame.
This applies to promoted methods, e.g. when we have some struct type U with
an embedded type *T and construct a wrapper like
func (recv *U) M(arg int) bool { return recv.T.M(i) }
See also test/abi/method_wrapper.go for a running example.
Code size difference measured with this change (tried for x86_64):
etcd binary:
.text section size: 21472251 -> 21432350 (0.2%)
total binary size: 32226640 -> 32191136 (0.1%)
compile binary:
.text section size: 17419073 -> 17413929 (0.03%)
total binary size: 26744743 -> 26737567 (0.03%)
Change-Id: I9bbe730568f6def21a8e61118a6b6f503d98049c
Reviewed-on: https://go-review.googlesource.com/c/go/+/578235
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
The telemetry script test checks for the existence of telemetry data as
a baseline before checking that the act of setting telemtetry to off
while in local mode doesn't produce telemetry data. Of course, when
we're running on platforms where telemetry is not supported, telemetry
data won't be produced on disk either way. Only check for the existence
of telemetry data on supported platforms.
For #69269
Change-Id: I3a06bbc3d3ca0cf0203b84883f632ecfd9445aae
Reviewed-on: https://go-review.googlesource.com/c/go/+/611876
Reviewed-by: Robert Findley <rfindley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Matloob <matloob@golang.org>
There is the expectation that if 'go telemetry off' is run with a clean
home directory that no counter files are written. But we were writing
counters in that case because the act of turning telemetry off was done
after the act of opening the counter files, so the counter files were
opened depending on what the previous mode was. Add a special check that
the command is not 'go telemetry off' before opening counter files.
Fixes#69269
Change-Id: I8fc37dfe24ec7f454676cc2fdd4b79a13a7aba9d
Reviewed-on: https://go-review.googlesource.com/c/go/+/611456
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Auto-Submit: Michael Matloob <matloob@golang.org>
On Android, faccessat2 syscall (which supports flags like AT_EACCESS) is
not allowed, so syscall.Faccessat tries to emulate AT_EACCESS check in
userspace using os.Stat, os.Geteuid etc.
Also, according to [1],
> Android doesn't have setuid programs, and never runs code with euid!=uid.
This means on Android the proper AT_EACCESS check is neither possible
nor really needed.
Let's skip the syscall.Faccessat userspace emulation of AT_EACCESS
check and return ENOSYS, so the callers can use a fallback.
This should speed up exec.LookPath on Android.
[1]: 508b2f6e5c/libc/bionic/faccessat.cpp (50)
Change-Id: If7b529fa314480b70e9ae9cdd8c7ce82cd55d233
Reviewed-on: https://go-review.googlesource.com/c/go/+/611298
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Eaccess, initially added by CL 414824 for linux only, was later
implemented for freebsd (CL 531155), netbsd (CL 531876), dragonfly
(CL 532675), openbsd (CL 538836), and darwin (CL 579976).
The only unix platforms which lack Eaccess are Solaris/Illumos and AIX.
For AIX, syscall.Faccessat is already available, the only missing piece
was AT_EACCESS constant. Let's take it from [1], which, judging by a few
other known AT_ constants, appears to be accurate.
For Solaris, wire the faccessat using the same logic as in the syscall
package.
Now, when we have faccessat for every unix, we can drop eaccess_other.go
and consolidate Eaccess implementations to use faccessat.
[1]: https://github.com/rust-lang/libc/blob/main/src/unix/aix/mod.rs
Change-Id: I7e1b90dedc5d8174235d3a79d5c662f3dcb909c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/611295
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Kirill Kolyshkin <kolyshkin@gmail.com>
Generally, the parser strips (i.e., does not record in the syntax tree)
unnecessary parentheses. Specifically, given a type parameter list of
the form
[P (C),]
it records it as
[P C]
and then no comma is required when printing. However it did only strip
one level of parentheses, and
[P ((C)),]
made it through, causing a panic when printing. Somewhat related,
the printer stripped parentheses around constraints as well.
This CL implements a more consistent behavior:
1) The parser strips all parentheses around constraints. For testing
purposes, a local flag (keep_parens) can be set to retain the
parentheses.
2) The printer code now correctly intruces a comma if parentheses
are present (e.g., when testing with keep_parens). This case does
not occur in normal operation.
3) The printer does not strip parentheses around constraints since
the parser does it already.
For #69206.
Change-Id: I974a800265625e8daf9477faa9ee4dd74dbd17ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/610758
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
This change makes sure that we do not format comments
as doc comments inside of a declaration and makes the
go doc formatter idempotent:
Previously:
// test comment
//go:directive2
// test comment
func main() {
}
was formatted to:
// test comment
//go:directive2
// test comment
func main() {
}
after another formatting, it got formatted with doc rules into:
// test comment
// test comment
//
//go:directive2
func main() {
}
With this change it gets directly to the correct form (last one).
Change-Id: Id7d8f03e43474357cd714e0672e886652c3fce86
GitHub-Last-Rev: 9833b87536
GitHub-Pull-Request: golang/go#69134
Reviewed-on: https://go-review.googlesource.com/c/go/+/609077
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
When testing with PWD set, it's possible for the stat of PWD to fail
with ENAMETOOLONG, and for syscall.Getwd to fail for the same reason.
If PWD contains symlinks, the fallback code won't know about them.
If Getwd returns the same result as PWD with resolved symlinks,
the test should not fail.
Change-Id: I39587ddb826d4e18339e185aad0cdd60167b1079
Reviewed-on: https://go-review.googlesource.com/c/go/+/610759
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Reviewed-by: Tim King <taking@google.com>
Among other things, this should fix a regression in printf
whereby materialized aliases caused "any" and "interface{}"
in printf signatures not to be recognized as identical.
It also updates ureader.go used by vendored x/tools during
some tests, including cmd/internal/moddeps.TestAllDependencies.
This test uses golang.org/x/tools/cmd/bundle which uses x/reader.
Fixes#68796
Change-Id: I9f0711e66a5c4daaffe695c515aea3b8fb3d01e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/610736
Reviewed-by: Robert Findley <rfindley@google.com>
Auto-Submit: Alan Donovan <adonovan@google.com>
Reviewed-by: Tim King <taking@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This implementation utilizes the same registers found in the reference
implementation, aiming to produce a minimal semantic diff between the
Avo-generated output and the original hand-written assembly.
To verify the Avo implementation, the reference and Avo-generated
assembly files are fed to `go tool asm`, capturing the debug output into
corresponding temp files. The debug output contains supplementary
metadata (line numbers, instruction offsets, and source file references)
that must be removed in order to obtain a semantic diff of the two
files. This is accomplished via a small utility script written in awk.
The reference assembly file does not specify a frame size for a number
of the defined assembly functions. Avo automatically infers the frame
size when generating the TEXT directive, leading to a diff on those
lines.
Commands used to verify Avo output:
GOROOT=$(go env GOROOT)
ASM_PATH="src/crypto/internal/nistec/p256_asm_amd64.s"
REFERENCE="54fe0fd43fcf8609666c16ae6d15ed92873b1564"
go tool asm -o /dev/null -I "$GOROOT"/src/runtime -debug \
<(git cat-file -p "$REFERENCE:$ASM_PATH") \
> /tmp/reference.s
go tool asm -o /dev/null -I "$GOROOT"/src/runtime -debug \
"$ASM_PATH" \
> /tmp/avo.s
normalize(){
awk '{
$1=$2=$3="";
print substr($0,4)
}'
}
diff <(normalize < /tmp/reference.s) <(normalize < /tmp/avo.s)
1c1
< TEXT <unlinkable>.p256OrdLittleToBig(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.p256OrdLittleToBig(SB), NOSPLIT, $0-16
3c3
< TEXT <unlinkable>.p256OrdBigToLittle(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.p256OrdBigToLittle(SB), NOSPLIT, $0-16
5c5
< TEXT <unlinkable>.p256LittleToBig(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.p256LittleToBig(SB), NOSPLIT, $0-16
7c7
< TEXT <unlinkable>.p256BigToLittle(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.p256BigToLittle(SB), NOSPLIT, $0-16
23c23
< TEXT <unlinkable>.p256MovCond(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.p256MovCond(SB), NOSPLIT, $0-32
74c74
< TEXT <unlinkable>.p256NegCond(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.p256NegCond(SB), NOSPLIT, $0-16
99c99
< TEXT <unlinkable>.p256Sqr(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.p256Sqr(SB), NOSPLIT, $0-24
234c234
< TEXT <unlinkable>.p256Mul(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.p256Mul(SB), NOSPLIT, $0-24
401c401
< TEXT <unlinkable>.p256FromMont(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.p256FromMont(SB), NOSPLIT, $0-16
465c465
< TEXT <unlinkable>.p256Select(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.p256Select(SB), NOSPLIT, $0-24
513c513
< TEXT <unlinkable>.p256SelectAffine(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.p256SelectAffine(SB), NOSPLIT, $0-24
566c566
< TEXT <unlinkable>.p256OrdMul(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.p256OrdMul(SB), NOSPLIT, $0-24
806c806
< TEXT <unlinkable>.p256OrdSqr(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.p256OrdSqr(SB), NOSPLIT, $0-24
Change-Id: I610b097c573b9d9018f0e26bc2afde5edb3f954b
Reviewed-on: https://go-review.googlesource.com/c/go/+/599875
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
This implementation utilizes the same registers found in the reference
implementation, aiming to produce a minimal semantic diff between the
Avo-generated output and the original hand-written assembly.
To verify the Avo implementation, the reference and Avo-generated
assembly files are fed to `go tool asm`, capturing the debug output into
corresponding temp files. The debug output contains supplementary
metadata (line numbers, instruction offsets, and source file references)
that must be removed in order to obtain a semantic diff of the two
files. This is accomplished via a small utility script written in awk.
The reference assembly file does not specify a frame size for some of
the defined assembly functions. Avo automatically infers the frame size
when generating TEXT directives, leading to a diff on those lines.
Commands used to verify Avo output:
GOROOT=$(go env GOROOT)
ASM_PATH="src/crypto/aes/asm_amd64.s"
REFERENCE="54fe0fd43fcf8609666c16ae6d15ed92873b1564"
go tool asm -o /dev/null -I "$GOROOT"/src/runtime -debug \
<(git cat-file -p "$REFERENCE:$ASM_PATH") \
> /tmp/reference.s
go tool asm -o /dev/null -I "$GOROOT"/src/runtime -debug \
"$ASM_PATH" \
> /tmp/avo.s
normalize(){
awk '{
$1=$2=$3="";
print substr($0,4)
}'
}
diff <(normalize < /tmp/reference.s) <(normalize < /tmp/avo.s)
1c1
< TEXT <unlinkable>.encryptBlockAsm(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.encryptBlockAsm(SB), NOSPLIT, $0-32
45c45
< TEXT <unlinkable>.decryptBlockAsm(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.decryptBlockAsm(SB), NOSPLIT, $0-32
89c89
< TEXT <unlinkable>.expandKeyAsm(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.expandKeyAsm(SB), NOSPLIT, $0-32
Change-Id: If647584df4137146d355f91ac0f6a8285d07c932
Reviewed-on: https://go-review.googlesource.com/c/go/+/600375
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
This implementation utilizes the same registers found in the reference
implementation, aiming to produce a minimal semantic diff between the
Avo-generated output and the original hand-written assembly.
To verify the Avo implementation, the reference and Avo-generated
assembly files are fed to `go tool asm`, capturing the debug output into
corresponding temp files. The debug output contains supplementary
metadata (line numbers, instruction offsets, and source file references)
that must be removed in order to obtain a semantic diff of the two
files. This is accomplished via a small utility script written in awk.
The reference assembly file does not specify a frame size for some of
the defined assembly functions. Avo automatically infers the frame size
when generating TEXT directives, leading to a diff on those lines. Some
metadata not included in the reference assembly has also been added,
which leads to a diff in the lines where that parameter symbol is
referenced.
Commands used to verify Avo output:
GOROOT=$(go env GOROOT)
ASM_PATH="src/crypto/aes/gcm_amd64.s"
REFERENCE="54fe0fd43fcf8609666c16ae6d15ed92873b1564"
go tool asm -o /dev/null -I "$GOROOT"/src/runtime -debug \
<(git cat-file -p "$REFERENCE:$ASM_PATH") \
> /tmp/reference.s
go tool asm -o /dev/null -I "$GOROOT"/src/runtime -debug \
"$ASM_PATH" \
> /tmp/avo.s
normalize(){
awk '{
$1=$2=$3="";
print substr($0,4)
}'
}
diff <(normalize < /tmp/reference.s) <(normalize < /tmp/avo.s)
1c1
< TEXT <unlinkable>.gcmAesFinish(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.gcmAesFinish(SB), NOSPLIT, $0-40
44c44
< TEXT <unlinkable>.gcmAesInit(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.gcmAesInit(SB), NOSPLIT, $0-32
131c131
< TEXT <unlinkable>.gcmAesData(SB), NOSPLIT, $0
---
> TEXT <unlinkable>.gcmAesData(SB), NOSPLIT, $0-40
325c325
< MOVQ dst+8(FP), DX
---
> MOVQ dst_base+8(FP), DX
1207c1207
< MOVQ dst+8(FP), SI
---
> MOVQ dst_base+8(FP), SI
Change-Id: Iad8f8c6ea5d50ac093c8535adc9d23fbf2612fc2
Reviewed-on: https://go-review.googlesource.com/c/go/+/601462
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
This implementation utilizes the same registers found in the reference
implementation, aiming to produce a minimal semantic diff between the
Avo-generated output and the original hand-written assembly.
To verify the Avo implementation, the reference and Avo-generated
assembly files are fed to `go tool asm`, capturing the debug output into
corresponding temp files. The debug output contains supplementary
metadata (line numbers, instruction offsets, and source file references)
that must be removed in order to obtain a semantic diff of the two
files. This is accomplished via a small utility script written in awk.
Metadata not found in the reference assembly file has been added to one
parameter symbol, resulting in a single line diff.
Commands used to verify Avo output:
GOROOT=$(go env GOROOT)
ASM_PATH="src/crypto/md5/md5block_amd64.s"
REFERENCE="54fe0fd43fcf8609666c16ae6d15ed92873b1564"
go tool asm -o /dev/null -I "$GOROOT"/src/runtime -debug \
<(git cat-file -p "$REFERENCE:$ASM_PATH") \
> /tmp/reference.s
go tool asm -o /dev/null -I "$GOROOT"/src/runtime -debug \
"$ASM_PATH" \
> /tmp/avo.s
normalize(){
awk '{
$1=$2=$3="";
print substr($0,4)
}'
}
diff <(normalize < /tmp/reference.s) <(normalize < /tmp/avo.s)
3c3
< MOVQ p+8(FP), SI
---
> MOVQ p_base+8(FP), SI
Change-Id: Ifecc84fd0f5a39a88350e6eaffb45ed3fdacf2fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/599935
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
This implementation utilizes the same registers found in the reference
implementation, aiming to produce a minimal semantic diff between the
Avo-generated output and the original hand-written assembly.
To verify the Avo implementation, the reference and Avo-generated
assembly files are fed to `go tool asm`, capturing the debug output into
corresponding temp files. The debug output contains supplementary
metadata (line numbers, instruction offsets, and source file references)
that must be removed in order to obtain a semantic diff of the two
files. This is accomplished via a small utility script written in awk.
Commands used to verify Avo output:
GOROOT=$(go env GOROOT)
ASM_PATH="src/crypto/sha512/sha512block_amd64.s"
REFERENCE="54fe0fd43fcf8609666c16ae6d15ed92873b1564"
go tool asm -o /dev/null -I "$GOROOT"/src/runtime -debug \
<(git cat-file -p "$REFERENCE:$ASM_PATH") \
> /tmp/reference.s
go tool asm -o /dev/null -I "$GOROOT"/src/runtime -debug \
"$ASM_PATH" \
> /tmp/avo.s
normalize(){
awk '{
$1=$2=$3="";
print substr($0,4)
}'
}
diff <(normalize < /tmp/reference.s) <(normalize < /tmp/avo.s)
Change-Id: I172f0cb97252635c657efe82d1b547e6b6f40ebb
Reviewed-on: https://go-review.googlesource.com/c/go/+/598958
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
This implementation utilizes the same registers found in the reference
implementation, aiming to produce a minimal semantic diff between the
Avo-generated output and the original hand-written assembly.
To verify the Avo implementation, the reference and Avo-generated
assembly files are fed to `go tool asm`, capturing the debug output into
corresponding temp files. The debug output contains supplementary
metadata (line numbers, instruction offsets, and source file references)
that must be removed in order to obtain a semantic diff of the two
files. This is accomplished via a small utility script written in awk.
Commands used to verify Avo output:
GOROOT=$(go env GOROOT)
ASM_PATH="src/crypto/sha1/sha1block_amd64.s"
REFERENCE="54fe0fd43fcf8609666c16ae6d15ed92873b1564"
go tool asm -o /dev/null -I "$GOROOT"/src/runtime -debug \
<(git cat-file -p "$REFERENCE:$ASM_PATH") \
> /tmp/reference.s
go tool asm -o /dev/null -I "$GOROOT"/src/runtime -debug \
"$ASM_PATH" \
> /tmp/avo.s
normalize(){
awk '{
$1=$2=$3="";
print substr($0,4)
}'
}
diff <(normalize < /tmp/reference.s) <(normalize < /tmp/avo.s)
1273c1273
< MOVQ $K_XMM_AR<>(SB), R8
---
> LEAQ K_XMM_AR<>(SB), R8
Change-Id: I39168fadb01baa9a96bc2b432fc94b492d036ce4
Reviewed-on: https://go-review.googlesource.com/c/go/+/598795
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
This implementation utilizes the same registers found in the reference
implementation, aiming to produce a minimal semantic diff between the
Avo-generated output and the original hand-written assembly.
To verify the Avo implementation, the reference and Avo-generated
assembly files are fed to `go tool asm`, capturing the debug output into
corresponding temp files. The debug output contains supplementary
metadata (line numbers, instruction offsets, and source file references)
that must be removed in order to obtain a semantic diff of the two
files. This is accomplished via a small utility script written in awk.
Commands used to verify Avo output:
GOROOT=$(go env GOROOT)
ASM_PATH="src/crypto/sha256/sha256block_amd64.s"
REFERENCE="54fe0fd43fcf8609666c16ae6d15ed92873b1564"
go tool asm -o /dev/null -I "$GOROOT"/src/runtime -debug \
<(git cat-file -p "$REFERENCE:$ASM_PATH") \
> /tmp/reference.s
go tool asm -o /dev/null -I $GOROOT/src/runtime -debug \
"$ASM_PATH" \
> /tmp/avo.s
normalize(){
awk '{
$1=$2=$3="";
print substr($0,4)
}'
}
diff <(normalize < /tmp/reference.s) <(normalize < /tmp/avo.s)
3513c3513
< MOVQ $K256<>(SB), BP
---
> LEAQ K256<>(SB), BP
4572c4572
< MOVQ $K256<>(SB), BP
---
> LEAQ K256<>(SB), BP
Change-Id: I637c01d746ca775b8a09f874f7925ffc3b4965ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/595559
Reviewed-by: Russell Webb <russell.webb@protonmail.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Currently the unique package tries to clone strings that get stored in
its internal map to avoid retaining large strings.
However, this falls over entirely due to the fact that the original
string is *still* stored in the map as a key. Whoops. Fix this by
storing the cloned value in the map instead.
This change also adds a test which fails without this change.
Change-Id: I1a6bb68ed79b869ea12ab6be061a5ae4b4377ddb
Reviewed-on: https://go-review.googlesource.com/c/go/+/610738
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
There's a bug in the weak-to-strong conversion in that creating the
*only* strong pointer to some weakly-held object during the mark phase
may result in that object not being properly marked.
The exact mechanism for this is that the new strong pointer will always
point to a white object (because it was only weakly referenced up until
this point) and it can then be stored in a blackened stack, hiding it
from the garbage collector.
This "hide a white pointer in the stack" problem is pretty much exactly
what the Yuasa part of the hybrid write barrier is trying to catch, so
we need to do the same thing the write barrier would do: shade the
pointer.
Added a test and confirmed that it fails with high probability if the
pointer shading is missing.
Fixes#69210.
Change-Id: Iaae64ae95ea7e975c2f2c3d4d1960e74e1bd1c3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/610396
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
In rare situations, like during same-sized grows, the source map for
maps.Clone may be overloaded (has more than 6.5 entries per
bucket). This causes the runtime to allocate a larger bucket array for
the destination map than for the source map. The maps.Clone code
walks off the end of the source array if it is smaller than the
destination array.
This is a pretty simple fix, ensuring that the destination bucket
array is never longer than the source bucket array. Maybe a better fix
is to make the Clone code handle shorter source arrays correctly, but
this fix is deliberately simple to reduce the risk of backporting this
fix.
Fixes#69110
Change-Id: I824c93d1db690999f25a3c43b2816fc28ace7509
Reviewed-on: https://go-review.googlesource.com/c/go/+/609757
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
We can see ENOMEM on FreeBSD.
Also don't fail the test if we get an EPERM error when reading
all the way up the tree; on Android we get that, perhaps because
the root directory is unreadable.
Also accept an EFAULT from a stat of a long name on Dragonfly,
which we see on the builders.
Change-Id: If37e6bf414b7b568c9a06130f71e79af153bfb75
Reviewed-on: https://go-review.googlesource.com/c/go/+/610415
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Change SimpleFold to search the CaseRanges table only once when no
folding is specified for the rune (previously up to two searches could
be performed). This improves performance by 2x for runes that have no
folds or are already upper case. As a side effect this improves the
performance of To by roughly ~15%
goos: darwin
goarch: arm64
pkg: unicode
cpu: Apple M1 Max
│ base.10.txt │ new.10.txt │
│ sec/op │ sec/op vs base │
ToUpper-10 11.860n ± 1% 9.731n ± 1% -17.95% (p=0.000 n=10)
ToLower-10 12.31n ± 1% 10.34n ± 1% -16.00% (p=0.000 n=10)
SimpleFold/Upper-10 19.16n ± 0% 15.98n ± 1% -16.64% (p=0.000 n=10)
SimpleFold/Lower-10 32.41n ± 1% 17.09n ± 1% -47.27% (p=0.000 n=10)
SimpleFold/Fold-10 8.884n ± 4% 8.856n ± 8% ~ (p=0.700 n=10)
SimpleFold/NoFold-10 30.87n ± 0% 15.49n ± 3% -49.84% (p=0.000 n=10)
geomean 17.09n 12.47n -26.99%
Change-Id: I6e5c7554106842955aadeef7b266c4c7944d3a97
Reviewed-on: https://go-review.googlesource.com/c/go/+/454958
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
addLocalFacts loop already ft.update which sets up limits correctly, but doing this in flowLimit help us since other values might depend on this limit.
Updates #68857
We could improve this further:
- remove mod alltogheter when we can prove a < b.
- we could do more adhoc computation in flowLimit to set umax and umin tighter
Change-Id: I5184913577b6a51a07cb53a6e6b73552a982de0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/605156
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
As of CL 257637, all currently supported platforms have syscall.Getwd
implemented, so the code which deduces wd by traversing up to root
directory is never used and thus can be removed.
Or, as it was suggested by Ian Lance Taylor in CL 607436 review
comments, it can be reused when syscall.Getwd returns ENAMETOOLONG
(which usually happens than the current working dir is longer than
syscall.PathMax).
Let's do that. The only caveat is, such a long path returned from Getwd
couldn't be used for any file-related operations (they will probably
fail with ENAMETOOLONG).
While at it:
- make the stat(".") code conditional, slightly improving the
performance on Unix when $PWD is not set;
- reuse variables dir and err;
- use openDirNolog instead of openFileNolog to obtain a dirfd;
- ensure the errors returned are wrapped;
- document the new functionality;
- add test cases (which fail before this change).
Change-Id: I60f7a70e6ebb1751699416f587688a1a97305fd7
Reviewed-on: https://go-review.googlesource.com/c/go/+/608635
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
```
export CC="zig cc -target x86_64-linux"
hyperfine '../pkg/tool/darwin_arm64/cgo -objdir /tmp net/cgo_linux.go net/cgo_resnew.go net/cgo_socknew.go net/cgo_unix_cgo.go net/cgo_unix_cgo_res.go'
```
**Before**
```
Time (mean ± sig): 1.293 s ± 0.017 s [User: 0.472 s, System: 0.451 s]
Range (min ... max): 1.263 s ... 1.316 s 10 runs
```
**After**
```
Time (mean ±sig): 986.5 ms ± 22.6 ms [User: 487.0 ms, System: 519.5 ms]
Range (min ... max): 950.7 ms ... 1022.2 ms 10 runs
```
The version after changes is 25% faster for 5 input files (std "net" package).
I also tried to make CC artifictially slower (wrapper with sleep 0.2) and it showes same 25% performance increase.
Change-Id: I7a26fdc8d8a23b0df9bc71d30b96e82e2ddb943b
Reviewed-on: https://go-review.googlesource.com/c/go/+/581336
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
A previous change [1] was introduced to enable MPTCP by default
for both the clients and servers, based on the discussions [2] in
golang#56539, where MPTCP would be an opt-in for a release or
two, and then would become an opt-out.
This change was not accepted at the time because the support for
a few socket options was missing [3]. Now that this support has been
added [4] and backported to stable versions not to block MPTCP
deployment with Go, it sounds like a good time to reconsider the use
of MPTCP by default.
Instead of enabling MPTCP on both ends by default, as a first step,
it seems safer to change the default behaviour only for the server
side (Listeners). On the server side, the impact is minimal: when
clients don't request to use MPTCP, server applications will create
"plain" TCP sockets within the kernel when connections are accepted,
making the performance impact minimal. This should also ease
experiments where MPTCP is enabled by default on the client side
(Dialer).
The changes in this patch consist of a duplication of the mptcpStatus
enumeration to have both a mptcpStatusDial and a mptcpStatusListen,
where MPTCP is enabled by default in mptcpStatusListen, but disabled
by default in mptcpStatusDial. It is still possible to turn MPTCP support
on and off by using GODEBUG=multipathtcp=1.
[1] https://go-review.googlesource.com/c/go/+/563575
[2] https://go.dev/issue/56539#issuecomment-1309294637
[3] https://github.com/multipath-tcp/mptcp_net-next/issues/383
[4] bd11dc4fb9
[5] https://www.mptcp.dev/faq.html#why--when-should-mptcp-be-enabled-by-default
Updates #56539
Change-Id: I1ca0d6aaf74d3bda5468af135e29cdb405d3fd00
GitHub-Last-Rev: 5f9f29bfc1
GitHub-Pull-Request: golang/go#69016
Reviewed-on: https://go-review.googlesource.com/c/go/+/607715
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Matthieu Baerts <matttbe@kernel.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
On Linux one process can call prlimit to change the resource limit
of another process. With this change we treat that as though the
current process called prlimit (or setrlimit) to set its own limit.
The cost is one additional getrlimit system call per fork/exec,
for cases in which the rlimit Cur and Max values differ at startup.
This revealed a bug: the setrlimit (not Setrlimit) function should not
change the cached rlimit. That means that it must call prlimit1, not prlimit.
Fixes#66797
Change-Id: I46bfd06e09ab7273fe8dd9b5b744dffdf31d828b
Reviewed-on: https://go-review.googlesource.com/c/go/+/607516
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Aleksa Sarai <cyphar@cyphar.com>
Reviewed-by: Kirill Kolyshkin <kolyshkin@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
If the aligned offset isn't sufficient for the field offset,
we were padding based on the aligned offset. We need to pad
based on the original offset instead.
Also set the Go alignment correctly for int128. We were defaulting
to the maximum alignment, but since we translate int128 into an
array of uint8 the correct Go alignment is 1.
Fixes#69086
Change-Id: I23ce583335c81beac2ac51f7f9336ac97ccebf09
Reviewed-on: https://go-review.googlesource.com/c/go/+/608815
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Change the rules for how //go:build "file versions" are applied: instead
of considering whether a file version is an upgrade or downgrade from
the -lang version, always use max(fileVersion, go1.21). This prevents
file versions from downgrading the version below go1.21. Before Go 1.21
the //go:build version did not have the meaning of setting the file's
langage version.
This fixes an issue that was appearing in GOPATH builds: Go 1.23.0
started providing -lang versions to the compiler in GOPATH mode (among
other places) which it wasn't doing before, and it set -lang to the
toolchain version (1.23). Because the -lang version was greater than
go1.21, language version used to compile the file would be set to the
//go:build file version. //go:build file versions below 1.21 could cause
files that could previously build to stop building.
For example, take a Go file with a //go:build line specifying go1.10.
If that file used a 1.18 feature, that use would compile fine with a Go
1.22 toolchain. But it would produce an error when compiling with the
1.23.0 toolchain because it set the language version to 1.10 and
disallowed the 1.18 feature. This breaks backwards compatibility: when
the build tag was added, it did not have the meaning of restricting the
language version.
For #68658
Change-Id: I6cedda81a55bcccffaa3501eef9e2be6541b6ece
Reviewed-on: https://go-review.googlesource.com/c/go/+/607955
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Griesemer <gri@google.com>
As of CL 580255, the runtime tracks the frame pointer (or base pointer,
bp) when entering syscalls, so that we can use fpTracebackPCs on
goroutines that are sitting in syscalls. That CL mostly got things
right, but missed one very subtle detail.
When calling from Go->C->Go, the goroutine stack performing the calls
when returning to Go is free to move around in memory due to growth,
shrinking, etc. But upon returning back to C, it needs to restore
gp.syscall*, including gp.syscallsp and gp.syscallbp. The way syscallsp
currently gets updated is automagically: it's stored as an
unsafe.Pointer on the stack so that it shows up in a stack map. If the
stack ever moves, it'll get updated correctly. But gp.syscallbp isn't
saved to the stack as an unsafe.Pointer, but rather as a uintptr, so it
never gets updated! As a result, in rare circumstances, fpTracebackPCs
can correctly try to use gp.syscallbp as the starting point for the
traceback, but the value is stale.
This change fixes the problem by just storing gp.syscallbp to the stack
on cgocallback as an unsafe.Pointer, like gp.syscallsp. It also adds a
comment documenting this subtlety; the lack of explanation for the
unsafe.Pointer type on syscallsp meant this detail was missed -- let's
not miss it again in the future.
Now, we have a fix, what about a test? Unfortunately, testing this is
going to be incredibly annoying because the circumstances under which
gp.syscallbp are actually used for traceback are non-deterministic and
hard to arrange, especially from within testprogcgo where we don't have
export_test.go and can't reach into the runtime.
So, instead, add a gp.syscallbp check to reentersyscall and
entersyscallblock that mirrors the gp.syscallbp consistency check. This
probably causes some miniscule slowdown to the syscall path, but it'll
catch the issue without having to actually perform a traceback.
Fixes#69085.
Change-Id: Iaf771758f1666024b854f5fbe2b2c63cbe35b201
Reviewed-on: https://go-review.googlesource.com/c/go/+/608775
Reviewed-by: Nick Ripley <nick.ripley@datadoghq.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
An invalid executable may claim to have a data section bigger than the
executable, causing readData in searchMagic to hit EOF. Since readData
suppresses all EOF errors, searchData would keep attempting to search
through a potentially huge "section" despite readData continuously
failing.
Fix by suppressing EOF only on partial read. If nothing is read, allow
EOF. Note that most of the admittedly tedious EOF handling in this
package is around ensuring we return errNotGoExe in most cases.
This was discovered by the new fuzz test. This fuzz test was inspired
by #69066, though it has not found that specific bug.
Change-Id: Icf413e996cecc583c084c9e44249b9294c3d8f10
Reviewed-on: https://go-review.googlesource.com/c/go/+/608637
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The $HOME/sdk/go1.17 and $HOME/go1.17 paths were initially added as
places to look for a bootstrap toolchain to make.{bash,bat,rc} scripts
and in cmd/dist (CL 370274). Those two go1.17 directories have since
been updated in the make scripts to go1.20.6 (CL 512275) and later on
to go1.22.6 (CL 606156), but the same list in cmd/dist was missed.
Fix the inconsistency now. But maybe cmd/dist doesn't need to maintain
this logic, if it's required to be invoked via one of the make scripts,
since they're responsible for setting GOROOT_BOOTSTRAP?
For #64751.
Change-Id: I0988005c559014791363138f2f722cc1f9a78bcf
Reviewed-on: https://go-review.googlesource.com/c/go/+/607821
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
If the length reported for the object file is more than the amount of
data we actually read, then the count can tell us that there is
sufficient remaining data but the slice operation can fail.
No test case because the problem can only happen for invalid data.
Let the fuzzer find cases like this.
Fixes#69066
Change-Id: I8d12ca8ade3330517ade45c7578b477772b7efd2
Reviewed-on: https://go-review.googlesource.com/c/go/+/608517
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Create an intrinsicBuilders type that has functions for adding and
looking up intrinsics. This makes the implementation more self contained,
readable and testable. Additionally, pass an *intrinsicBuildConfig to
initIntrinsics to improve testability without needing to modify package
level variables.
Change-Id: I0ee0a19c192dd6da9f1c5f1c29b98a3ad8161fe2
Reviewed-on: https://go-review.googlesource.com/c/go/+/605478
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Joel Sing <joel@sing.id.au>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
filepath.Rel can sometimes return the a relative path that doesn't work.
If the basepath contains a symlink as a path component, and the targpath
does not exist with the directory pointed to by the innermost symlink,
the relative path can "cross" the symlink. The issue is that for the
return value for filepath.Rel to be correct, the ".." components of the
relative path would need to be collapsed before the symlinks are
expanded, but it was verified by doing local testing that the opposite
is true.
go work use (and cmd/go/internal/modload.ReadModFile) both try to
shorten absolute path arguments to relative paths from the working
directory (for better error messages, for instance). Avoid doing so when
the relative path could be wrong using a more conservative rule than the
above: if expanding the symlinks in the current directory produces a
different result, and the relative path we'd return starts with ".." and
then the path separator.
Fixes#68383
Change-Id: I0a6202be672484d4000fc753c69f2165615f3f72
Reviewed-on: https://go-review.googlesource.com/c/go/+/603136
TryBot-Bypass: Michael Matloob <matloob@golang.org>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Run-TryBot: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The current implementation has a panic when the database is closed
concurrently with a new connection attempt.
connRequestSet.CloseAndRemoveAll sets connRequestSet.s to a nil slice.
If this happens between calls to connRequestSet.Add and
connRequestSet.Delete, there is a panic when trying to write to the nil
slice. This is sequence is likely to occur in DB.conn, where the mutex
is released between calls to db.connRequests.Add and
db.connRequests.Delete
This change updates connRequestSet.CloseAndRemoveAll to set the curIdx
to -1 for all pending requests before setting its internal slice to nil.
CloseAndRemoveAll already iterates the full slice to close all the request
channels. It seems appropriate to set curIdx to -1 before deleting the
slice for 3 reasons:
1. connRequestSet.deleteIndex also sets curIdx to -1
2. curIdx will not be relevant to anything after the slice is set to nil
3. connRequestSet.Delete already checks for negative indices
Fixes#68949
Change-Id: I6b7ebc5a71b67322908271d13865fa12f2469b87
GitHub-Last-Rev: 7d2669155b
GitHub-Pull-Request: golang/go#68953
Reviewed-on: https://go-review.googlesource.com/c/go/+/607238
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Commit-Queue: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
CL 574695 added caching the os.Chdir argument for Windows, and used the
cached value to assess the length of the current working directory in
addExtendedPrefix (used by fixLongPath).
It did not take into account that Chdir can accept relative paths, and
thus the pathLength calculation in addExtendedPrefix can be wrong.
Let's only cache the os.Chdir argument if it's absolute, and clean the
cache otherwise, thus improving the correctness of fixLongPath.
For #41734
For #21782
For #36375
Change-Id: Ie24a5ed763a7aacc310666d2e4cbb8e298768670
Reviewed-on: https://go-review.googlesource.com/c/go/+/607437
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The commands to build the bootstrap toolchains and go commands are run
from modules created by two bootstrap go.mod files: one is used when
building toolchain1 and go_bootstrap, and the other is used for
toolchain2 and toolchain3, and the final build. Currently the first has
a go directive specifying go 1.20, and the second one does not have a go
directive at all. This affects the default GODEBUG setting when building
the final toolchain: the default GODEBUG value is based on the go
version of the go.mod file, and when the go.mod file does not have a
version it defaults to go1.16. We should set the go directive on the
bootstrap used for the second half of the builds to use the current go
verison from the std's go.mod file (which is the same as the version on
cmd's go.mod file).
The go.mod file used for the initial bootstrap should have a go
directive with the minimum version of the toolchain required for
bootstrapping. That version is the current version - 2 rounded down to
an even number.
For #64751Fixes#68797
Change-Id: Ibdddf4bc36dc963291979d603c4f3fc55264f65b
Reviewed-on: https://go-review.googlesource.com/c/go/+/604799
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
[This is a roll-forward of CL 597255, which had to be rolled back
because it broke the windows-arm64 builder, whose current user display
name is unavailable. This new CL fixes the issue by reintroducing the
historical behavior of falling back to the user name instead of
returning an error].
user.Current is slow on Windows sessions connected to an Active
Directory domain. This is because it uses Windows APIs that do RPC
calls to the domain controller, such as TranslateAccountW and
NetUserGetInfo.
This change speeds up user.Current by using the GetUserNameEx API
instead, which is already optimized for retrieving the current user
name in different formats.
These are the improvements I see with the new implementation:
goos: windows
goarch: amd64
pkg: os/user
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
│ old.txt │ new.txt │
│ sec/op │ sec/op vs base │
Current-12 501.8µ ± 7% 118.6µ ± 11% -76.36% (p=0.000 n=10)
│ old.txt │ new.txt │
│ B/op │ B/op vs base │
Current-12 888.0 ± 0% 832.0 ± 0% -6.31% (p=0.000 n=10)
│ old.txt │ new.txt │
│ allocs/op │ allocs/op vs base │
Current-12 15.00 ± 0% 11.00 ± 0% -26.67% (p=0.000 n=10)
Updates #5298Fixes#21867Fixes#68312
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-arm64
Change-Id: Ib7f77086d389cccb9d91cb77ea688d438a0ee5fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/605135
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently the first thing Make does it get the abi.Type of its argument,
and uses abi.TypeOf to do it. However, this has a problem for interface
types, since the type of the value stored in the interface value will
bleed through. This is a classic reflection mistake.
Fix this by implementing and using a generic TypeFor which matches
reflect.TypeFor. This gets the type of the type parameter, which is far
less ambiguous and error-prone.
Fixes#68990.
Change-Id: Idd8d9a1095ef017e9cd7c7779314f7d4034f01a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/607355
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Adds a new Version type to pkgbits to represent the version of the
bitstream. Versions let readers and writers know when different data is
expected to be present or not in the bitstream. These different pieces
of data are called Fields, as an analogy with fields of a struct.
Fields can be added, removed or changed in a Version. Extends Encoder
and Decoder to report which version they are.
Updates #68778
Change-Id: Iaffa1828544fb4cbc47a905de853449bc8e5b91f
Reviewed-on: https://go-review.googlesource.com/c/go/+/605655
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Adds a -gomodversion flag to testdir. This sets the go version
in generated go.mod files. This is just runindir tests at the moment.
This is a building block so that tests can be written for exported
type parameterized aliases (like reproducing #68526).
This also adds a test that uses this feature. A type parameterized
alias is used so aliastypeparams and gotypesalias must be enabled.
gotypesalias is enabled by the go module version. The alias is not
exported and will not appear in exportdata. The test shows the
package containing the alias can be imported. This encapsulates
the level of support of type parameterized aliases in 1.23.
Updates #68526
Updates #68778
Change-Id: I8e20df6baa178e1d427d0fff627a16714d9c3b18
Reviewed-on: https://go-review.googlesource.com/c/go/+/604102
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Austin Clements <austin@google.com>
The AIX ABI requires allocating parameter save space when calling
a function, even if the arguments are passed via registers.
gcc sometimes uses this space. In the case of the cgo c-archive
tests, it clobbered the storage space of argc/argv which prevented
the test program from running the expected test.
Fixes#68957
Change-Id: I8a267b463b1abb2b37ac85231f6c328f406b7515
Reviewed-on: https://go-review.googlesource.com/c/go/+/606895
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Mutex contention events with delay of 0 need more than CL 604355 added:
When deciding which event to store in the M's single available slot,
always choose to drop the zero-delay event. Store an explicit flag for
whether we have an event to store, rather than relying on a non-zero
delay.
And, fix a test of sync.Mutex contention that expects those events to
have non-zero delay. The reporting of non-runtime contention like this
has long allowed zero-delay events, which we see when cputicks has low
resolution.
Fixes#68892Fixes#68906
Change-Id: Id412141e4eb09724f3ce195899a20d59c92d7b78
Reviewed-on: https://go-review.googlesource.com/c/go/+/606115
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Some tests need to use os.Chdir, but the use is complicated because
- they must change back to the old working directory;
- they must not use t.Parallel.
Add Chdir that covers these cases, and sets PWD environment variable
to the new directory for the duration of the test for Unix platforms.
Unify the panic message when t.Parallel is used together with t.Setenv
or t.Chdir.
Add some tests.
For #62516.
Change-Id: Ib050d173b26eb28a27dba5a206b2d0d877d761c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/529895
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
In Go 1.22 we added code to the go/build package to ignore #cgo noescape
and nocallback directives. That permits us to enable these directives in Go 1.24.
Also, this fixed a Bug in CL 497837:
After retiring _Cgo_use for parameters, the compiler will treat the
parameters, start from the second, as non-alive. Then, they will be marked
as scalar in stackmap, which means the pointer won't be copied correctly
in copystack.
Fixes#56378.
Fixes#63739.
Change-Id: I46e773240f8a467c3c4ba201dc5b4ee473cf6e3e
GitHub-Last-Rev: 42fcc506d6
GitHub-Pull-Request: golang/go#66879
Reviewed-on: https://go-review.googlesource.com/c/go/+/579955
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
For the moment, Go calls sendfile(2) to transfer at most 4MB at a time
while sendfile(2) actually allows a larger amount of data on one call.
To reduce system calls of sendfile(2) during data copying, we should
specify the number of bytes to copy as large as possible.
This optimization is especially advantageous for bulky file-to-file copies,
it would lead to a performance boost, the magnitude of this performance
increase may not be very exciting, but it can also cut down the CPU overhead
by decreasing the number of system calls.
This is also how we've done in sendfile_windows.go with TransmitFile.
goos: linux
goarch: amd64
pkg: os
cpu: DO-Premium-AMD
│ old │ new │
│ sec/op │ sec/op vs base │
SendFile-8 1.135 ± 4% 1.052 ± 3% -7.24% (p=0.000 n=10)
│ old │ new │
│ B/s │ B/s vs base │
SendFile-8 902.5Mi ± 4% 973.0Mi ± 3% +7.81% (p=0.000 n=10)
│ old │ new │
│ B/op │ B/op vs base │
SendFile-8 272.0 ± 0% 272.0 ± 0% ~ (p=1.000 n=10) ¹
¹ all samples are equal
│ old │ new │
│ allocs/op │ allocs/op vs base │
SendFile-8 20.00 ± 0% 20.00 ± 0% ~ (p=1.000 n=10) ¹
¹ all samples are equal
Change-Id: Ib4d4c6bc693e23db24697363b29226f0c9776bb0
Reviewed-on: https://go-review.googlesource.com/c/go/+/605235
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Jorropo <jorropo.pgm@gmail.com>
Run-TryBot: Andy Pan <panjf2000@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
The block and mutex profiles have slightly different behaviors when a
sampled event has a negative (or zero) duration. The block profile
enforces a minimum duration for each event of "1" in the cputicks unit.
It does so by clamping the duration to 1 if it was originally reported
as being smaller. The mutex profile for app-level contention enforces a
minimum duration of 0 in a similar way: by reporting any negative values
as 0 instead.
The mutex profile for runtime-internal contention had a different
behavior: to enforce a minimum event duration of "1" by dropping any
non-conforming samples.
Stop dropping samples, and use the same minimum (0) that's in place for
the other mutex profile events.
Fixes#64253Fixes#68453Fixes#68781
Change-Id: I4c5d23a2675501226eef5b9bc1ada2efc1a55b9e
Reviewed-on: https://go-review.googlesource.com/c/go/+/604355
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com>
After newobject, we don't need to write zeroes to initialize the
object. It has already been zeroed by the allocator.
This is already handled in most cases, but because we run builtin
decomposition after the opt pass, we don't handle cases where the zero
of a compound builtin is being written. Improve the zero detector to
handle those cases.
Fixes#68845
Change-Id: If3dde2e304a05e5a6a6723565191d5444b334bcc
Reviewed-on: https://go-review.googlesource.com/c/go/+/605255
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Auto-Submit: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
This CL adds support of "library", i.e. c-shared, build mode on
wasip1. When -buildmode=c-shared is set, it builds a Wasm module
that is intended to be used as a library, instead of an executable.
It does not have the _start function. Instead, it has an
_initialize function, which initializes the runtime, but not call
the main function.
This is similar to the c-shared build mode on other platforms. One
difference is that unlike cgo callbacks, where Ms are created on-
demand, on Wasm we have only one M, so we just keep the M (and the
G) for callbacks.
For #65199.
Change-Id: Ieb21da96b25c1a9f3989d945cddc964c26f9085b
Reviewed-on: https://go-review.googlesource.com/c/go/+/604316
Reviewed-by: Achille Roussel <achille.roussel@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Changes the type inference error message so that the position is
proceeded by a space. cmd/go rewrites the output of gc to replace
absolute paths at the beginning of lines and those proceeded by a
space or a tab to relative paths.
Updates testdir to do the same post processing on the output
of tests as cmd/go.
Fixes#68292
Change-Id: Ie109b51143e68f6e7ab4cd19064110db0e609a7e
Reviewed-on: https://go-review.googlesource.com/c/go/+/603097
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Get rid of TODO in prove pass.
We currently avoid marking shifts of constants as bounded, where
bounded means we don't have to worry about <0 or >=bitwidth shifts.
We do this because it causes different rule applications during lowering
which cause some codegen tests to fail.
Add some new rules which ensure that we get the right final instruction
sequence regardless of the ordering. Then we can remove this special case.
Change-Id: I4e962d4f09992b42ab47e123de5ded3b8b8fb205
Reviewed-on: https://go-review.googlesource.com/c/go/+/602935
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
The spec says that an embedded field must be specified
as a type name (or a pointer to a type name). This is
explicit in the prose and the FieldDecl syntax.
However, the prose on promoted methods required a named
type (originally the term used for a "defined type").
Before the introduction of alias types, type names could
only refer to named/defined types, so the prose was ok.
With the introduction of alias types in Go 1.9, we
distinguished between defined types (i.e., types given
a name through a type declaration) and type aliases
(types given an alternative name), and retired the notion
of a named type since any type with a name (alias type
and defined type) could be considered a "named type".
To make things worse, with Go 1.18 we re-introduced the
notion of a named type which now includes predeclared
types, defined types, type parameters (and with that
type aliases denoting named types).
In the process some of the wording on method promotion
didn't get updated correctly. At attempt to fix this
was made with CL 406054, but while that CL's description
correctly explained the intent, the CL changed the prose
from "defined type" to "named type" (which had the new
meaning after Go 1.18), and thus did not fix the issue.
This CL fixes that fix by using the term "type name".
This makes the prose consistent for embedded types and
in turn clarifies that methods of embedded alias types
(defined or not) can be promoted, consistent with the
implementation.
While at it, also document that the type of an embedded
field cannot be a type parameter. This restriction has
been in place since the introduction of type parameters
with Go 1.18 and is enforced by the compiler.
Fixes#66540.
For #41687.
Change-Id: If9e6a03d7b84d24a3e6a5ceda1d46bda99bdf1f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/603958
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Bypass: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Axel Wagner <axel.wagner.hh@googlemail.com>
Only honor //go:build language version downgrades if the version
specified is 1.21 or greater. Before 1.21 the version in //go:build
lines didn't have the meaning of setting the file's language version.
This fixes an issue that was appearing in GOPATH builds: Go 1.23 started
providing -lang versions to the compiler in GOPATH mode (among other
places) which it wasn't doing before.
For example, take a go file with a //go:build line specifying go1.10.
If that file used a 1.18 feature, that use would compile fine with a Go
1.22 toolchain. But, before this change, it would produce an error when
compiling with the 1.23 toolchain because it set the language version to
1.20 and disallowed the 1.18 feature. This breaks backwards
compatibility: when the build tag was added, it did not have the meaning
of restricting the language version.
Fixes#68658
Change-Id: I4ac2b45a981cd019183d52ba324ba8f0fed93a8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/603895
Reviewed-by: Robert Griesemer <gri@google.com>
Commit-Queue: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Matloob <matloob@golang.org>
Currently the crash function on Wasm is implemented as a nil
pointer dereference, which turns into a sigpanic, which turns into
"panic during runtime execution" as we're already in runtime when
crash is called. Instead, just abort, which crashes hard and
terminates the Wasm module execution, and the execution engine
often dumps a stack trace.
Change-Id: I3c57f8ff7a0c0015e4abcd7bf262bf9001624b85
Reviewed-on: https://go-review.googlesource.com/c/go/+/604515
Reviewed-by: Achille Roussel <achille.roussel@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
CL 603055 added basic support of wasmexport. This CL follows it
and adds stack unwinding handling. If the wasmexport Go function
returns normally, we directly return to the host. If the Go
function unwinds the stack (e.g. goroutine switch, stack growth),
we need to run a PC loop to call functions on the new stack,
similar to wasm_pc_f_loop. One difference is that when the
wasmexport function returns normally, we need to exit the loop and
return to the host.
Now a wasmimport function can call back into the Go via wasmexport.
During the callback the stack could have moved. The wasmimport
code needs to read a new SP after the host function returns,
instead of assuming the SP doesn't change.
For #65199.
Change-Id: I62c1cde1c46f7eb72625892dea41e8137b361891
Reviewed-on: https://go-review.googlesource.com/c/go/+/603836
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Achille Roussel <achille.roussel@gmail.com>
When running a go binary compiled to wasm using node.js on a Windows platform,
the absolute path passed in is also incorrectly forced to expand.
For example:
E:\Project\CS_Project\gsv\testdata\result.gob.gz
will results to
open C:\Users\zxilly\AppData\Local\wasm-exec\go1.23rc1\E:\Project\CS_Project\gsv\testdata\result.gob.gz: No such file or directory
C:\Users\zxilly\AppData\Local\wasm-exec\go1.23rc1 is the place of
wasm_exec_node.js
Fixes: #68820
Change-Id: Ic30c6242302f8915ac1b8ea9f24546935cbb791e
GitHub-Last-Rev: f35ff1a2ee
GitHub-Pull-Request: golang/go#68255
Reviewed-on: https://go-review.googlesource.com/c/go/+/595797
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
This CL adds a compiler directive go:wasmexport, which applies to
a Go function and makes it an exported function of the Wasm module
being built, so it can be called directly from the host. As
proposed in #65199, parameter and result types are limited to
32-bit and 64-bit integers and floats, and there can be at most
one result.
As the Go and Wasm calling conventions are different, for a
wasmexport function we generate a wrapper function does the ABI
conversion at compile time.
Currently this CL only adds basic support. In particular,
- it only supports executable mode, i.e. the Go wasm module calls
into the host via wasmimport, which then calls back to Go via
wasmexport. Library (c-shared) mode is not implemented yet.
- only supports wasip1, not js.
- if the exported function unwinds stacks (goroutine switch, stack
growth, etc.), it probably doesn't work.
TODO: support stack unwinding, c-shared mode, js.
For #65199.
Change-Id: Id1777c2d44f7d51942c1caed3173c0a82f120cc4
Reviewed-on: https://go-review.googlesource.com/c/go/+/603055
Reviewed-by: Than McIntosh <thanm@golang.org>
Reviewed-by: Randy Reddig <randy.reddig@fastly.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(Second attempt at CL 529816 (f1d6050), reverted in
CL 571695 (1304d98) due to broken longtest builder.)
The tests analyser reports structural problems in test
declarations. Presumably most of these would be caught by
go test itself, which compiles and runs (some subset of) the
tests, but Benchmark and Fuzz functions are executed less
frequently and may benefit more from static checks.
A number of tests of "go vet" needed to be updated, either
to avoid mistakes caught by the analyzer, or to suppress
the analyzer when the mistakes were intended.
Also, reflect the change in go test help message.
+ release note
Fixesgolang/go#44251
Change-Id: I1c311086815fe55a66cce001eaab9b41e27d1144
Reviewed-on: https://go-review.googlesource.com/c/go/+/603476
Auto-Submit: Alan Donovan <adonovan@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
user.Current is slow on Windows sessions connected to an Active
Directory domain. This is because it uses Windows APIs that do RPC
calls to the domain controller, such as TranslateAccountW and
NetUserGetInfo.
This change speeds up user.Current by using the GetUserNameEx API
instead, which is already optimized for retrieving the current user
name in different formats.
These are the improvements I see with the new implementation:
goos: windows
goarch: amd64
pkg: os/user
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
│ old.txt │ new.txt │
│ sec/op │ sec/op vs base │
Current-12 501.8µ ± 7% 118.6µ ± 11% -76.36% (p=0.000 n=10)
│ old.txt │ new.txt │
│ B/op │ B/op vs base │
Current-12 888.0 ± 0% 832.0 ± 0% -6.31% (p=0.000 n=10)
│ old.txt │ new.txt │
│ allocs/op │ allocs/op vs base │
Current-12 15.00 ± 0% 11.00 ± 0% -26.67% (p=0.000 n=10)
Updates #5298Fixes#21867Fixes#68312
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest
Change-Id: I893c5fcca6969050d73a20ed34770846becd5f5e
Reviewed-on: https://go-review.googlesource.com/c/go/+/597255
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
It appears that some builders (notably, linux-arm) have some additional
security software installed, which apparently reads the files created by
tests. As a result, test file atime is changed, making the test fail
like these:
=== RUN TestChtimesOmit
...
os_test.go:1475: atime mismatch, got: "2024-07-30 18:42:03.450932494 +0000 UTC", want: "2024-07-30 18:42:02.450932494 +0000 UTC"
=== RUN TestChtimes
...
os_test.go:1539: AccessTime didn't go backwards; was=2024-07-31 20:45:53.390326147 +0000 UTC, after=2024-07-31 20:45:53.394326118 +0000 UTC
According to inode(7), atime is changed when more than 0 bytes are read
from the file. So, one possible solution to these flakes is to make the
test files empty, so no one can read more than 0 bytes from them.
Fixes#68687Fixes#68663
Change-Id: Ib9234567883ef7b16ff8811e3360cd26c2d6bdab
Reviewed-on: https://go-review.googlesource.com/c/go/+/604315
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Kirill Kolyshkin <kolyshkin@gmail.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Go utilizes copy_file_range(2) for file-to-file copying only on kernel 5.3+,
but even on 5.3+ this system call can still go wrong for some reason (check
out the comment inside poll.CopyFileRange).
Before Linux 2.6.33, out_fd must refer to a socket, but since Linux 2.6.33
it can be any file. Thus, we can employ sendfile(2) for copy between files
when copy_file_range(2) fails to handle the copy, that way we can still
benefit from the zero-copy technique on kernel <5.3 and wherever
copy_file_range(2) is available but broken.
Change-Id: I3922218c95ad34ee649ccdf3ccfbd1ce692bebcc
Reviewed-on: https://go-review.googlesource.com/c/go/+/603295
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
We don't need noLimit checks in a bunch of places.
Also simplify folding of provable constant results.
At this point in the CL stack, compilebench reports no performance
changes. The only thing of note is that binaries got a bit smaller.
name old text-bytes new text-bytes delta
HelloSize 960kB ± 0% 952kB ± 0% -0.83% (p=0.000 n=10+10)
CmdGoSize 12.3MB ± 0% 12.1MB ± 0% -1.53% (p=0.000 n=10+10)
Change-Id: Id4be75eec0f8c93f2f3b93a8521ce2278ee2ee2c
Reviewed-on: https://go-review.googlesource.com/c/go/+/599197
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Here begins a pretty major rewrite of the prove pass. The fundamental
observation is that although keeping facts about relations between
two SSA values could use O(n^2) space, keeping facts about relations
between an SSA value and constants needs only O(n) space. We can just
keep track of min/max for every SSA value at little cost.
Redo the limit table to just keep track of limits for all SSA values.
Use just a slice instead of a map. It may use more space (but still
just O(n) space), but accesses are a lot faster. And with the cache
in the compiler, that space will be reused quickly.
This is part of my planning to add lots more constant limits in the
prove pass.
Change-Id: Ie36819fad5631a8b79c3630fe0e819521796551a
Reviewed-on: https://go-review.googlesource.com/c/go/+/599255
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
The syntax parser complains about invalid identifiers.
Don't report a typechecker error when such an identifier
cannot be found in the current scope.
For now add a local test for types2 only because the
go/parser behaves differently than the syntax parser
which leads to slightly different error positions.
Fixes#68183.
Change-Id: Idbfe62fafcd704886069182744ec5e6b37ffc4e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/602476
Auto-Submit: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Tim King <taking@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
When producing an ImportPathError from ImportErrorf, we check to see
whether the error string contains the path for the error. The issue is
that we were checking for the exact path string when sometimes the
string is quoted when the error is constructed, and the escaping in the
quote may not match the path string. Check for both the path string, and
the quoted path string.
Fixes#68737
Change-Id: I01bf4e495056e929570bc11bc1f2000ce6d2802b
Reviewed-on: https://go-review.googlesource.com/c/go/+/603475
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
This CL does some minor refactoring of the code handling
wasmimport.
- Put the WasmImport aux reading and writing code together for
symmetry.
- Define WasmFuncType, embedded in WasmImport. WasmFuncType could
also be used (later) for wasmexport.
- Move code generation code to a separate function. The containing
function is already pretty large.
- Simplify linker code a little bit. The loader convention is to
return the 0 Sym for nonexistent symbol, instead of a separate
boolean.
No change in generated code. Passes toolstash -cmp
(GOARCH=wasm GOOS=wasip1 go build -toolexec "toolstash -cmp" -a std cmd).
Change-Id: Idc2514f84a08621333841ae4034b81130e0ce411
Reviewed-on: https://go-review.googlesource.com/c/go/+/603135
Reviewed-by: Than McIntosh <thanm@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Fixes#68548
Add GOENV=off, GOFLAGS= to the build of the stdlib, so that it matches
what runcmd does. This ensures that the runtime and the test are built
with the same flags. As opposed to before this CL, where flags were used
in the stdlib build but not the runcmd build.
(Part of the problem here is that cmd/internal/testdir/testdir_test.go
plays fast and loose with the build cache to make the tests run faster.
Maybe some of that fast-and-loose mechanism can be removed now that we
have a better build cache? I'm not sure.)
Change-Id: I449d4ff517c69311d0aa4411e7fb96c0cca49269
Reviewed-on: https://go-review.googlesource.com/c/go/+/600276
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
During the GC mark phase, one of the first behaviors of findRunnable is
to check if it should execute a GC mark worker. Mark workers often run
for many milliseconds in a row, so programs that invoke the scheduler
more frequently will see that condition trigger only a tiny fraction of
the time.
Obtaining a mark worker from the gcBgMarkWorkerPool involves a CAS on a
single memory location that's shared across the process. When GOMAXPROCS
is large, the resulting contention can waste a significant amount of CPU
time. But a sufficiently large GOMAXPROCS also means there's no need for
fractional mark workers, making it easier to check ahead of time if we
need to run a worker.
Check, without committing to a particular worker, whether we would even
want to run one.
For #68399
Change-Id: I5d8578c2101ee20a8a4156a029584356095ea118
Reviewed-on: https://go-review.googlesource.com/c/go/+/602477
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Rhys Hiltner <rhys.hiltner@gmail.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
notes are used in sensitive locations in the runtime, such as those with
write barriers forbidden. Maps aren't designed for this sort of internal
use.
Notably, newm -> notewakeup doesn't allow write barriers, but mapaccess1
-> panic contains write barriers. The js runtime only builds right now
because the map access is optimized to mapaccess1_fast64, which happens
to not have a panic call.
The initial swisstable map implementation doesn't have a fast64 variant.
While we could add one, it is a bad idea in general to use a map in such
a fragile location. Simplify the implementation by storing the metadata
directly in the note, and using a linked list for checkTimeouts.
For #54766.
Cq-Include-Trybots: luci.golang.try:gotip-js-wasm
Change-Id: Ib9d39f064ae4ad32dcc873f799428717eb6c2d5a
Reviewed-on: https://go-review.googlesource.com/c/go/+/595558
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
These are delay primitives for lock2. If a mutex isn't immediately
available, we can use procyield to tell the processor to wait for a
moment, or osyield to allow the OS to run a different process or thread
if one is waiting. We expect a processor-level yield to be faster than
an os-level yield, and for both of them to be fast relative to entering
a full sleep (via futexsleep or semasleep).
Each architecture has its own way of hinting to the processor that it's
in a spin-wait loop, so procyield presents an architecture-independent
interface for use in lock_futex.go and lock_sema.go.
Measure the (single-threaded) speed of these to confirm.
For #68578
Change-Id: I90cd46ea553f2990395aceb048206285558c877e
Reviewed-on: https://go-review.googlesource.com/c/go/+/601396
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
This change replaces the usage of the "sort" package with the "slices"
package for sorting IP addresses and DNS records. The new approach
simplifies the code and improves readability by leveraging the
slices.SortFunc and slices.SortStableFunc functions.
- Updated addrselect.go to use slices.SortStableFunc for sorting IP
addresses based on RFC 6724.
- Refactored dnsclient.go to use slices.SortFunc for sorting SRV and MX
records by priority and weight.
This change also reduces the dependency tree for the package by
removing the dependency on "sort" and its transitive dependencies,
resulting in a leaner build.
Change-Id: I436dacc8dd1e8f2f7eeac44d6719ce248394d8a9
GitHub-Last-Rev: 3720a49081
GitHub-Pull-Request: golang/go#67503
Reviewed-on: https://go-review.googlesource.com/c/go/+/586635
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
CL 594740 rewrote type checking of method receiver types. Because that
CL takes apart receivers "manually" rather than using the regular code
for type checking type expressions, type parameters in receiver type
expressions were only recorded as definitions (in Info.Defs).
Before that CL, such type parameters were simultaneously considered
definitions (they are declared by the receiver type expression) and
uses (they are used to instantiate the receiver type expression).
Adjust the receiver type checking code accordingly and record its
type parameters also in Info.Uses and Info.Types.
While at it, in go/types, replace declareTypeParams (plural) with
declareTypeParam (singular) to more closely match types2 code.
No functionality or semantic change.
Fixes#68670.
For #51343.
Change-Id: Ibbca1a9b92e31b0dc972052a2827deeab49da98b
Reviewed-on: https://go-review.googlesource.com/c/go/+/601935
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Tim King <taking@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Rather than reading the entire data segment into memory, read it in
smaller chunks to keep memory usage low.
For typically Go binaries, this doesn't matter much. For those, we read
the .go.buildinfo section, which should be quite small. But for non-Go
binaries (or Go binaries with section headers stripped), we search the
entire loadable data segment, which could be quite large.
This reduces the time for `go version` on a 2.5GB non-Go binary from
~1.2s and 1GB RSS (!!) to ~1s and ~25MB RSS.
Fixes#68592.
Change-Id: I9218854c5b6f2aa1331f561ab0850a9fd62ef23b
Reviewed-on: https://go-review.googlesource.com/c/go/+/601459
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Add in automatic README generation and README consistency checking for
the cmd/compile and cmd/link script tests. This code is adapted from
the similar facility in cmd/go (e.g. scriptreadme_test.go); the README
helps folks writing new tests understand the mechanics.
Updates #68606.
Change-Id: I8ff7ff8e814abd4385bd670440511b2c60a4cef6
Reviewed-on: https://go-review.googlesource.com/c/go/+/601756
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Introduce a new function AddToolChainScriptConditions that augments a
default "script.Cond" set with a collection of useful conditions,
including godebug/goexperiment, cgo, race support, buildmode, asan,
msan, and so on. Having these conditions available makes it easier to
write script tests that deal with specific build-flavor corner cases.
The functions backing the new conditions are helper functions migrated
over from the Go command's script test setup.
Updates #68606.
Change-Id: I14def1115b54dc47529c983abcd2c5ea9326b9de
Reviewed-on: https://go-review.googlesource.com/c/go/+/601715
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
CL 594740 rewrote type checking of method receiver types. Because that
CL takes apart receivers "manually" rather than using the regular code
for type checking type expressions, pointer and parenthesized receiver
type expressions were not recorded anymore.
Adjust the code that typechecks method receivers to a) use ordinary
type expression checking for non-generic receivers, and b) to record
a missing pointer and any intermediate parenthesized expressions in
case of a generic receiver.
Add many extra tests verifying that the correct types for parenthesized
and pointer type expressions are recorded in various source positions.
Note that the parser used by the compiler and types2 doesn't encode
unnecessary parentheses in type expressions in its syntax tree.
As a result, the tests that explicitly test parentheses don't work
in types2 and are commented out.
This CL adds code (disabled by default) to the parser to encode
parentheses in type expressions in the syntax tree. When enabled,
the commented out types2 tests pass like in go/types.
Fixes#68639.
For #51343.
Change-Id: Icf3d6c76f7540ee53e229660be8d78bb25380539
Reviewed-on: https://go-review.googlesource.com/c/go/+/601657
Reviewed-by: Tim King <taking@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
In clang 16 the option -fsanitize-memory-param-retval was turned on by
default. That option causes MSAN to issue a warning when calling a
function with an uninitialized value. The msan8 test relies on being
able to do this, in order to get uninitialized values into registers.
This CL fixes the test by adding maybe_undef attributes that tell
clang that it's OK to pass an uninitialized variable. The docs for
maybe_undef say: "Please note that this is an attribute that is used as
an internal implementation detail and not intended to be used by
external users." So this may break in the future, but it does work for now.
Fixes#64616
Change-Id: I0ac8c0520fce8c32e26d2a5efb7ae5e02461c1ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/601779
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
The buildinfo used for a testmain is a copy from the buildinfo produced
for the package under test, and that in turn is only computed if the
package under test is package main. If there are //go:debug directives
in a test file for package main, the godebugs for the testmain (which
are computed using the regular package files as well as the test files'
//go:debug directives) will be different from those used to produce the
buildinfo of the package under test (computed using the //go:debug
directives only in the main package). In that case, recompute the
buildinfo for the testmain to incorporate the new godebug information.
Since we've only been generating buildinfo for tests on package main, in
this CL we'll only recompute the buildinfo if the test is for package
main. It's not clear to me though if we should be computing the
buildinfo for all test mains (or none of them?)
Fixes#68053
Change-Id: Ib6cdb118e2f233de483c33e171c0cd03df1fc7be
Reviewed-on: https://go-review.googlesource.com/c/go/+/595961
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
The current test often doesn't actually generate enough interleaving
to result in multiple log shards. This CL rewrites this test to
forcibly create at least 10 log shards with interleaved log messages.
It also tests dlog's robustness to being held across M and P switches.
Change-Id: Ia913b17c0392384ff679832047f359945669bb15
Reviewed-on: https://go-review.googlesource.com/c/go/+/600699
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Austin Clements <austin@google.com>
This CL adds a "deadlocals" pass, which runs after inlining and before
escape analysis, to prune any unneeded local variables and
assignments. In particular, this helps avoid unnecessary Addrtaken
markings from unreachable closures.
Deadlocals is sensitive to "_ = ..." as a signal of explicit
use for testing. This signal occurs only if the entire
left-hand-side is "_" targets; if it is
`_, ok := someInlinedFunc(args)`
then the first return value is eligible for dead code elimination.
Use this (`_ = x`) to fix tests broken by deadlocals elimination.
Includes a test, based on one of the tests that required modification.
Matthew Dempsky wrote this, changing ownership to allow rebases, commits, tweaks.
Fixes#65158.
Old-Change-Id: I723fb69ccd7baadaae04d415702ce6c8901eaf4e
Change-Id: I1f25f4293b19527f305c18c3680b214237a7714c
Reviewed-on: https://go-review.googlesource.com/c/go/+/600498
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: David Chase <drchase@google.com>
Commit-Queue: David Chase <drchase@google.com>
When generating DW_TAG_subroutine_type DIEs during linker DWARF type
synthesis, ensure that in the list of children of the subroutine type
DIE (correspondings to input/output params) the output params are
marked with the DW_AT_variable_parameter attribute. In addition, fix
up the generated types of the output params: prior to this patch for a
given output parameter of type T, we would emit the DIE type as *T
(presumably due to how parameter passing/returning worked prior to the
register ABI); with this patch the emitted type will just be T, not *T.
Fixes#59977.
Change-Id: I5b5600be86473695663c75b85baeecad667b9245
Reviewed-on: https://go-review.googlesource.com/c/go/+/595715
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Currently, the debuglog tests only run when the debuglog build tag is
set because, until the last few CLs, all of debuglog was compiled away
without that build tag. This causes two annoying problems:
1. The tests basically never run, because we don't regularly test this
configuration.
2. If you do turn on the debuglog build tag, it's probably because
you're adding debuglogs into the runtime, which are very likely to
mess up these tests, so you wind up disabling the tests and they,
again, don't get coverage.
Now we've set things up so the debuglog implementation is always
accessible, if you ask nicely enough. So we can switch these tests to
run when the tag is *not* set, and turn off when the tag *is* set (and
you're probably adding actual log statements).
Change-Id: Ib68d7a5022d4f5db96e9c7c8010cbef21d11fe11
Reviewed-on: https://go-review.googlesource.com/c/go/+/600697
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Currently, the debuglog build tag controls the dlogEnabled const, and
all methods of dlogger first check this const and immediately return
if dlog is not enabled. With constant folding and inlining, this makes
the whole dlog implementation compile away if it's not enabled.
However, we want to be able to test debuglog even when the build tag
isn't set. For that to work, we need a different mechanism.
This CL changes this mechanism so the debuglog build tag instead
controls the type alias for dlogger to be either dloggerImpl or
dloggerFake. These two types have the same method set, but one is just
stubs. This way, the methods of dloggerImpl don't need to be
conditional dlogEnabled, which sets us up to use the now
fully-functional dloggerImpl type in the test.
I confirmed that this change has no effect on the final size of the
cmd/go binary. It does increase the size of the runtime.a file by 0.9%
and make the runtime take ever so slightly longer to compile because
the compiler can no longer simply eliminate the bodies of the all of
dlogger during early deadcode. However, this all gets eliminated by
the linker. I consider this worth it to always get build and test
coverage of debuglog.
Change-Id: I81759e9e1411b7d369a23383a18b022ab7451421
Reviewed-on: https://go-review.googlesource.com/c/go/+/600696
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Add support for running script tests as part of the compiler's suite
of tests, hooking in the script test engine packages recently moved
from cmd/go to cmd/internal. These script tests will use the test
binary itself as the compile tool for Go builds, and can also run the
C compiler if needed. New script test cases (*.txt files) should be
added to the directory cmd/compile/testdata/script.
Updates #68606.
Change-Id: I9b056a07024b0a72320a89ad734e4b4a51f1c10c
Reviewed-on: https://go-review.googlesource.com/c/go/+/601361
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Add support for running script tests as part of the linker's suite of
tests, hooking in the script test engine packages recently moved from
cmd/go to cmd/internal. Linker script tests will use the test binary
itself as the linker for Go builds, and can also run the C compiler if
needed. New script test cases (*.txt files) should be added to the
directory cmd/link/testdata/script.
For demo purposes, this patch also adds a new "randlayout_option.txt"
script test that replicates the existing linker's TestRandLayout
testpoint in script form.
Updates #68606.
Change-Id: Icf26bf657850e39548d6ea819f2542fc68a3899b
Reviewed-on: https://go-review.googlesource.com/c/go/+/601360
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
Relocate cmd/go's internal/robustio package up a level into
cmd/internal/robustio, so that it can be used by other cmd/internal
packages. No change in functionality. This change is intended to be in
support of making the cmd/go script test framework available to other
commands in addition to just the Go command.
Updates #68606.
Change-Id: Ic8421ef59d9b7d79a50c3679d180cfa2546c4cd3
Reviewed-on: https://go-review.googlesource.com/c/go/+/601356
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Relocate cmd/go's internal/par package up a level into
cmd/internal/par, so that it can be used by other cmd/internal
packages. No change in functionality. This change is intended to be in
support of making the cmd/go script test framework available to other
commands in addition to just the Go command.
Updates #68606.
Change-Id: I920a5d5c9b362584fabdbb2305414325b42856a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/601355
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
The trace parser was using an otherwise-unused event argument to hold an
extra goroutine state argument for the GoStatus & GoStatusStack events.
This is needed because the execution tracer just records the "after" for
state transitions, but we want to have both the "before" and "after"
states available in the StateTransition info for the parsed event. When
GoStatusStack was added, the size of the argument array was increased to
still have room for the extra status. However, statuses are currently
only 1 byte, and the status argument is 8 bytes, so there is plenty of
room to pack the "before" and "after" statuses in a single argument. Do
that instead to avoid the need for an extra argument.
Change-Id: I6886eeb14fb8e5e046b6afcc5b19e04218bcacd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/601455
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
On Loong64, the two input operands and one output operand of the ADDF
instruction are both floating-point registers; and the floating-point
comparison instruction CMPEQ{F,D}, CMPGE{F,D}, CMPGT{F,D} both input
operands are floating-point registers, and the output operation is a
floating-point condition register, currently, only FCC0 is used as the
floating-point condition register.
Example:
ADDF F0, F1, F0
CMPEQF F0, F1, FCC0
Change-Id: I4c1c453e522d43f294a8dcab7b6b5247f41c9c68
Reviewed-on: https://go-review.googlesource.com/c/go/+/580281
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: abner chenc <chenguoqi@loongson.cn>
Auto-Submit: abner chenc <chenguoqi@loongson.cn>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
1) Factor out handling of receiver from Checker.funcType into
Checker.collectRecv. Analyze the receiver parameter "manually"
without resorting to calling Checker.collectParams.
The code is more straight-forward and error handling is simpler
because constructing the receiver type and variable is all handled
in one function.
2) Change Checker.collectParams to collect parameter names and
corresponding parameter variables, but do not declare them.
Instead return two equal-length slices of parameter names
and variables for later declaration.
3) Streamline Checker.funcType into a sequence of simple steps.
By declaring the receiver and parameters after type parameters,
there is no need for a temporary scope and scope squashing anymore.
4) Simplify Checker.unpackRecv some more: don't strip multiple
*'s from receiver type expression because we don't typecheck
that expression as a whole later (we don't use collectParams
for receiver types anymore). If we have a **T receiver, we
need to use *T (one * stripped) as receiver base type expression
so that we can report an error later.
5) Remove Checker.recvTParamMap and associated machinery as it is
not needed anymore.
6) Remove Scope.Squash/squash as it is not needed anymore.
7) Remove the explicit scope parameter from Checker.collectParams
as it is not needed anymore.
8) Minor adjustments to tests: in some cases, error positions have
shifted slightly (because we don't use Checker.collectParams to
typecheck receivers anymore), and in some cases duplicate errors
don't appear anymore (resolves TODOs).
Fixes#51343.
Change-Id: Ia77e939bb68e2912ef2e4ed68d2a7a0ad605c5ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/594740
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Instead of returning the receiver type name (rname), return the
receiver type base expression (base), with pointer indirections
stripped. The type base may or may not not be a type name. This
is needed for further rewrites of the signature type-checking code.
Adjust call sites accordingly to preserve existing behavior.
For #51343.
Change-Id: Ib472ca25d43ec340762d0a8dd1ad038568c2b2bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/595335
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Tim King <taking@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Previously, the end position for a select statement clause body was
computed explicitly as the start of the next clause or the closing "}"
of the select statement, respectively.
Since syntax.EndPos computes the end position of a node, there's no
need to compute these positions "manually", we can simply use the
syntax.ExdPos for each clause. The positions are not exactly the
same as before but for the purpose of identifier visibility in
scopes there is no semantic change.
Simplifies the code and brings it more in line with go/types.
Change-Id: I24bca85a131a0ea31a2adaafc08ab713450258fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/593016
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Move logic for type-specific variable type into typeCases function
which already does all the relevant work.
Add more detailed documentation to typeCases function.
Uncomment alernative typeCases function so that it is being type-
checked and kept up-to-date. Since it's not (yet) used, the code
will not appear in the binary.
Follow-up on CL 592555.
Change-Id: I6e746503827d512a1dbf7b99b48345c480e61200
Reviewed-on: https://go-review.googlesource.com/c/go/+/592616
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Currently, we can only cache regular trace event buffers on each M. As
a result, calling unsafeTraceExpWriter will, in effect, always return
a new trace batch, with all of the overhead that entails.
This extends that cache to support buffers for experimental trace
data. This way, unsafeTraceExpWriter can return a partially used
buffer, which the caller can continue to extend. This gives the caller
control over when these buffers get flushed and reuses all of the
existing trace buffering mechanism.
This also has the consequence of simplifying the experimental batch
infrastructure a bit. Now, traceWriter needs to know the experiment ID
anyway, which means there's no need for a separate traceExpWriter
type.
Change-Id: Idc2100176c5d02e0fbb229dc8aa4aea2b1cf5231
Reviewed-on: https://go-review.googlesource.com/c/go/+/594595
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The abi.NoEscape function is introduced to replace all usages of
noescape wrapper in the standard library. However, the last usage in
reflect package is still present, because the inlining test failed if
abi.NoEscape were used. The reason is that reflect.noescape is treated
as a cheap call, while abi.NoEscape is not.
By treating abi.NoEscape a cheap call, the last usage of noescape in
reflect package can now be removed.
Change-Id: I798079780129221a5a26cbcb18c95ee30855b784
Reviewed-on: https://go-review.googlesource.com/c/go/+/601275
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
This change allows the tracer to be reentrant by restructuring the
internals such that writing an event is atomic with respect to stack
growth. Essentially, a core set of functions that are involved in
acquiring a trace buffer and writing to it are all marked nosplit.
Stack growth is currently the only hidden place where the tracer may be
accidentally reentrant, preventing the tracer from being used
everywhere. It already lacks write barriers, lacks allocations, and is
non-preemptible. This change thus makes the tracer fully reentrant,
since the only reentry case it needs to handle is stack growth.
Since the invariants needed to attain this are subtle, this change also
extends the debugTraceReentrancy debug mode to check these invariants as
well. Specifically, the invariants are checked by setting the throwsplit
flag.
A side benefit of this change is it simplifies the trace event writing
API a good bit: there's no need to actually thread the event writer
through things, and most callsites look a bit simpler.
Change-Id: I7c329fb7a6cb936bd363c44cf882ea0a925132f3
Reviewed-on: https://go-review.googlesource.com/c/go/+/587599
Reviewed-by: Austin Clements <austin@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The Go release notes are served on multiple domains (go.dev,
golang.google.cn, tip.golang.org, localhost:8080 and so on), so links
pointing to the Go website itself need to be relative to work in all
those contexts.
Caught by a test in x/website. The next CL adds the same test to this
repository so these kinds of problems are caught sooner and with less
friction.
For #68545.
Fixes#68575.
Change-Id: I08056b98968c77a1d0ed93b63fccfbe41274ec8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/600656
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
For atomic AND and OR operations on memory, we currently have two
views of the op. One just does the operation on the memory and returns
just a memory. The other does the operation on the memory and returns
the old value (before having the logical operation done to it) and
memory.
Update #61395
These two type differently, and there's currently some confusion in
our rules about which is which. Use different names for the two
different flavors so we don't get them confused.
Change-Id: I07b4542db672b2cee98169ac42b67db73c482093
Reviewed-on: https://go-review.googlesource.com/c/go/+/594976
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Reviewed-by: Keith Randall <khr@google.com>
When there are multiple declarations of a function, ensure that
those declarations at least agree on the size/alignment of arguments
and return values.
It's hard to be stricter given existing code and situations where
arguments differ only by typedefs. For instance:
int usleep(unsigned);
int usleep(useconds_t);
Fixes#67699.
Change-Id: I3b4b17afee92b55f9e712b4590ec608ab1f7ac91
Reviewed-on: https://go-review.googlesource.com/c/go/+/588977
Auto-Submit: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
An InlMark "read" can't make an otherwise dead store live. Without this
CL, we sometimes zero an object twice in succession because we think
there is a reader in between.
Kind of challenging to make a test for this. The second zeroing has the
same instruction on the same line number, so codegen tests can't see it.
Fixes#67957
Change-Id: I7fb97ebff50d8eb6246fc4802d1136b7cc76c45f
Reviewed-on: https://go-review.googlesource.com/c/go/+/592615
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
The Go 1.24 development tree has opened. This is a time to update
all golang.org/x/... module versions that contribute packages to the
std and cmd modules in the standard library to latest master versions.
For #36905.
[git-generate]
go install golang.org/x/build/cmd/updatestd@latest
go install golang.org/x/tools/cmd/bundle@latest
updatestd -goroot=$(pwd) -branch=master
Change-Id: I5a012af9f041f79ab4d5b30569a154e0c2d1617e
Reviewed-on: https://go-review.googlesource.com/c/go/+/600535
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Currently internal/trace/testdata contains three debugging tools which
were written early in the trace rewrite for debugging. Two of these are
completely redundant with go tool trace -d=1 and go tool trace -d=2. The
only remaining one landed in the last cycle and could easily also be
another debug mode.
This change thus merges gotraceeventstats into go tool trace as a new
debug mode, and updates the debug mode flag (-d) to accept a string,
giving each mode a more descriptive name.
Change-Id: I170f30440691b81de846b4e247deb3d0982fc205
Reviewed-on: https://go-review.googlesource.com/c/go/+/593975
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
The current memmove implementation uses REP MOVSB to copy data larger than
2KB when the useAVXmemmove global variable is false and the CPU supports
the ERMS feature.
This feature is currently only enabled on CPUs in the Sandy Bridge (Client)
, Sandy Bridge (Server), Ivy Bridge (Client), and Ivy Bridge (Server)
microarchitectures.
For modern Intel CPU microarchitectures that support the ERMS feature, such
as Ice Lake (Server), Sapphire Rapids , REP MOVSB achieves better
performance than the AVX-based copy currently implemented in memmove.
Benchstat result:
goos: linux
goarch: amd64
pkg: runtime
cpu: Intel(R) Xeon(R) Gold 6348 CPU @ 2.60GHz
│ ./old.txt │ ./new.txt │
│ sec/op │ sec/op vs base │
Memmove/2048-2 25.24n ± 0% 24.27n ± 0% -3.84% (p=0.000 n=10)
Memmove/4096-2 44.87n ± 0% 33.16n ± 1% -26.11% (p=0.000 n=10)
geomean 33.65n 28.37n -15.71%
│ ./old.txt │ ./new.txt │
│ B/s │ B/s vs base │
Memmove/2048-2 75.56Gi ± 0% 78.59Gi ± 0% +4.02% (p=0.000 n=10)
Memmove/4096-2 85.01Gi ± 0% 115.05Gi ± 1% +35.34% (p=0.000 n=10)
geomean 80.14Gi 95.09Gi +18.65%
Fixes#66958
Change-Id: I1fafd1b51a16752f83ac15047cf3b29422a79d5d
GitHub-Last-Rev: 89cf5af32b
GitHub-Pull-Request: golang/go#66959
Reviewed-on: https://go-review.googlesource.com/c/go/+/580735
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
@@ -82,7 +82,7 @@ while still insisting that races are errors and that tools can diagnose and repo
<p>
The following formal definition of Go's memory model closely follows
the approach presented by Hans-J. Boehm and Sarita V. Adve in
“<ahref="https://www.hpl.hp.com/techreports/2008/HPL-2008-56.pdf">Foundations of the C++ Concurrency Memory Model</a>”,
“<ahref="https://dl.acm.org/doi/10.1145/1375581.1375591">Foundations of the C++ Concurrency Memory Model</a>”,
published in PLDI 2008.
The definition of data-race-free programs and the guarantee of sequential consistency
for race-free programs are equivalent to the ones in that work.
@@ -453,7 +453,7 @@ crash, or do something else.)
</p>
<pclass="rule">
The <i>k</i>th receive on a channel with capacity <i>C</i> is synchronized before the completion of the <i>k</i>+<i>C</i>th send from that channel completes.
The <i>k</i>th receive from a channel with capacity <i>C</i> is synchronized before the completion of the <i>k</i>+<i>C</i>th send on that channel.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
