-
Notifications
You must be signed in to change notification settings - Fork 4.2k
Fix incorrect code generation on async iterators #81468
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
src/Compilers/CSharp/Portable/Lowering/StateMachineRewriter/StateMachineRewriter.cs
Outdated
Show resolved
Hide resolved
| return result; | ||
| } | ||
| static bool IsDisposed(CancellationTokenSource cts) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a better way to do it?
I tested with reflection and the names of the fields differs from net framework (m_disposed) and net 10 (_disposed)
| [WorkItem(81467, "https://github.com/dotnet/roslyn/issues/81467")] | ||
| public void Multiple_GetAsyncEnumerator_Calls_Should_Dispose_Linked_Tokens() | ||
| { | ||
| string source = @" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider using raw string literals (""") for new tests.
| var currentIterator = Create(cts.Token); | ||
| var currentIteratorType = currentIterator.GetType(); | ||
| var combinedTokensField = currentIteratorType.GetField(""<>x__combinedTokens"", BindingFlags.NonPublic | BindingFlags.Instance)!; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need reflection? Is this bug not reproable using public APIs only?
Side question - how did you discover this bug if not in real-world code (which presumably wouldn't use reflection like this)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need reflection? Is this bug not reproable using public APIs only?
I'm not sure exactly how to check if this bug occurs without using reflection
Side question - how did you discover this bug if not in real-world code (which presumably wouldn't use reflection like this)?
Really interesting question
I was reverse engineering the IAsyncEnumerable implementation in SharpLab, saw the combinedTokens field and though it could be overriden by calling GetAsyncEnumerator multiple times
So, I wrote in c# to debug, and saw that the bug was indeed happening
Then I went to the Roslyn repo to find out how this was implemented and realized that the comments didn't really reflect what the code did, and then everything made sense
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We definitely shouldn't rely on internal standard library implementation details via reflection. But it is possible to redefine standard library APIs which is especially useful for testing. For example, this seems to produce different output before and after this PR:
#pragma warning disable CS0436
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices;
using System.Threading;
using System.Threading.Tasks;
var token1 = new CancellationTokenSource();
var iterator = Create(token1.Token);
var token2 = new CancellationTokenSource();
var enumerator1 = iterator.GetAsyncEnumerator(token2.Token);
await Move(enumerator1);
var token3 = new CancellationTokenSource();
var enumerator2 = iterator.GetAsyncEnumerator(token3.Token);
await Move(enumerator2);
await Move(enumerator2);
await Move(enumerator2);
static async Task Move(IAsyncEnumerator<int> enumerator)
{
try
{
if (await enumerator.MoveNextAsync())
Console.Write(enumerator.Current);
else
Console.Write('f');
}
catch (OperationCanceledException) { Console.Write('c'); }
}
static async IAsyncEnumerable<int> Create([EnumeratorCancellation] CancellationToken cancellationToken)
{
yield return 1;
yield return 2;
}
namespace System.Threading
{
public class CancellationTokenSource
{
public static CancellationTokenSource CreateLinkedTokenSource(CancellationToken token1, CancellationToken token2) => new();
public CancellationToken Token => new(this);
public void Dispose() { Console.Write('d'); }
public void Cancel() { }
}
public record struct CancellationToken(CancellationTokenSource Source);
}
namespace System.Collections.Generic
{
public interface IAsyncEnumerable<T>
{
IAsyncEnumerator<T> GetAsyncEnumerator(CancellationToken cancellationToken = default);
}
public interface IAsyncEnumerator<T>
{
ValueTask<bool> MoveNextAsync();
T Current { get; }
}
}|
@jcouv FYI |
|
@henriquewr Please use more detailed PR title. Also, please include detailed description of the the change: what you are planning to change, how do you plan to change, why, why do you think this is a right change, etc. |
Fixes #81467
The implementation of Async iterators currently overrides the value of the field
_combinedTokenswhen callingGetAsyncEnumeratormultiple times, preventing it from being disposedI changed it so that, instead of being assigned to
thisinstance, it is now assigned to the result instance (the instance that the method is returning), to not override the previous value