Skip to content

Conversation

@henriquewr
Copy link

@henriquewr henriquewr commented Nov 27, 2025

Fixes #81467

The implementation of Async iterators currently overrides the value of the field _combinedTokens when calling GetAsyncEnumerator multiple times, preventing it from being disposed

I changed it so that, instead of being assigned to this instance, it is now assigned to the result instance (the instance that the method is returning), to not override the previous value

@henriquewr henriquewr requested a review from a team as a code owner November 27, 2025 20:42
@dotnet-policy-service dotnet-policy-service bot added the Community The pull request was submitted by a contributor who is not a Microsoft employee. label Nov 27, 2025
return result;
}
static bool IsDisposed(CancellationTokenSource cts)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a better way to do it?
I tested with reflection and the names of the fields differs from net framework (m_disposed) and net 10 (_disposed)

[WorkItem(81467, "https://github.com/dotnet/roslyn/issues/81467")]
public void Multiple_GetAsyncEnumerator_Calls_Should_Dispose_Linked_Tokens()
{
string source = @"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using raw string literals (""") for new tests.

var currentIterator = Create(cts.Token);
var currentIteratorType = currentIterator.GetType();
var combinedTokensField = currentIteratorType.GetField(""<>x__combinedTokens"", BindingFlags.NonPublic | BindingFlags.Instance)!;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need reflection? Is this bug not reproable using public APIs only?

Side question - how did you discover this bug if not in real-world code (which presumably wouldn't use reflection like this)?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need reflection? Is this bug not reproable using public APIs only?

I'm not sure exactly how to check if this bug occurs without using reflection

Side question - how did you discover this bug if not in real-world code (which presumably wouldn't use reflection like this)?

Really interesting question

I was reverse engineering the IAsyncEnumerable implementation in SharpLab, saw the combinedTokens field and though it could be overriden by calling GetAsyncEnumerator multiple times

So, I wrote in c# to debug, and saw that the bug was indeed happening

Then I went to the Roslyn repo to find out how this was implemented and realized that the comments didn't really reflect what the code did, and then everything made sense

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We definitely shouldn't rely on internal standard library implementation details via reflection. But it is possible to redefine standard library APIs which is especially useful for testing. For example, this seems to produce different output before and after this PR:

#pragma warning disable CS0436

using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices;
using System.Threading;
using System.Threading.Tasks;

var token1 = new CancellationTokenSource();
var iterator = Create(token1.Token);
var token2 = new CancellationTokenSource();
var enumerator1 = iterator.GetAsyncEnumerator(token2.Token);
await Move(enumerator1);
var token3 = new CancellationTokenSource();
var enumerator2 = iterator.GetAsyncEnumerator(token3.Token);
await Move(enumerator2);
await Move(enumerator2);
await Move(enumerator2);

static async Task Move(IAsyncEnumerator<int> enumerator)
{
    try
    {
        if (await enumerator.MoveNextAsync())
            Console.Write(enumerator.Current);
        else
            Console.Write('f');
    }
    catch (OperationCanceledException) { Console.Write('c'); }
}

static async IAsyncEnumerable<int> Create([EnumeratorCancellation] CancellationToken cancellationToken)
{
    yield return 1;
    yield return 2;
}

namespace System.Threading
{
    public class CancellationTokenSource
    {
        public static CancellationTokenSource CreateLinkedTokenSource(CancellationToken token1, CancellationToken token2) => new();
        public CancellationToken Token => new(this);
        public void Dispose() { Console.Write('d'); }
        public void Cancel() { }
    }

    public record struct CancellationToken(CancellationTokenSource Source);
}

namespace System.Collections.Generic
{
    public interface IAsyncEnumerable<T>
    {
        IAsyncEnumerator<T> GetAsyncEnumerator(CancellationToken cancellationToken = default);
    }

    public interface IAsyncEnumerator<T>
    {
        ValueTask<bool> MoveNextAsync();
        T Current { get; }
    }
}

@AlekseyTs
Copy link
Contributor

@jcouv FYI

@AlekseyTs
Copy link
Contributor

@henriquewr Please use more detailed PR title. Also, please include detailed description of the the change: what you are planning to change, how do you plan to change, why, why do you think this is a right change, etc.

@jcouv jcouv self-assigned this Dec 1, 2025
@jcouv jcouv added the Feature - Async Streams Async Streams label Dec 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area-Compilers Community The pull request was submitted by a contributor who is not a Microsoft employee. Feature - Async Streams Async Streams

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect code generation in Async Iterator for combinedTokens

4 participants