Getting "Cannot use ref or out parameter inside an anonymous method, lambda expression, or query expression" for an "in" parameter

C# 7.2 added readonly structs and the in modifier for method parameters. However, you get a compiler error when you try to use these structs with reference-like semantics in lambda expressions:

public readonly struct Point {
    public Struct(int x, int y) {
        X = x;
        Y = y;
    }
    public int X { get; }
    public int Y { get; }
}

public IEnumerable<Point> FindMatching(
    this IEnumerable<Point> points, 
    in Point toMatch) {
    return point.Where(p => p.X == point.X && p.Y == point.Y);
}

Compiling returns an error:

error CS1628: Cannot use ref or out parameter 'toMatch' inside an anonymous method, lambda expression, or query expression.

It's not a ref or out parameter, however.