WPF Flyout Positioning Issue with FlowDirection="RightToLeft" [closed]

Problem Description

I'm developing a custom WPF Flyout component that uses Popup with CustomPopupPlacementCallback. The Flyout positions correctly when the Window has FlowDirection="LeftToRight", but when I set FlowDirection="RightToLeft" on the Window, the Flyout appears in the wrong position - often completely misaligned with its placement target.

Environment

• Framework: .NET 9.0 / WPF
• Issue: Custom Popup placement is incorrect in RTL layouts
• Affected Components: CustomPopupPlacementCallback, PopupPositioner

Code Structure

My implementation consists of three main components:

1. CustomPopupPlacementHelper - Calculates popup positions using CustomPopupPlacementCallback
2. PopupPositioner - Uses WPF internal methods for advanced positioning
3. FlyoutBase - Main Flyout control that manages the Popup

What I Tried

Attempt 1: Manual RTL Mirroring in CustomPopupPlacementHelper

I detected RTL FlowDirection and manually swapped coordinates:

private static bool ShouldMirrorForRTL(FrameworkElement child)
{
    var popup = FindParentPopup(child);
    if (popup?.PlacementTarget is FrameworkElement target)
    {
        return target.FlowDirection == FlowDirection.RightToLeft;
    }
    return false;
}

// In CalculatePopupPlacement:
case CustomPlacementMode.TopEdgeAlignedLeft:
    point = shouldMirrorForRTL 
        ? new Point(targetSize.Width - popupSize.Width, -popupSize.Height)
        : new Point(0, -popupSize.Height);
    break;

Result: This caused double-transformation issues because WPF Popup already has internal RTL handling.

Attempt 2: RTL Correction in PopupPositioner

I added RTL correction after position calculation:

if (_popup.PlacementTarget is FrameworkElement feRtl && 
    feRtl.FlowDirection == FlowDirection.RightToLeft)
{
    if (PlacementInternal == PlacementMode.Left || 
        PlacementInternal == PlacementMode.Right)
    {
        double mirroredX = bestTranslation.X + (targetWidth - popupWidth) 
            - 2 * (bestTranslation.X - targetBounds.Left);
        bestTranslation.X = mirroredX;
    }
}

Result: Still incorrect - the manual correction conflicted with WPF's internal RTL transformations.

Attempt 3: Adjusted PlacementRectangle in FlyoutBase

I tried adjusting the placement rectangle for RTL:

bool rtl = target is FrameworkElement fe && 
    fe.FlowDirection == FlowDirection.RightToLeft;

if (rtl)
{
    // Move rect left by target width so right edge (x=0) aligns with target's right edge
    value = new Rect(
        new Point(-targetSize.Width, -Offset), 
        new Point(0, targetSize.Height + Offset));
}

Result: This also caused positioning issues due to conflicting with WPF's native handling.

Root Cause Analysis

After investigating WPF's Popup source code (from Reference Source), I discovered that WPF Popup has built-in RTL support:

1. Transform Undoing - Popup automatically applies scale transform to undo FlowDirection mirroring:

   if (parent != null && 
       (FlowDirection)parent.GetValue(FlowDirectionProperty) == FlowDirection.RightToLeft)
   {
       popupTransform.Scale(-1.0, 1.0); // Undo FlowDirection Mirror
   }
   

2. Interest Point Swapping - When FlowDirection differs between target and child, WPF swaps interest points:

   if ((FlowDirection)target.GetValue(FlowDirectionProperty) !=
       (FlowDirection)child.GetValue(FlowDirectionProperty))
   {
       SwapPoints(ref interestPoints[(int)InterestPoint.TopLeft], 
                  ref interestPoints[(int)InterestPoint.TopRight]);
       SwapPoints(ref interestPoints[(int)InterestPoint.BottomLeft], 
                  ref interestPoints[(int)InterestPoint.BottomRight]);
   }
   

The Solution

Remove all manual RTL transformations and let WPF Popup handle RTL natively.

Changes Made:

1. CustomPopupPlacementHelper - Removed ShouldMirrorForRTL() and all RTL-specific positioning logic
2. PopupPositioner - Removed manual RTL corrections
3. FlyoutBase - Simplified GetPlacementRectangle() to always use standard coordinates

Updated CustomPopupPlacementHelper:

internal static CustomPopupPlacement[] PositionPopup(
    CustomPlacementMode placement,
    Size popupSize,
    Size targetSize,
    Point offset,
    FrameworkElement child = null)
{
    Matrix transformToDevice = default;
    if (child != null)
    {
        Helper.TryGetTransformToDevice(child, out transformToDevice);
    }

    // Let WPF Popup handle RTL natively - no manual RTL transformations
    CustomPopupPlacement preferredPlacement = CalculatePopupPlacement(
        placement, popupSize, targetSize, offset, child, transformToDevice);

    // ... rest of the method
}

private static CustomPopupPlacement CalculatePopupPlacement(
    CustomPlacementMode placement,
    Size popupSize,
    Size targetSize,
    Point offset,
    FrameworkElement child = null,
    Matrix transformToDevice = default)
{
    Point point;
    PopupPrimaryAxis primaryAxis;

    switch (placement)
    {
        case CustomPlacementMode.TopEdgeAlignedLeft:
            // Always use standard LTR coordinates
            point = new Point(0, -popupSize.Height);
            primaryAxis = PopupPrimaryAxis.Horizontal;
            break;
        case CustomPlacementMode.TopEdgeAlignedRight:
            point = new Point(targetSize.Width - popupSize.Width, -popupSize.Height);
            primaryAxis = PopupPrimaryAxis.Horizontal;
            break;
        // ... other cases
    }

    return new CustomPopupPlacement(point, primaryAxis);
}

Updated FlyoutBase:

internal Rect GetPlacementRectangle(UIElement target)
{
    Rect value = Rect.Empty;

    if (target != null)
    {
        Size targetSize = target.RenderSize;

        // Simple placement rectangle without RTL transformations
        // WPF Popup will handle RTL layout natively
        switch (Placement)
        {
            case FlyoutPlacementMode.Top:
            case FlyoutPlacementMode.Bottom:
            case FlyoutPlacementMode.TopEdgeAlignedLeft:
            case FlyoutPlacementMode.TopEdgeAlignedRight:
            case FlyoutPlacementMode.BottomEdgeAlignedLeft:
            case FlyoutPlacementMode.BottomEdgeAlignedRight:
                value = new Rect(
                    new Point(0, -Offset), 
                    new Point(targetSize.Width, targetSize.Height + Offset));
                break;
            case FlyoutPlacementMode.Left:
            case FlyoutPlacementMode.Right:
            // ... other horizontal placements
                value = new Rect(
                    new Point(-Offset, 0), 
                    new Point(targetSize.Width + Offset, targetSize.Height));
                break;
        }
    }
    return value;
}

Questions

1. How should I handle RTL in custom popup placement logic? Should I:

   • Let WPF handle RTL entirely and avoid any manual transformations?
   • Detect RTL and mirror my calculations?
   • Compare FlowDirection of target vs. child like WPF does?

2. Should CustomPopupPlacementCallback calculations be RTL-aware? Or does WPF apply RTL transformations after the callback returns?

3. How do I properly detect when RTL mirroring should be applied? Is it:

   • When window FlowDirection = RightToLeft?
   • When target and popup child have different FlowDirection values?
   • Something else?

Expected Behavior

When FlowDirection="RightToLeft":

• BottomEdgeAlignedLeft should align to the visual left edge (right in RTL coordinates)
• BottomEdgeAlignedRight should align to the visual right edge (left in RTL coordinates)
• Center-aligned flyouts should remain centered
• All placements should respect RTL layout flow

Additional Context

• The component works perfectly with FlowDirection="LeftToRight"
• I'm trying to maintain compatibility with WPF's native RTL behavior
• The component supports animations, corner radius adjustments, and offset inversions
• Full source code: WPF-Flyout on GitHub

References

• WPF Popup Placement Behavior
• WPF FlowDirection
• CustomPopupPlacementCallback

Any guidance on the correct approach to handle RTL in custom popup placement would be greatly appreciated!