1

I'm designing a connection retry function. It retries if the error was ConnectionClosed. In the full implementation it has other things that justify it being a recursion, but this is off topic. Anyways, since it's a connection, it makes sense to be async. However, async recursives force me to use BoxFuture, and we know that asyncs capture and return all the arguments.

use std::future::Future;
use futures::future::{BoxFuture, FutureExt};

struct Message;

struct Client;

enum Error {
    ConnectionClosed
}

impl Client {
    fn send_and_expect<'a>(
        &'a mut self,
        message: &'a Message
    ) -> BoxFuture<'a, Result<(), Error>> {
        async move {
            Ok(())
        }.boxed()
    }
    
    //With this function I can wrap any async function f that grabs data from the internet
    pub fn connection_retrier<'a, T>(
        f: fn(&'a mut Self, &'a Message) -> T,
        f_self: &'a mut Self,
        f_m: &'a Message,
    ) -> BoxFuture<'a, Result<(),Error>>
    where
        T: Future<Output = Result<(), Error>> + 'a + Send
    {
        /*
            By making f: fn(&'a mut Self, &'a Message) -> T, we tell the compiler that
            `f` is a specific function: one that specifically requires the `Self` and `Message` 
            arguments to live as long as `'a`, where `'a` is the min of the lifetimes of the `f_self` and `f_m`
            arguments passed to `connection_retrier`.
            Thus this forces `f_self` and `f_m` to live as long as `connection_retrier`.
            The call `let r = f(f_self, f_m).await` returns a `Future` that lives as long as `'a`.
            I think this is the problem. The result of the first call borrows `f_self` and `f_m`
            for at least the entire lifetime of `r`. This would make it impossible to use
            `f_self` and `f_m` again inside `connection_retrier`. As you see, I tried making sure
            that `r` destructs before we use `f_self` in `connection_retrier` again, but somehow
            `r` is marked to live as long as `connection_retrier`, because `f_self` is still considered
            to be borrowed.
        */
        async move {
            let ok: bool; 
            {
                let r = f(f_self, f_m).await;
                match r {
                    Ok(_) => ok = true,
                    Err(Error::ConnectionClosed) => {
                        ok = false;
                    }
                }
            }
            match ok {
                true => Ok(()),
                false => Client::connection_retrier(f, f_self, f_m).await
            }
        }.boxed()
    }
    
    async fn send_with_retry<'a> (
        &'a mut self,
        message: &'a Message,
    ) -> Result<(), Error>
    {
        Client::connection_retrier(
            Client::send_and_expect,
            self,
            message
        ).await
    }
}

Error:

error[E0499]: cannot borrow `*f_self` as mutable more than once at a time
  --> src/lib.rs:58:56
   |
24 |         f: fn(&'a mut Self, &'a Message) -> T,
   |         - lifetime `'1` appears in the type of `f`
...
48 |                 let r = f(f_self, f_m).await;
   |                         --------------
   |                         | |
   |                         | first mutable borrow occurs here
   |                         argument requires that `*f_self` is borrowed for `'1`
...
58 |                 false => Client::connection_retrier(f, f_self, f_m).await
   |                                                        ^^^^^^ second mutable borrow occurs here

Playground

I understand why the error occurs: Client::connection_retrier(f, f_self, f_m).await, let's call it r, holds a mutable reference to f_self, so I cannot use it again while it's being held. However, after I check that this result r is Error::ConnectionClosed, I don't need it anymore, so there should be a way to discard it so I can reborrow it mutably.

2 Answers 2

1
+250

The problem

As @moy2010 said, the problem is here:

pub fn connection_retrier<'a, T>(
        f: fn(&'a mut Self, &'a Message) -> T,
        f_self: &'a mut Self,
        f_m: &'a Message,
    ) -> BoxFuture<'a, Result<(), Error>>
where T: Future<Output = Result<(), Error>> + 'a + Send

This signature defines f to be a function that accepts arguments with exactly the lifetime 'a, which is alive throughout the body of connection_retrier.

When you are able to temporarily pass a mutable reference to a function and then 'recover' it later, this is called a reborrow. Reborrowing from a mutable reference works by creating a new mutable reference of shorter lifetime to the data it points to, and preventing use of the original reference until this lifetime has ended. Usually a reborrow happens during a function call, in which case that lifetime is just long enough for the function call to be made.

As you can see, reborrowing always produces a reference with lifetime shorter than the original. But f needs a reference whose lifetime is exactly 'a, so the compiler correctly deduces that the call to f cannot reborrow f_self and instead, f_self is permanently moved into f. Hence the error when you attempt to use it again.

Solutions

If you can change the signature of send_and_expect so that the returned future only borrows from message (i.e. not the Client), then @moy2010 has already provided a correct solution in the comments to their answer. I will proceed on the assumption that send_and_expect must borrow from both its arguments, as indicated by the signature in the question.

Since you would like to reborrow f_self before passing it to f, what you really need is to define f to be a function that can be called for any lifetime (not just 'a). You might think that means we can just change the declaration of f to for<'b> fn(&'b mut Self, &'b Message) -> T. But wait! Your where bound specifies that T borrows from the lifetime 'a, not 'b, so this signature doesn't line up with that of send_and_expect. But also notice that we can't change the bound to say 'b instead, because 'b is not in scope outside of the declaration of f! So there is no way to express the correct bound on T here. There are, however, a few workarounds.

Use a Boxed trait object

The simplest solution is just to make the return type of f a boxed trait object instead of a generic parameter, like this:

pub fn connection_retrier<'a>(
        f: for<'b> fn(&'b mut Self, &'b Message) -> BoxFuture<'b, Result<(), Error>>,
        f_self: &'a mut Self,
        f_m: &'a Message,
    ) -> BoxFuture<'a, Result<(), Error>>

Now we can specify what the return value borrows from inline, avoiding the problem that 'b isn't in scope for a where bound. Since send_and_expect already returns a BoxFuture anyway in your example, this may be the best solution for you.

Use a trait implemented on a ZST instead of a fn pointer

Resorting to boxing and dynamic dispatch to satisfy a compile-time issue might strike some as distasteful, and there is a small performance penalty. This is a trick that can be used to make this work with static dispatch, at the cost of some verbosity. We will have to define our own trait, ConnectFn:

trait ConnectFn<'a> {
    type ConnectFuture: Future<Output=Result<(), Error>> + Send + 'a;
    fn connect(&self, client: &'a mut Client, message: &'a Message) -> Self::ConnectFuture;
}

Then, we can make the signature of connection_retrier the following:

pub fn connection_retrier<'a, F>(
        f: F, 
        f_self: &'a mut Self, 
        f_m: &'a Message,
    ) -> BoxFuture<'a, Result<(), Error>>
where F: for<'b> ConnectFn<'b>

And replace f(f_self, f_m) in the body with f.connect(f_self, f_m). This avoids the need for the return type of f to appear anywhere in connection_retrier's signature, circumventing the issue we had before.

We can then replace our function send_and_expect with a zero-sized type SendAndExpect which implements ConnectFn, like so:

struct SendAndExpect;
impl<'a> ConnectFn<'a> for SendAndExpect
{
    type ConnectFuture = BoxFuture<'a, Result<(), Error>>;
    fn connect(&self, client: &'a mut Client, message: &'a Message) -> Self::ConnectFuture {
        /* the body of Client::send_and_expect goes here. `self` is ignored */
    }
}

With this, we can call connection_retrier like so:

Client::connection_retrier(SendAndExpect, &mut client, &message)

Which is pretty nice. The definition of SendAndExpect itself can be made even nicer, close to that of a regular fn, using macros, but this is left as an exercise to the reader.

Sidenote: You might think that it would be possible to have a blanket implementation of ConnectFn on suitable fn pointers and avoid the need for a new type, but sadly you would be wrong. You'll either run into the same issue with impossible where clause bounds or trait solver limitations when dealing with universally quantified types.

Sign up to request clarification or add additional context in comments.

10 Comments

ok I'm understanding everything you wrote. First, play.rust-lang.org/… I tried his solution but I couldn't make it work. Do you see what's wrong?
@Gatonito Yep, they just accidentally left in both the &self and the &Self references. Here's a fixed version that compiles: play.rust-lang.org/…
Now I get it. I don't see how it would be possible to still use client but not borrow from it in send_and_expect. I need to use it and since I'm passing by reference, I want to use it by reference. Is there a way that I don't know.
@Gatonito Nope, if you need to use the Client in the future you can't avoid borrowing from both arguments.
I understood the Boxed example. I'm not designing something performance critical, but I also don't like to use dynamic allocation. Unfortunately I'm forced due to async recursion, which is only possible with a Box return type as far as I know. So adding another box is not a huge problem, but I still don't like it very much.
|
1

The reason lies within these three lines:

pub fn connection_retrier<'a, T>(
    f: fn(&'a mut Self, &'a Message) -> T,
    f_self: &'a mut Self,

For f, you are telling the compiler that it will receive a mutable reference with a lifetime equal to that of the mutable reference that connection_retrier receives for f_self (in this case it is 'a).

When you change f's signature to f: fn(&mut Self, &Message) -> T, you are letting the compiler determine the lifetime, which it correctly does by assigning it a lifetime shorter than 'a and hence why it compiles.

9 Comments

I think I'm really stuck in the concept of pub fn connection_retrier<'a, T>( f: fn(&'a mut Self, &'a Message) -> T,. First I thought fn was the signature for a function generic over the lifetime of the arguments, but now I see it as a function that receives arguments with the exact lifetime 'a. However, I don't see why when I call f, f_self needs to be borrowed for 'a. It could only be borrowed for as long f needs it. Then it could be borrowed again after the match. Unfortunately I cannot use your fix, I really need the lifetimes of fn to be 'a because of..
Don't feel bad about it, the concept of lifetimes in Rust is not easy to grasp. Turn your stress in your favor and try to dig deeper on the subject. What you need to understand is that, in your code, the mutable reference to f_self will live as long as connection_retrier lives, and you are telling the compiler exactly this by giving it the lifetime 'a. On the other hand, f is called within connection_retrier, so any reference that f takes, must have a lifetime shorter than that of connection_retrier.
I get that f_self lives as long as connection_retrier lives. I don't get how a lifetime for the reference f takes must be less than that of connection_retrier I think any lifetime larger than or equal to connection_retrier would work
Please look at my new update. I changed the code a bit. You can see that doing your solution will not work because of how send_and_expect works now.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.