Setting fields in Django form, when using ModelForm

My model is that I have a list of jobs, and each job can only be done by a subset of users (ManyToManyField). People can then submit job requests, and assign the job to someone from the subset of people who can do the job:

class Job(models.Model):
    ...
    users = models.ManyToManyField(User)

class Job_Request(models.Model):
    ...
    job = models.ForeignKey(Job)
    assigned_to = models.ForeignKey(User)

I then created a form using ModelForm, to allow people to edit the job request, to reassign the job to someone else. My problem, is that ModelForm creates a menu for the "assigned_to" field in the form, which lists all of our users. I only want it to show the subset of users that can do that job. How can I do this?

Below is my forms.py, where I tried setting the assigned_to field to the subset of users that can do the job, but I don't know the correct syntax. The following is definitely wrong, as it creates an empty menu. How can I do this, either in the form, or the template? Thanks.

class EditJobRequestForm(ModelForm):
    def __init__(self,  *args, **kwargs):
        super(EditJobRequestForm, self).__init__(*args, **kwargs)
        self.fields['assigned_to'].queryset =
            User.objects.filter(username__in=self.instance.job.users.all()]