The way we do it is to start with the data. Most of our sections have only one type of question, but several can have two or more types of question. Each question type has a factory that can take the JSON that started out as XML and was run through a conversion process and adds in all the "hooks" each question needs. We have something called a "questionBuilder" that looks at a questionType property on each section and looks in the $injector for questionType + 'Factory'.
Edit: You can think of a builder as like when you order something from Amazon. Amazon has no idea how to make a coffee pot or a can of tennis balls. What it does have is a long list of suppliers that do know how to make these things. So when you order from Amazon, it goes in its list of factories that supply these things, pulls out the one that can make a coffee pot, and asks the factory for a coffee pot. It then sticks it in a box and ships it to you.
Angular is like the UPS. It doesn't know or care what's in the box. When you get to the View, you pull out the boxes that are the questions you care about at the moment. Your View understands the question type (just like you know how to use that coffee pot, or you can find out).
Conveniently, Angular is also like the list of suppliers (or the $injector is). Anything the $injector knows about, you can pull out in your builder, which is how you can have whatever factories you need without cluttering up your code with a bunch of logic to hard-code the dependencies on particular factories. You're looking in the $injector for the factory that matches the question type specified in the data.
If there is more than one question type in the section, we put down "composite" for the questionType. The compositeFactory looks at a property of each question to figure out where to look in the $injector for the factory. It also looks at information in the section that tells it to add a property to each question that tells a view further down the line which partial to use in an ng-include.
Each question type has its own feature directory that contains a module that says what routes match what views that go with that feature (some of our question types have different variations based on client needs). The composite feature has single view that basically just pulls in the ng-include partial that was added in the factory earlier. The partial might be either directly something from one of the feature directories or a wrapper for one of the feature directives, depending on exactly what we need and how it was originally built.