I agree that we need to have a pilot study of a small group of people to see if there is any immediate problems that happen. We have to then ask the questions of time spent in a specific section of the app, how long can we retain their attention, and of course where does the app fail.
I also agree that after finding the answer to those problems, to expand the amount of participants to ensure that the problems do not arise again.
Again I do agree that we need to have those follow up questions of simplicity and functionality. The one thing that I would add to the questions is visual appeal. Does the app have visual appeal?
Other than that question, I totally agree with what Allison said for the testing.