There is very little difference; one of them has a more formal register than the other.
In Britain, we normally use the preposition phrase "out of", and that's accepted in formal English. In informal use, some dialects regularly omit the "of", and it is understood in context. I don't recommend this form when you are writing.
Other regions appear to have this distinction reversed.
As an alternative, you could also consider, "He gazed through the window", but that is slightly different, as that can mean gazing in to rather than out of.
If you are speaking literally, "in" usually carries the meaning of "surrounded by", or "completely enclosed":
"There's a fly in my soup."
"The dog is in the doghouse."
"I made an error in that sentence."
and "on" implies "attached to" or "touching":
"I wish I were a fly on the wall."
"They were on bikes."
"We put another coat of paint on it."
But it's a little bit trickier when you are speaking figuratively, using idiomatic expressions, or discussing abstract ideas where the physical concept of "completely surrounded" vs. "merely touching on the outside" doesn't really apply.
When it comes to set expressions like "on the bus", "in synch","on the team", or the cases you mention, "on the test" or "in this chapter", often these just have to be learned by rote.
Not to mention confusing pairs like:
"on fire" vs. "in flames"
and
"on the money" vs. "in the money"
Many of these are not subject to reason at all, just custom.
Sometimes it will even vary by region. Where I come from, people "stand in line", but my husband always refers to people "standing on line". He claims that's the way they say it in New York City.
Another example is the comment of Ronald Sole above, where he says, "In the test is more common than on the test." But in the variety of English spoken where I live, (California, USA), it is just the opposite. We would be much more likely to ask, "How did you do on the test?" than "in the test."
Best Answer
I don't think there's a difference. For example, these two sentences mean the same thing to me:
I think that technically, "about" could be considered more correct. For example, The Free Dictionary uses this as an example for the idiom "change your mind": "After the interview I completely changed my mind about her abilities."