comparative-degree
To make a comparative form, one can add –er (as in ‘nearer’) in some cases or one can use the word “more” as a prefix (as in "more beautiful") in some others.
Is there any rule that says where can one add –er and where can one use “more” as a prefix?
@J.R. is absolutely correct, and has provided an excellent example of the kind of ambiguity that can result when basic comparative adjective grammar rules are not followed. But the basic, teachable, and, in an EFL/ESL context, extremely relevant reason that bigger is correct, and more big is ungrammatical as a comparative adjective is because those rules have not been observed. The rules for forming comparative adjectives are fairly straightforward: (1) for one-syllable adjectives, add -er, (2) for two-syllable adjectives ending in -y, change the -y to -i and add -er, and (3) for two-syllable adjectives not ending in -y and all three-or-more-syllable adjectives, use the form "more + adjective".
There are exceptions to these rules, such as fun, the comparative of which is more fun even though it is only one syllable. Another example is often, which is as commonly rendered comparatively using oftener as it is more often. But @Carlo_R was asking for the grammar rules, and I have summarized them.
SHORT VERSION:
Don’t use it.
LONG VERSION:
In your first example not quite better is not parsed as [not][quite better] but as [not quite][better]; that is, not quite is a fixed expression meaning ‘almost’, ‘falling just short of’. This is fine.
But this not quite does not behave the same way as bare quite. Quite ADJer is and not unknown, but it is very rare, comparable to very ADJer. Take a look at this Google Ngram, where I have multiplied the instances of quite better and very better by 1000 and they are still outranked by much better:
I note that your second and third examples are uttered by a native speaker of Italian and a speaker of Indian English.
There is as far as I know no reason why very and quite should not be used with comparatives; but they aren’t.
Best Answer
This is not exactly an answer. This is an explanation of how one native speaker thinks about this.
Hopefully this will stimulate someone to post a better answer, maybe disproving what I say below.
Not that rule
I do not think about -er vs. more in terms of a rule regarding the number of syllables. Some people say that one-syllable words get -er and multi-syllable words get more, except all words ending in -y get -ier. That's probably right most of the time, but it sounds to me like a pseudo-rule similar to "i before e except after c", another rule that's often right but goes against the spirit of the language. English is more origin-, precedent-, and conflict-oriented than rule-oriented.
It didn't me long to think of funner, which is not a standard English word. Sometimes people say it, but even the first time I heard it (as a teenager), I cringed. To my native ear, it sounds ridiculous, suggesting that the person saying it just doesn't "get" English. Even people who say it know it sounds silly. Unfortunately, I don't know how I know that.
I think that when you are learning a foreign language, you should mainly be interested in how to perceive it the way a native does. So, for this reason, I think you shouldn't think about this in terms of a rule that goes by the number of syllables, even if it's approximately correct. Somehow, through exposure to many examples in a lot of real usage, natives figure out which words don't sound right with -er. Learning it that way will take a while, but what you learn will be real—the spirit of the language, not a "rule".
Some exceptions
Here are some exceptions to keep in mind as you're considering hypotheses:
One-syllable adjectives that don't take -er:
There may be a legitimate rule that past participles can't take -er.
Far sort of takes -er, but it becomes farther or further. Bad doesn't become badder (except in lower-class slang); it becomes worse. An interesting and unusual one is aft.
One hypothesis is that if the adjective is also a noun that you can "have", as in "have some fun", then it can't take -er, that's refuted by calm.
Multi-syllable adjectives that don't end in -y but can still take -er:
Another observation: most adjectives that can take -er can also take more, but not all. For example more fast sounds weird--though you must say it in a weird comparison like "more fast than inquisitive".
A hypothesis
A hypothesis that comes to mind is that Anglo-Saxon adjectives naturally make comparatives with -er and Latinate or loan words don't. Anglo-Saxon words usually have a different "feel" than imported words, which might affect how we alter their forms even if we don't consciously know their etymologies. Adjectives from Latin usually have two or more syllables (like "terrific", "circuitous", "accidental"). And Anglo-Saxon adjectives are often one-syllable (like "red", "clean", high") or end in -y. Unfortunately, this hypothesis runs into the ground with fun.
You might think this hypothesis is, er, stupider than the number-of-syllables hypothesis, but it might actually be in the vicinity of a genuine explanation for why some words feel natural with -er and others can only take more. This would be an explanation, not a rule. An explanation can shed light on cases you haven't encountered yet without pre-deciding them; other factors may enter into other situations.
Sure enough, this ngram shows that stupider got started long after more stupid. Apparently, the need to compare levels of stupidity was so great that people granted stupid a sort of honorary Anglo-Saxon status in order to use the more-convenient comparative -er.
And once stupider is in, by analogy vapider eventually starts sounding more acceptable. Vapider doesn't show up on Google ngrams, but dictionaries accept it.
This is what I mean by English being more origin-, precedent-, and conflict-oriented than rule-oriented. Word with different origins follow different rules, conflicts result, they get resolved, the resolutions set new precedents that people apply to other words, and on it goes.
Regarding other factors that affect some other words, the word fun might not take -er because even though fun is an adjective, we usually use it as a mass noun when speaking of degrees: "that was a lot of fun", "we could have a bit of fun". This might explain the feeling of unease at giving it -er. If the noun is felt to be primary, that may conflict with -er; but more works with mass nouns as well as adjectives.
Evidence against this hypothesis would be two-syllable Anglo-Saxon adjectives which don't take -er.