Your 'world' will dictate for the most part how magic is viewed.
NPCs - how other characters react to mages,spells, magic items etc. are a good way to add a sense of how magic is regarded in your game world. A glowing item or demonstration of magic frightens or amazes passersby.. when word reaches a towns ruler of adventurers with a unusual item or abilities she requests an audience to see for herself.
Abundance vs Scarcity - The less there is the more novel magic becomes. If every goblin enountered is wandering around with enchanted daggers and there's a magical supplies merchant in every town its very difficult to maintain a sense of wonder. On the other hand if magic items can only be found after herculean efforts (slaying the dragon etc..) or in exotic locations (in a secret chamber beneath the ancient ruins of a legendary castle ).
Mysterious behavior - Sure that longsword gives you a +1 when fighting orcs. It also cries out the name 'Larissa' under a full moon, Or simply dissapears from time to time...
Your mages 'Magic Missile' spell, instead of appearing as glowing arrows every now and then manifests itself differently (a flaming spear, or icy daggers) perhaps modifying its game affect slightly (and unexpectedly!)
Description - be vague in the descriptions/naming of spells/items, the Call of Cthulhu spell list is deliberately obtuse in places (and encourages the Keeper to do the same), an Identify spell for a wand of fireball gives the mage a vision of enemies perishing in flames or suchlike.
That depends on your definition of "illogical." While I've personally never been a fan of Vancian magic due to cognitive dissonance ("It's like only being able to use the quadratic formula once on a test!"), there are in-fiction explanations for it.
Spell Preparation
What's important to understand is that the spells are not simply memorized, recalled, and then forgotten. They are specifically prepared, and then expended. The wizard remembers how to cast the spell, but is no longer prepared to do so.
"Memorizing" a spell is a misnomer. "Preparing" is the more appropriate word.
I don't have core rulebooks for D&D floating around at the moment, but this quote from the SRD says enough, I think:
[the prepared spell] remains in her mind as a nearly cast spell until she uses the prescribed components to complete and trigger it or until she abandons it.
In other words: during spell preparation, the wizard performs some ritual that "casts" his spells into the ether. He can either "cast" the same spell many times, or spread his allotment out over many spells.
Later, he performs some action (specified by the spell) that triggers the spell. That particular "casting" of the spell is completed, and is no longer available until the wizard is able to prepare it again.
In this way, the spells follow an internally consistent model, without relying on what the wizard should or should not be able to remember.
The Metaphor of the Surgeon
Suppose we use a surgeon in place of the wizard, and performing surgery in place of casting spells.
Spell preparation for our surgeon would be the act of preparing his tools: sterilizing them, arranging for the proper facilities to be made available, scheduling staff, and so on.
When the surgeon runs out of prepared tools, he can no longer perform surgery. He hasn't forgotten how to perform the surgery, but nonetheless, he is unable to proceed until he's had a chance to prepare again.
If the surgeon prepares tools for a single surgery, he can perform surgery once. If he prepares for two, he can perform it twice, and so on.
The Man Himself: Jack Vance
What discussion about Vancian spellcasting would be complete, without some input from Jack Vance himself? The following are a number of descriptions of spell casting, all taken from Mazirian the Magician:
On memorizing spells:
...They would be poignant corrosive spells, of such a nature that one would daunt the brain of an ordinary man and two render him mad. Mazirian, by dint of stringent exercise, could encompass four of the most formidable, or six of the lesser spells.
...Mazirian made a selection from his books and with great effort forced five spells upon his brain: ...
On having used a spell:
...The mesmeric spell had been expended, and he had none other in his brain.
On having spells at the ready:
"You may in any event, Mazirian. Are you with powerful spells today?"
I have yet to see Vance refer to the spell casting process with the terms "memorize" and "forget." So far, it's always been oblique, like the passages above.
I think that memorizing and forgetting is a very reasonable interpretation of the lines above. But I also think that their vagueness is important. When I read them, I think of an other-worldly force that the wizard has to hold ready with some kind of psychic "muscle" (like telekinesis). To me, the spells are "in his mind" in the sense that someone possessing you is "in your mind," not in the sense of memories.
But that's purely interpretation.
Jack Vance vs. D&D
In reading Vance, I found the Vancian system much more appealing than I ever found it in D&D. I've put a little thought into it, and here are some of the key differences that I've been able to spot. Your mileage may vary; some of these might not apply to you.
Vancian Magic Takes Skill
Vance's spellcasters take hours of preparation combined with years of experience to select a handful of spells that they can use to maximum effect. It also helps that the author of the story actively wants them to use exactly all of their spells.
To recreate this on the table-top takes an enormous amount of skill on the part of both the player and the DM. For the player's part, they need to know the spells well enough to make a good selection, need to know the world well enough to know what they're going to encounter, need to be clever enough to make spells fit unexpected situations, restrained enough not to spend spells heedlessly, and patient enough to thoroughly plan and research their upcoming adventure.
On the part of the DM, the DM needs to present a world consistent with published materials, vary encounters enough that the optimum spell build isn't just "Fireball, Fireball, Fireball, Magic Missile," and find some way to give the players the ability to do advanced legwork without having it take over the plot (because after all, if we wanted to spend an adventure planning the adventure we'd be playing Shadowrun).
Vance's Spells are Powerful
In Vance, one spell solves most problems. Attack spells kill their targets. Protection spells render the user invulnerable. Blessings can last days or years.
There's More to Vance's Wizards than Spells
Vance's wizards spend a lot of their time not casting spells. Whether they fall back on powerful magic items (on the scale of "I am immune to all magic"), magic-that-isn't spells (there's a number of instances of wizards "muttering counter-curses" or otherwise using "magic" that they haven't prepared), or even engaging in outright sword-play, Vance's wizards have options when a spell doesn't suffice.
Vance's Wizards are Limited to Concurrent Spells, not Spells Per Day
This is a matter of "gameplay" vs. "realism." Vance's wizards have a limit on the number of spells available at one time, without easy access to replenish their spells. D&D wizards are limited to the more "gamey" spells per day, with ready access to their spell books.
Best Answer
Preliminary
I think there a couple of preliminary points that need to be addressed. This is marked as system agnostic, and that is probably a good call in the sense that neither the question nor this answer are tied to any particular setting, but it does not apply to all settings. Some settings make it very easy to make magic feel "science-y" and others makes it nigh impossible either deliberately or just by not having things well defined.
Second, what makes something feel like science, especially in the way this question seems to be asking is being well defined, and susceptible to something akin to the scientific process, not using scientific sounding words. In fact, scientific sounding words might make things worse in terms of realism or making something feel like science. They can lead to someone with real science knowledge leaping up and saying "That's not how that works at all" (my wife gets really mad when I do that with Star Trek) or "That might work, but what about the side effects?"
Something which is predictable and can be studied experimentally feels like science.
Which brings me to the core of my answer: To make magic feel like science you let it be consistent. Shadowrun's Hermetic magic (my knowledge comes from 2e, I don't know if this has changed) felt that way to me. There was a whole background theory of mana and astral space and their interactions that felt like something a scientist could study. The results were predictable, a magic theory skill existed, and some of the fiction around it suggested that magic theorists worked a lot like scientists and used a lot of math. It felt a lot like science, though it used magi-babble instead of science babble.
In fiction, the Iron Druid Chronicles come to mind, at least in terms of druidic magic. The basics of how druidic magic are laid out in the first book of the series and never deviated from. After the first book, the druids occasionally pull out new tricks, but they always flow form the principles laid out in the first book and always feel consistent. The main character also spends a lot of time talking about how his studies of science have helped him use his magic better. Druidic magic is predictable, and can be informed by knowledge of other fields of science. This makes it feel a lot like a science.
What makes magic feel un-scientific?
It might help to look at the opposite end of it. To make magic not feel like science, you deliberately leave it vague, unpredictable, and poorly understood even by its users. It then becomes hard to study (in any conventional sense of the word study) and mysterious. It might even be a living thing with something of a mind of its own. Magic in The Last Unicorn felt magical and was certainly not scientific because of that.