Yes, both effects can apply to the same spell.
There's nothing in the wording of either feature to indicate that it excludes other features adding damage as well. Many other combinations of features can work this way -- a barbarian's damage bonus from Rage is cumulative with their Strength damage bonus and other bonuses such as the Dueling fighting style. The word all of these have in common is add.
Contrast this with Thirsting Blade, which says:
You can attack with your pact weapon twice, instead of once, whenever you take the Attack action on your turn.
It doesn't say "you can attack one additional time", so the reason it doesn't stack with Extra Attacks is that neither one adds attacks, they both just change the number of attacks from one to two.
This is also similar to how the barbarian and monk Unarmored Defense features don't "stack" with armor. Armor provides one way of calculating AC, Unarmored Defense provides another. A given character can only use one.
So, in general, effects or features that let you add something to a roll are cumulative, but effects or features that change a value are not.
Rules as written, it is left to DM or the table for interpretation
Rules as written what you're supposed to do is look at Nondetection and Portent and decide for your table how it works. You're supposed to look at what the intent of Nondetection and the intent of Portent is supposed to be, which means you'll have to read and decide for yourself based on how you understand the rules and what works best for the fun and enjoyment of your table. That is 5e's rules philosophy.
I don't buy the "It doesn't say it is or it isn't, therefore it isn't." You can just as easily say, "It doesn't say it is or it isn't, therefore it is." Lack of evidence doesn't prove anything. The game rules do not claim to be comprehensive, and this is the natural consequence of that. This is the entire point the "rulings not rules" idiom is making. There is no burden of proof on Portent or Nondetection or the PHB as a whole to provide an answer. While that line of reasoning was implicit to 3e and 4e -- both editions that sought comprehensive and complete rules sets -- 5e actively does not do that. 5e D&D is often intentionally vague so as to leave rules open for interpretation and the system intentionally doesn't use keywords or raw mechanics. It does this so that players and DMs have the explicit freedom to interpret the rules for themselves and do what makes sense for that interaction. The designers know they can't possibly foresee every interaction in the rules, so they no longer try.
There is no clear answer by design. Yes, this means that asking, "What is RAW?" on RPG SE for 5e D&D is often a pointless exercise because the answer you should often get is, "RAW it depends on your table." This is why there are so many conflicting answers on RPG SE for 5e questions and why Sage Advice contradicts itself so often.
The rules are less important than the game. What Mearls and Crawford want is for players and DMs to stop asking WotC how to play and just make a decisions and play for themselves. If you make a mistake, admit it and make a correction. It's no less destructive than doing nothing until WotC makes a decision and then maybe changes their mind later.
You're expected to look at whole picture that the rules are giving you and to make a judgement call on what feels the most consistent and correct for your table. Is it reasonable for Nondetection to block Portent? Sure, it almost certainly is divination magic given that it's an ability of the Diviner subclass. How about a Ranger's Primeval Awareness? Well, that works like a spell, even consuming a spell slot, and it would have to be divination magic given the distance, so sure. How about a Paladin's Divine Sense? Hm... possibly, it's pretty close to Detect Good and Evil, but it's really described as the Paladin's senses. A Warlock's Devil's Sight? Hrm, hard to say. It's got elements that only True Seeing can accomplish, and Nondetection probably blocks True Seeing, but it's basically an improved Darkvision spell and that's not even Divination. So maybe partially? A Barbarian's Feral Instinct? Eh, that doesn't seem right, it's not magical. A Rogue's Blindsense? Yeah, probably not unless Blindsense is supposed to be magical, but I don't get that impression.
Best Answer
This Will Lead to Power Creep
Whether or not it "breaks your game" depends on what you already allow at your table (unlimited multiclassing vs "story justified" multiclassing vs. no multiclassing, feats, etc.), but it definitely allows single class characters to reach heights they otherwise couldn't if they are specializing in one thing (i.e., min-maxing).
There are many different things a PC can optimize their character for. One of these is "Ambush" damage, or damage in the first round of combat (preferably against surprised opponents). I will focus on this as one relatively easy to understand example and a couple of subclasses, but the problem absolutely exists for almost any optimization objective you can imagine (for any class that has subclass features related to that objective), for many classes.
Both Rogue and Ranger's can be built using these rules to be far more effective at round 1 damage (especially surprise rounds) + combat generally than any individual published subclass of those respective classes. The features they give up (while thematic) are largely worthless for this optimization target and are thus precisely the types of trades a min-maxer would look for.
Example 1
Class: Rogue
Race: Harengon (+ Proficiency to initiative)
Subclass level 3: Assassin's Assassinate (automatic crits against surprised enemies; advantage against anyone who hasn't taken a turn yet).
Subclass level 9: Phantom's Tokens of the Departed (even without the ability to use level 3 feature Wails of the Grave, this gives advantage to 2 types of saving throws + out of combat utility)
Subclass level 13: Scout's Ambush Master (Adv. on Initiative + you and all allies have adv. against first creature you hit during first round until start of next turn)
Subclass level 17: Thief's Thief's Reflexes (2 turns in first round of combat, 1 at normal intiative + 1 at initiative - 10)
Compared to Assassin, this build has an overwhelming combat advantage. It replaces 2 ribbon/flavor abilities at levels 9 and 13 with combat utility: It more reliably goes first (thanks to Ambush Master) allowing it to trigger both parts of the strong Assassinate level 3 feature more regularly, it shores up some weaknesses by giving the Rogue the ability to have persistent advantage on Constitution and Death saving throws, and it even has some supporting qualities buffing party damage via Ambush Master
Thief's Reflexes further improves round 1 damage in multiple ways:
You could also substitute in Scout's Sudden Strike to be able to use your auto-crit sneak attack twice at your normal initiative (instead of waiting) and allow your Ambush Master debuff to last longer, but you give up the ability to sneak attack the same target twice if that's important.
Compared to Scout (who also has features that help with surprise round attacks) you are also up multiple combat relevant features, and in particular Assassinate is a massive damage boost beyond what a normal scout could achieve without it.
Example 2
Class: Ranger
Race: Bugbear
Subclass Level 3: Gloom Stalker's Dread Ambusher + Umbral Sight (Permanent invisibility vs darkvision, extra attack w/extra damage round 1 of combat, bonus to initiative)
Subclass Level 7: Horizon Walker's Ethereal Step (Ability to sneak/slip past even enemies who can see invisibility or have blind sight to set up ambushes)
Subclass Level 11: Hunter's Multiattack (unlimited targets within 10 ft of a point) or Horizon Walker's Distant Strikes (Extra attack as long as you attack--not hit-- multiple targets w/attack action, no radius limit)
Subclass Level 15: Horizon Walker's Spectral Defense (Defensive Reaction you can choose to use only if you're actually going to be hit by attack, halve damage taken)
Compared to Gloom Stalker (who gets a weaker version of extra attack at level 11 that salvages one miss, which hopefully doesn't happen much when attacking with advantage), you replace it with a much stronger version of extra attack (either unlimited within radius, or a third attack with conditions + also 10 ft teleport per attack). You also replace a purely defensive Subclass level 7 ability with one that can set up ambushes and help you get the surprise round you rely on. Level 15 is the least impactful choice, but I'd still prefer a reaction that I can choose to use after knowing whether I'll be hit or not vs. one I use early without knowing (especially if I have other uses for reaction).
Compared to Horizon Walker (who we've stolen multiple features from), we stack a bonus to initiative + extra attack in turn 1 + extra source of advantage, all while keeping the same amount of extra damage the Horizon Walker subclass feature would provide (1d8) AND dropping the requirement to use our bonus action for the damage.
Compared to Hunter the level 3 feature is on another level: Dread Ambusher + Umbral Sight provides another additional attack, source of advantage, match the damage bonus (for round 1), and also gives an initiative advantage.
The ability to both add absurd numbers of attacks (Via Hunter's Multiattack) and also an easy source of advantage AND initiative (Gloomstalker's level 3) combines insanely well with Bugbear's surprise attack feature to weaken whole armies (usually focusing down 1 enemy at a time is ideal, but if you have a wizard following up with an AOE or similar, this can be an effective party tactic).
Multiclass Note
If you do allow multiclassing, these options don't disappear. You can take the upgraded Ambush Rogue to level 17 and stack Gloomstalker on top.
Or you can take Gloomstalker to 15 and add Assassin in as well.
In both cases, you've got the the full power of the "flexible subclass" house rule adding to the shenanigans multiclassing already permits, not just as an alternative to it.
But does it Break the Game?
If you have a specific thing you're worried about, asking a more targeted follow up question including what you consider "game-breaking" could give a specific answer to that question. It would require details like the level range you anticipate your parties playing, whether it's a published adventure with known enemies, whether you are allowing common optional rules like feats or multiclassing, etc.
If you have a very clear target like "could a party of level 7 trivialize the final boss of campaign X with this houserule", "could a single level appropriate character solo the final boss of campaign X with this houserule", or "Could a character achieve higher resourceless DPS with this houserule than under typical multicalssing rules", that type of question would be easier to answer definitively than a general question about "game breaking" in an abstract sense.