A very thoughtful and hard question indeed; it pushed me to research a bit on the subject Continuous Conditional.
Your first example is a past perfect continuous sentence. In general, it is used to indicate an action was happening before another action happened. However, it can also be used to indicate past unreal condition. For example:
If I had been talking to him when he said that, I would have punched him in the face.
But fortunately, he was not talking to him when he said that and that's how he missed my punch.
So according to this theory, your first sentence which is
If I had been with you, I might be taking care.
is absolutely meaningful and grammatical.
Now, there is a vital fact when using these type of constructs as described by data.grammarbook.com,
When talking about something that didn’t happen in the past, many English speakers use the conditional perfect (if I would have done) when they should be using the past perfect (if I had done).
For example, you find out that your brother saw a movie yesterday. You would have liked to see it too, but you hadn’t known he was going. To express this, you can use an if – then clause. The correct way to say this is with the past perfect in the “if” clause, and the conditional perfect in the “then” clause:
Correct: If I had known that you were going to the movies, [then] I would have gone too.
The conditional perfect can only go in the “then” clause — it is grammatically incorrect to use the conditional perfect in the “if” clause:
Incorrect: If I would have known that you were going to the movies, I would have gone too.
More examples:
Correct: If I had gotten paid, we could have traveled together.
Incorrect: If I would have gotten paid, we could have traveled together.
Correct: If you had asked me, I could have helped you.
Incorrect: If you would have asked me, I could have helped you.
The same mistake occurs with the verb “wish.” You can’t use the conditional perfect when wishing something had happened; you again need the past perfect.
Correct: I wish I had known.
Incorrect: I wish I would have known.
Correct: I wish you had told me.
Incorrect: I wish you would have told me.
Correct: We wish they had been honest.
Incorrect: We wish they would have been honest.
So this theory, in a nutshell, says you can't use "would" part with the "if" clause, rather it should be used in the result clause. So, according to this theory, your second sentence stands incorrect.
OK - my background is as a native British English speaker.
1) If I knew you were coming, I would've arrived from work earlier.
This is perfectly easy to understand, though as you correctly say, the grammar is not correct. "Had known" is really needed. Still, some native speakers will say this.
2) If I knew you would/were gonna come, I would've arrived from work earlier.
I find use of the contraction "gonna" the worst part of this construction. I think it may be more acceptable in American English. The same comment as above applies, of course, to the verb.
3) If I knew you had come, I would've arrived from work earlier.
This sounds unnatural and needs a slight pause in thought to understand. I'd suggest you avoid it.
To a British English speaker all of these constructions sound a little jarring, but in informal situations, native speakers commonly use constructions which are grammatically incorrect. There is a danger in formal situations, that a native (British) English listener will assess your level of education or ability, to be less than it really is, if you use constructions like this. That will also depend on factors like the age and background of the listener.
Incidentally, and again in British English, the contraction "I'd have" is probably more usually used, than "I would've" unless there is emphasis on the "would".
Best Answer
(Note that people come to parties, as in attend a social gathering. Coming in a party would be incorrect, or suggest another meaning of party, which now mostly is used in the context of a group or party of people arriving at a restaurant to be served.)
This means that the party has passed, and the person you're talking to did not attend the party. The speaker is saying that if the other person had come, they would have enjoyed it. They are also saying that they did not have a good time at the party.
This mean that the party hasn't happened yet (or is still happening), and the speaker is saying that they will enjoy the party if the other person comes. This could be a slightly awkward way of expressing that they hope the other person comes (since it seems to hinge their enjoyment on the other person coming). Some more casual ways of expressing something like that would be, for example: