They still do
Do English teachers no longer teach that this is a grammatically incorrect thing to do?
One day in first grade, in the United States, my teacher told the class that it's wrong to start a sentence with a conjunction. The example she used was "And". She gave a straightforward reason: since a conjunction joins two things, it doesn't make sense to start a sentence with one, since nothing has yet been said that could be joined to something else.*
A few minutes later, we came across a sentence in a book we were reading aloud from that started with "And". Someone pointed it out. The teacher explained, "Well, you can only do it if you're an author."
Lesson learned. The rule is fake.
Not so fast
I could open up the King James Bible to a random page and point out sentences that begin with "And". Anyone with access to Google Books or Google Ngram could beat you over the head with a thousand sentences starting with any conjunction you choose. I hope you would find that unconvincing. People violate subject-verb agreement all the time, it's easy to find examples in "corpora" of any grammatical violation you like—but that doesn't mean they aren't grammatical violations. I'd like to give you something genuinely convincing: an explanation of the grain of truth within the rule, and an explanation of why it makes grammatical sense to violate it.
Sentence fragments and sloppy writing
First, let's look at the real basis of the rule.
When children are learning to write, they often don't understand the difference between a complete sentence and a sentence fragment. They don't understand yet that written language is not merely a transcription of speech, but a refined, more formal version of the language. In written language, we normally expect a sentence to express a proposition clearly: to affirm or deny a predicate of a subject. Conjunctions play a simple and straightforward role in this: they join elements of sentences together. "You can have cake and ice cream but not coffee or cigarettes."
As my first-grade teacher said, there is something odd about starting a sentence with a conjunction when there's nothing to join yet. When children are just learning to write, requiring them to use conjunctions only for their primary role as joiners of elements within sentences can be a useful, temporary constraint. Temporary constraint is a very effective teaching method for many skills: by focusing you on one way of doing something, you master it much more quickly than without the constraint; and when you release the constraint, you gain keen insight into the full range of what's really possible beyond it.
People who haven't mastered the strict use of conjunctions, or who don't know a sentence from a sentence fragment, often put conjunctions at the beginnings of sentences carelessly and sloppily, as in the examples you offered. Many real-life examples are much worse, of course. The authors of such writing haven't gathered and organized their thoughts in way that is suitable for clear writing. To a reader, it comes across as incoherent and careless. Much writing that starts a lot of its sentences with conjunctions is quite tiresome. Just have a look at YouTube comments or the Book of Mormon.
Beyond the constraint
Another book I read when I was a little boy told a story about a princess who was looking for a husband. She rejected one of her suitors because he began every sentence with "I". That little tidbit teaches much wisdom! People who start every sentence with "I" quickly become tiresome. It's a thoughtless, inconsiderate, unempathic way of speaking.
But does that mean that starting a sentence with "I" is ungrammatical? Of course not. Grammar is only the nuts and bolts of how words connect and inflect to make meaning. What you choose to mean is outside the scope of grammar.
Even though conjunctions at the start of sentences can be tiresome, they serve an important grammatical role there:
They join elements of discourse at a higher level than within a sentence.
When used at an opportune moment, to signal a reversal from a previous line of thought or to deny the expected outcome of a sequence of events just described, "But" at the beginning of a sentence can make clear, well-organized rhetoric. Upon seeing a sentence starting with "But", outside its usual role as signaler of an exception within a sentence, the reader understands that the new sentence as a whole will describe a reversal or exception to the preceding sentences.
"But" at the start of a sentence is most commonly useful to deny an expectation that arose from several preceding sentences, or to introduce a compound sentence containing several exceptions, but of course that can't be a rule. When to use it is a rhetorical choice, subject simultaneously to all the considerations of rhetoric. Its grammatical role, as joiner and organizer at a higher level than within a sentence, is straightforward.
* Obviously, that only applies to coordinating conjunctions, not subordinating conjunctions. Let's not split hairs.
Best Answer
Your theory is correct. But that leaves us with 1) and 2).
Between I and me, we can decide like this: I corresponds to we and me corresponds to us. 'He was with us' and not 'He was with we.' So, 2) is the correct option.