[SalesForce] Why is ant trying to deploy components which are not described in the package.xml

This is kind of linked to my other question

How to Integrate github and Ant migration tool?

Here is my package.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <types>
        <members>*</members>
        <name>ApexClass</name>
    </types>
    <types>
        <members>*</members>
        <name>ApexComponent</name>
    </types>
    <types>
        <members>*</members>
        <name>ApexPage</name>
    </types>
    <types>
        <members>*</members>
        <name>ApexTrigger</name>
    </types>
    <types>
        <members>*</members>
        <name>CustomLabels</name>
    </types>
    <types>
        <members>*</members>
        <name>StaticResource</name>
    </types>
    <version>31.0</version>
</Package>

Here I did not specify the CustomObjects type intentionally because we are always working on the classes and triggers and there will be no changes in the Objects,profiles,permission sets..So I deleted those parts from the package.xml

And when I do the command ant deployUnpackaged It is trying to deploy everything to server and reports the below errors:

objects/Case.object (Case.OwnerId) -- Error: Not in package.xml

Any idea??

Thanks in Advance..