[SalesForce] Correct approach uploading File using REST API (file size 100 MB)

Here is some Java code that uploads Attachments that you may be able to use as a starting point:

package com.claimvantage.ant;

import java.io.File;
import java.io.IOException;
import java.io.OutputStream;

import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.methods.multipart.FilePart;
import org.apache.commons.httpclient.methods.multipart.MultipartRequestEntity;
import org.apache.commons.httpclient.methods.multipart.Part;
import org.apache.commons.httpclient.methods.multipart.PartBase;

public class AttachmentExample {

    private class JsonPart extends PartBase {

        private byte[] bytes;

        public JsonPart(String name, String json) throws IOException {
            super(name, "application/json", "UTF-8", null);
            this.bytes = json.getBytes("UTF-8");
        }

        @Override
        protected void sendData(OutputStream os) throws IOException {
            os.write(bytes);
        }

        @Override
        protected long lengthOfData() throws IOException {
            return bytes.length;
        }
    }

    private String baseUrl;     // Initialization not shown here
    private String sessionId;   // Initialization not shown here

    /**
     * Create attachment SObject from its JSON populating its Body from a file at the same time.
     */
    public void createAttachment(String attachmentJson, File attachmentFile) throws Exception {

        PostMethod post = new PostMethod(baseUrl + "/services/data/v23.0/sobjects/Attachment");
        post.setRequestHeader("Authorization", "OAuth " + sessionId);
        Part[] parts = new Part[] {
                new JsonPart("Json", attachmentJson),
                new FilePart("Body", attachmentFile)
                };
        post.setRequestEntity(new MultipartRequestEntity(parts, post.getParams()));
        try {
            new HttpClient().executeMethod(post);
            if (post.getStatusCode() == HttpStatus.SC_CREATED) {
                // Logic for OK
            } else {
                // Error handling logic
            }
        } finally {
            post.releaseConnection();
        }
    }
}

A few more details here.

I eventually ruled out being able to use base64, this is because the API does not seem to support the "Content-Transfer-Encoding: base64" header.

Further to this, when I tried sending raw binary data I had a problem where RESTed (the native Mac REST client that I was using) was escaping all of the UTF8 characters before sending the request.

When I used curl with the --data-binary flag the image uploaded and worked with no problems. I am kicking myself now because the examples in the documentation all mentioned using curl!

Thanks for all of those people who tried to help!