[SalesForce] Timeout error when posting blob data to the REST API from a VF page

I am attempting to use a newly added feature of the REST API which allows insert and update of blob data. I am doing so via the ajax proxy and I have been unsuccessful in posting a new blob Document to the org from a VF page. SFDC Partner support has also been unable to provide resolution or insight into the issue.

Feature Documentation:
REST API – Insert or Update Blob Data

Per SFDC support, "[t]his feature is already live and available for all the Orgs that have the REST API enabled", I can confirm that this is the case and my org does have this enabled.

After being unsuccessful with the VisualForce implementation, I used cURL in order to verify that I had a valid payload for the request and that all of the parameters were correct. For this troubleshooting effort, I replicated the documentation example exactly.

Using a valid SessionId from the VF page in the cURL request Authorization header, the Document was inserted correctly into the org into the 'Shared Documents' folder, per the ID in the payload. The successful request and response of that transaction:

cURL

Request

POST https://na15.salesforce.com/services/data/v23.0/sobjects/Document/ HTTP/1.1
User-Agent: curl/7.26.0
Host: na15.salesforce.com
Accept: */*
Authorization: Bearer 00Di0000000ZEt5!ASAAQAI3zZNJH0S6T4ybIy1Vo2kTo... [snip]
Content-Type: multipart/form-data; boundary="boundary_string"
Content-Length: 448

--boundary_string
Content-Disposition: form-data; name="entity_content"; 
Content-Type: application/json

{"Description":"Marketing brochure for Q1 2013","Keywords":"marketing,sales,update","FolderId":"00li0000000ujbN","Name":"Marketing Brochure Q1","Type":"text"}

--boundary_string
Content-Disposition: form-data; name="Body"; filename="fakefile.txt";
Content-Type: text/plain

This is the content of my fake file

--boundary_string--

Response

HTTP/1.1 201 Created
Date: Tue, 04 Jun 2013 22:22:36 GMT
Location: /services/data/v23.0/sobjects/Document/015i0000000IYIsAAO
Content-Type: application/json;charset=UTF-8
Content-Length: 54

{"id":"015i0000000IYIsAAO","success":true,"errors":[]}

VisualForce, ForceTK & Ajax Proxy

I have a minimal VF page which has references in it to jQuery and the forcetk script and then adds a few prototype functions which mimic the behavior of the forcetk script in its request pattern.

The plain-text blob payload is hardcoded and is identical to the one from the successful cURL request above.

Strangely, this multipart/form-data POST when sent through the proxy ends in a timeout and the Document is not inserted. I cannot determine what part of the request payload is causing the timeout

FileUpload.page (prerequisites: forcetk.js in a static resource & Remote Site added to the org to allow request to the REST API through the proxy)

<apex:page title="REST API File Upload">

    <script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
    <script src="{!URLFOR($Resource.ForceTK, 'forcetk.js')}"></script>

    <script>
        // Get a reference to jQuery that we can work with
        jQuery.noConflict();

        // Get an instance of the REST API client and set the session ID
        var client = new forcetk.Client();
        client.setSessionToken('{!$Api.Session_ID}');

        // ***********
        // new functions to handle the POST of the multipart form payload to the REST API endpoint with OAuth
        // ***********

        // upload blob part in the same manner that the .ajax function does in the forcetk script
        forcetk.Client.prototype.ajaxBlob = function(path, callback, error, method, payload, retry, boundaryString) {
            var that = this;
            var url = this.instanceUrl + '/services/data' + path;

            return $j.ajax({
                type: method || "GET",
                async: this.asyncAjax,
                url: (this.proxyUrl !== null) ? this.proxyUrl: url,
                contentType: false,
                cache: false,
                processData: false,
                data: payload,
                success: callback,
                error: (!this.refreshToken || retry ) ? error : function(jqXHR, textStatus, errorThrown) {
                    if (jqXHR.status === 401) {
                        that.refreshAccessToken(function(oauthResponse) {
                            that.setSessionToken(oauthResponse.access_token, null, oauthResponse.instance_url);
                            that.ajaxBlob(path, callback, error, method, payload, true);
                        },
                        error);
                    } else {
                        error(jqXHR, textStatus, errorThrown);
                    }
                },
                beforeSend: function(xhr) {
                    if (that.proxyUrl !== null) {
                        xhr.setRequestHeader('SalesforceProxy-Endpoint', url);
                    }
                    xhr.setRequestHeader('Content-Type', 'multipart/form-data; boundary=' + boundaryString);
                    xhr.setRequestHeader(that.authzHeader, "OAuth " + that.sessionId);
                    xhr.setRequestHeader('X-User-Agent', 'salesforce-toolkit-rest-javascript/' + that.apiVersion);
                }
            });         
        }

        // simplified function implementation called by the caller
        forcetk.Client.prototype.createBlob = function(objtype, formData, boundaryString, callback, error) {
            return this.ajaxBlob('/' + this.apiVersion + '/sobjects/' + objtype + '/'
            , callback, error, "POST", formData, null, boundaryString);
        }


        // a function to build the test payload and execute the call the REST API through the proxy
        function sendPayload(buttonElement) {

            // remove any output we added to the textarea tag in a previous sendPayload
            jQuery('#fielddataBody').empty();

            var boundary = 'boundary_string';

            /* Parameters for building the payload */
            var params = {
                "entity_content": {
                    type: 'application/json',
                    content: JSON.stringify({
                        "Description" : "Fake File: " + (new Date()).toUTCString(), 
                        "Keywords" : "fake,file,test", 
                        "FolderId" : "00li0000000ujbN", /* folder Id of 'Shared Documents' */
                        "Name" : "Fake File: " + (new Date()).toUTCString(), 
                        "Type" : "text"
                    })
                },
                "Body": {
                    type: 'text/plain',
                    content: 'This is the text content of my fake file',
                    filename: 'fakefile.txt'
                }
            };

            /* Array to contain the payload pieces */
            var content = [];
            for (var i in params) {
                // add the leading boundary for this item
                content.push('--' + boundary);

                // create the content-disposition section header with the name of this element
                var mimeHeader = 'Content-Disposition: form-data; name="' + i + '"; ';

                // if we've got a filename parameter, add that to the section header
                if(params[i].filename) {
                    mimeHeader += 'filename="'+ params[i].filename +'";';
                }
                // add the header into the array
                content.push(mimeHeader);

                // add the content type section header with the type value 
                if (params[i].type) {
                    content.push('Content-Type: ' + params[i].type);
                }
                // add white space
                content.push('');

                // add the body content
                content.push(params[i].content || params[i]);

                // add white space
                content.push('');
            };

            // add the closing boundary string
            content.push('--' + boundary + '--');

            // create the payload with line breaks between array elements
            var fielddata = content.join('\r\n');

            // add the payload to the body of the page for troubleshooting
            jQuery('#fielddataBody').text(fielddata);

            // make the request through the added prototype method on the forcetk client instance
            client.createBlob('Document', fielddata, boundary, createBlobResultHandler, createBlobResultHandler);

            // add the ajax image after the button
            jQuery(buttonElement).after('<img src="/img/loading.gif" class="waitingImage" title="Please Wait...">');
        }

        function createBlobResultHandler(response) {
            jQuery('#result').html(response.statusText);
            jQuery('.waitingImage').remove();       
        }
    </script>

    <apex:sectionHeader title="REST API" subtitle="REST API File Upload" />
    <div>
        <strong>REST API: Insert or Update Blob Data</strong><br />
        <a href="http://www.salesforce.com/us/developer/docs/api_restpre/Content/dome_sobject_insert_update_blob.htm">http://www.salesforce.com/us/developer/docs/api_restpre/Content/dome_sobject_insert_update_blob.htm</a>
        <br /><br />
    </div>

    <button onclick="sendPayload(this);">Send Payload to REST API</button><br /><br />  
    <div id="result">-- Result Message --</div><br /><br />
    fielddata (payload):<br />
    <textarea id="fielddataBody" style="width: 100%; height: 40em; font-family: monospace;"></textarea>
</apex:page>

Request via VF page

POST https://c.na15.visual.force.com/services/proxy HTTP/1.1
Host: c.na15.visual.force.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
SalesforceProxy-Endpoint: https://na15.salesforce.com/services/data/v27.0/sobjects/Document/
Content-Type: multipart/form-data; charset=UTF-8; boundary=boundary_string
Authorization: OAuth 00Di0000000ZEt5!ASAAQI8rBRpPixB7ydK8 ...[snip]
X-User-Agent: salesforce-toolkit-rest-javascript/v27.0
X-Requested-With: XMLHttpRequest
Referer: https://c.na15.visual.force.com/apex/FileUpload
Content-Length: 448
Cookie: [snip]
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache

--boundary_string
Content-Disposition: form-data; name="entity_content"; 
Content-Type: application/json

{"Description":"Marketing brochure for Q1 2013","Keywords":"marketing,sales,update","FolderId":"00li0000000ujbN","Name":"Marketing Brochure Q1","Type":"text"}

--boundary_string
Content-Disposition: form-data; name="Body"; filename="fakefile.txt";
Content-Type: text/plain

This is the content of my fake file

--boundary_string--

Response via VF page

HTTP/1.1 400 Unable to forward request due to: Read timed out
Date: Tue, 04 Jun 2013 09:18:31 GMT
Content-Type: text/html;charset=ISO-8859-1
Cache-Control: must-revalidate,no-cache,no-store
Content-Length: 1457

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 400 Unable to forward request due to: Read timed out</title>
</head>
<body><h2>HTTP ERROR 400</h2>
<p>Problem accessing /services/proxy. Reason:
<pre>    Unable to forward request due to: Read timed out</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/>                                                
<br/>

[snip]                                             

</body>
</html>

Related, possibly helpful information from SFDC support:

In our proxy, we create a org.apache.commons.httpclient.HttpClient with the connection timeout and socket timeout, both of them are hard coded to 10 secs. Then we send the request via HttpClient. HttpClient will make the actual connection to the endpoint url. It will time out if it cannot establish the connection in 10 secs. We check the request timeout that you set in the client code in our proxy only after we get the response back. During the time we send the response back to client, we check if the request already passed the timeout, if it did, we send the error "Request timedout after: xxxxx". So in your case, it is the connection that times out. We haven't got the response at all by that time, so request timeout does not take effect.

I also located a post from 2008 which appears to indicate that Manoj Cheenath identified an issue with requests through the proxy which appears to be similar to this which was documented as bug #192181.

http://boards.developerforce.com/t5/Other-Salesforce-Applications/remoteFunction-returning-Read-Timeout/m-p/80292#M8578

Does anyone have a working example which performs a successful multipart/form-data POST to the REST API from VF through the proxy?

A working example using sforce.connection.remoteFunction({}) from the ajax toolkit would also suffice.

Can anyone provide insight into what is wrong with the transaction?

Update: Seems very likely that there is a problem with the multipart/form-data type when traversing the ajax proxy

I decided to boil it down further by using a raw XMLHttpRequest object to perform the request through the proxy. The results are below.

Payload setup

// the payload for the body of the http request
var payload = '\r\n';
payload += '--boundary_string\r\n';
payload += 'Content-Disposition: form-data; name="entity_content";\r\n';
payload += 'Content-Type: application/json\r\n\r\n';
payload += '{"Description":"Marketing brochure for Q1 2013","Keywords":"marketing,sales,update","FolderId":"00li0000000ujbN","Name":"Marketing Brochure Q1","Type":"text"}\r\n\r\n';
payload += '--boundary_string\r\n';
payload += 'Content-Disposition: form-data; name="Body"; filename="fakefile.txt";\r\n';
payload += 'Content-Type: text/plain\r\n\r\n';
payload += 'This is the content of my fake file\r\n\r\n';
payload += '--boundary_string--\r\n';

// the SFDC sessionid
var mySessionId = '{!$Api.Session_ID}';

HTTP 500 – Error (endpoint outside of salesforce, content-type is 'multipart/form-data'):

var myRequest = new XMLHttpRequest();
myRequest.open('POST', '/services/proxy', true);
myRequest.setRequestHeader('SalesforceProxy-Endpoint', 'https://cheenath.com/tutorial/sfdc/sample2/echo.php');                  
myRequest.setRequestHeader("Authorization", "OAuth " + mySessionId);
myRequest.setRequestHeader('Content-Type', 'multipart/form-data; boundary=boundary_string');
myRequest.onreadystatechange = function(data) { if (myRequest.readyState === 4) { alert(myRequest.responseText); }};
myRequest.send(payload);

HTTP 400 – Timeout (endpoint inside of salesforce, content-type is 'multipart/form-data')

var myRequest = new XMLHttpRequest();
myRequest.open('POST', '/services/proxy', true);
myRequest.setRequestHeader('SalesforceProxy-Endpoint', 'https://na15.salesforce.com/services/data/v27.0/sobjects/Document/');
myRequest.setRequestHeader("Authorization", "OAuth " + mySessionId);
myRequest.setRequestHeader('Content-Type', 'multipart/form-data; boundary=boundary_string');
myRequest.onreadystatechange = function(data) { if (myRequest.readyState === 4) { alert(myRequest.responseText); }};
myRequest.send(payload);

HTTP 200 – Success with echo back (endpoint outside of salesforce, content-type is 'text/plain')

var myRequest = new XMLHttpRequest();
myRequest.open('POST', '/services/proxy', true);
myRequest.setRequestHeader('SalesforceProxy-Endpoint', 'https://cheenath.com/tutorial/sfdc/sample2/echo.php');                  
myRequest.setRequestHeader("Authorization", "OAuth " + mySessionId);
myRequest.setRequestHeader('Content-Type', 'text/plain');
myRequest.onreadystatechange = function(data) { if (myRequest.readyState === 4) { alert(myRequest.responseText); }};
myRequest.send(payload);

HTTP 415 – MediaType is unsupported (endpoint inside of salesforce, content-type is 'text/plain')

var myRequest = new XMLHttpRequest();
myRequest.open('POST', '/services/proxy', true);
myRequest.setRequestHeader('SalesforceProxy-Endpoint', 'https://na15.salesforce.com/services/data/v27.0/sobjects/Document/');
myRequest.setRequestHeader("Authorization", "OAuth " + mySessionId);
myRequest.setRequestHeader('Content-Type', 'text/plain');
myRequest.onreadystatechange = function(data) { if (myRequest.readyState === 4) { alert(myRequest.responseText); }};
myRequest.send(payload);