I found an answer to your question a couple if years back and it works well up to about a total of 5 megabytes for all of the files total. I do not remember the source but will look as I want to give due credit. I will add the controller, component, javascript and css fileto this post response. If you have additional question please let me know.
I added my own test code for controller and the custom objects in which I use this feature, so you will have to write some of your own test code. This example was old enough I was allowed to add the test code to the same class. Something we cannot do anymore. I used another utility class called TestConfiguration in which I call static methods to create objects for testing. There are many good sources on using this concept to manage creation of test objects.
I will find the source and add this information as I need to give due credit as much of this work is not my own.
Controller
`global with sharing class FileUploadController {
@RemoteAction
global static String attachBlob(String parentId, String attachmentId, String fileName, String contentType, String base64BlobValue){
/*
parentId: The sfdc object Id this file will be attached to
attachmentId: The record of the current Attachment file being processed
fileName: Name of the attachment
contentTye: Content Type of the file being attached
base64BlobValue: Base64 encoded string of the file piece currently processing
*/
//If recordId is blank this is the first part of a multi piece upload
if(attachmentId == '' || attachmentId == null){
Attachment att = new Attachment(
ParentId = parentId,
Body = EncodingUtil.Base64Decode(base64BlobValue),
Name = fileName,
ContentType = contentType
);
insert att;
//Return the new attachment Id
return att.Id;
}else{
for(Attachment atm : [select Id, Body from Attachment where Id = :attachmentId]){
//Take the body of the current attachment, convert to base64 string, append base64 value sent from page, then convert back to binary for the body
update new Attachment(Id = attachmentId, Body = EncodingUtil.Base64Decode(EncodingUtil.Base64Encode(atm.Body) + base64BlobValue));
}
//Return the Id of the attachment we are currently processing
return attachmentId;
}
}
@isTest
private static void testFileUploads(){
//to add additional information to the account record iterate through the returned list before inserting
List<Account> accts = TestConfiguration.createAccounts('Account', 1);
insert accts;
//to add additional information to the course record iterate through the returned list before inserting
List<Course_Detail__c> courses = TestConfiguration.createCourses('Course', 1, accts);
insert courses;
List<Course_Inspection__c> inspections = TestConfiguration.createInspections('Inspection', 1, courses);
insert inspections;
//this has no coverage as is
Blob bodyBlob=Blob.valueOf('Unit Test Attachment Body');
FileUploadController fc = new FileUploadController();
String result = FileUploadController.attachBlob(inspections.get(0).Id, '', 'test.js', 'javascript', bodyBlob.toString());
List<Attachment> a = [Select Id, ContentType, Body, ParentId from Attachment where Id =: result];
System.assertNotEquals(a.get(0), null);
Blob newBody = EncodingUtil.Base64Decode(EncodingUtil.Base64Encode(a.get(0).Body) + bodyBlob.toString());
String result2 = FileUploadController.attachBlob(inspections.get(0).Id, a.get(0).Id,'','',bodyBlob.toString());
System.assertNotEquals(result2, null);
}
}`
Component (to be added to the page)
<apex:component controller="FileUploadController">
<apex:attribute name="parentId" description="The ID of the record uploaded documents will be attached to." type="String" required="true"/>
<link rel="stylesheet" type="text/css" href="{!$Resource.FileUploadCSS}"/>
<script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"/>
<script type="text/javascript" src="{!$Resource.FileUploadJS}"/>
<script type="text/javascript">
var parentId = '{!parentId}'; //Will be used by FileUploadJS.js but must be declared here. Static resources don't support dynamic values.
</script>
<div class="uploadBox">
<table cellpadding="0" cellspacing="0" class="uploadTable">
<tr>
<td><input type="file" multiple="true" id="filesInput" name="file" /></td>
<td class="buttonTD">
<input id="uploadButton" type="button" title="Upload" class="btn" value=" Upload "/>
<input id="clear" type="button" title="Clear" class="btn" value=" Clear "/>
</td>
</tr>
</table>
</div>
</apex:component>
JavaScript
var j$ = jQuery.noConflict();
j$(document).ready(function() {
//Event listener for click of Upload button
j$("#uploadButton").click(function(){
prepareFileUploads();
});
//Event listener to clear upload details/status bars once upload is complete
j$("#clear").on('click',function(){
j$(".upload").remove();
});
});
var byteChunkArray;
var files;
var currentFile;
var $upload;
var CHUNK_SIZE = 180000; //Must be evenly divisible by 3, if not, data corruption will occur
var VIEW_URL = '/servlet/servlet.FileDownload?file=';
//var parentId, you will see this variable used below but it is set in the component as this is a dynamic value passed in by component attribute
//Executes when start Upload button is selected
function prepareFileUploads(){
//Get the file(s) from the input field
files = document.getElementById('filesInput').files;
//Only proceed if there are files selected
if(files.length == 0){
alert('Please select a file!');
return; //end function
}
//Disable inputs and buttons during the upload process
j$(".uploadBox input").attr("disabled", "disabled");
j$(".uploadBox button").attr({
disabled: "disabled",
class: "btnDisabled"
});
//Build out the upload divs for each file selected
var uploadMarkup = '';
for(i = 0; i < files.length; i++){
//Determine file display size
if(files[i].size < 1000000){
var displaySize = Math.floor(files[i].size/1000) + 'K';
}else{
var displaySize = Math.round((files[i].size / 1000000)*10)/10 + 'MB';
}
//For each file being uploaded create a div to represent that file, includes file size, status bar, etc. data-Status tracks status of upload
uploadMarkup += '<div class="upload" data-status="pending" data-index="'+i+'">'; //index used to correspond these upload boxes to records in the files array
uploadMarkup += '<div class="fileName"><span class="name">'+ files[i].name + '</span> - '+ displaySize+ '</div>';
uploadMarkup += '<div class="percentComplete">0%</div>'
uploadMarkup += '<div class="clear"/>';
uploadMarkup += '<div class="statusBar">';
uploadMarkup += '<div class="statusBarPercent"/>';
uploadMarkup += '</div>';
uploadMarkup += '</div>';
}
//Add markup to the upload box
j$('.uploadBox').append(uploadMarkup);
//Once elements have been added to the page representing the uploads, start the actual upload process
checkForUploads();
}
function checkForUploads(){
//Get div of the first matching upload element that is 'pending', if none, all uploads are complete
$upload = j$(".upload:first[data-status='pending']");
if($upload.length != 0){
//Based on index of the div, get correct file from files array
currentFile = files[$upload.attr('data-index')];
/*Build the byteChunkArray array for the current file we are processing. This array is formatted as:
['0-179999','180000-359999',etc] and represents the chunks of bytes that will be uploaded individually.*/
byteChunkArray = new Array();
//First check to see if file size is less than the chunk size, if so first and only chunk is entire size of file
if(currentFile.size <= CHUNK_SIZE){
byteChunkArray[0] = '0-' + (currentFile.size - 1);
}else{
//Determine how many whole byte chunks make up the file,
var numOfFullChunks = Math.floor(currentFile.size / CHUNK_SIZE); //i.e. 1.2MB file would be 1000000 / CHUNK_SIZE
var remainderBytes = currentFile.size % CHUNK_SIZE; // would determine remainder of 1200000 bytes that is not a full chunk
var startByte = 0;
var endByte = CHUNK_SIZE - 1;
//Loop through the number of full chunks and build the byteChunkArray array
for(i = 0; i < numOfFullChunks; i++){
byteChunkArray[i] = startByte+'-'+endByte;
//Set new start and stop bytes for next iteration of loop
startByte = endByte + 1;
endByte += CHUNK_SIZE;
}
//Add the last chunk of remaining bytes to the byteChunkArray
startByte = currentFile.size - remainderBytes;
endByte = currentFile.size;
byteChunkArray.push(startByte+'-'+endByte);
}
//Start processing the byteChunkArray for the current file, parameter is '' because this is the first chunk being uploaded and there is no attachment Id
processByteChunkArray('');
}else{
//All uploads completed, enable the input and buttons
j$(".uploadBox input").removeAttr("disabled");
j$(".uploadBox button").removeAttr("disabled").attr("class","btn");
/*Remove the browse input element and replace it, this essentially removes
the selected files and helps prevent duplicate uploads*/
j$("#filesInput").replaceWith('<input type="file" name="file" multiple="true" id="filesInput">');
}
}
//Uploads a chunk of bytes, if attachmentId is passed in it will attach the bytes to an existing attachment record
function processByteChunkArray(attachmentId){
//Proceed if there are still values in the byteChunkArray, if none, all piece of the file have been uploaded
if(byteChunkArray.length > 0){
//Determine the byte range that needs to uploaded, if byteChunkArray is like... ['0-179999','180000-359999']
var indexes = byteChunkArray[0].split('-'); //... get the first index range '0-179999' -> ['0','179999']
var startByte = parseInt(indexes[0]); //0
var stopByte = parseInt(indexes[1]); //179999
//Slice the part of the file we want to upload, currentFile variable is set in checkForUploads() method that is called before this method
if(currentFile.webkitSlice){
var blobChunk = currentFile.webkitSlice(startByte , stopByte + 1);
}else if (currentFile.mozSlice) {
var blobChunk = currentFile.mozSlice(startByte , stopByte + 1);
}
//Create a new reader object, part of HTML5 File API
var reader = new FileReader();
//Read the blobChunk as a binary string, reader.onloadend function below is automatically called after this line
reader.readAsBinaryString(blobChunk);
//Create a reader.onload function, this will execute immediately after reader.readAsBinaryString() function above;
reader.onloadend = function(evt){
if(evt.target.readyState == FileReader.DONE){ //Make sure read was successful, DONE == 2
//Base 64 encode the data for transmission to the server with JS remoting, window.btoa currently on support by some browsers
var base64value = window.btoa(evt.target.result);
//Use js remoting to send the base64 encoded chunk for uploading
FileUploadController.attachBlob(parentId,attachmentId,currentFile.name,currentFile.type,base64value,function(result,event){
//Proceed if there were no errors with the remoting call
if(event.status == true){
//Update the percent of the status bar and percent, first determine percent complete
var percentComplete = Math.round((stopByte / currentFile.size) * 100);
$upload.find(".percentComplete").text(percentComplete + '%');
$upload.find(".statusBarPercent").css('width',percentComplete + '%');
//Remove the index information from the byteChunkArray array for the piece just uploaded.
byteChunkArray.shift(); //removes 0 index
//Set the attachmentId of the file we are now processing
attachmentId = result;
//Call process byteChunkArray to upload the next piece of the file
processByteChunkArray(attachmentId);
}else{
//If script is here something broke on the JavasSript remoting call
//Add classes to reflect error
$upload.attr('data-status','complete');
$upload.addClass('uploadError');
$upload.find(".statusPercent").addClass('statusPercentError');
$upload.attr('title',event.message);
//Check and continue the next file to upload
checkForUploads();
}
});
}else{
//Error handling for bad read
alert('Could not read file');
}
};
}else{
//This file has completed, all byte chunks have been uploaded, set status on the div to complete
$upload.attr('data-status','complete');
//Change name of file to link of uploaded attachment
$upload.find(".name").html('<a href="' + VIEW_URL + attachmentId + '" target="_blank">'+currentFile.name+'</a>');
//Call the checkForUploads to find the next upload div that has data-status="incomplete" and start the upload process.
checkForUploads();
}
}
CSS File
.buttonTD{
padding-left: 6px;
}
.clear{
clear:both;
}
.fileName{
float: left;
max-width: 235px;
overflow: hidden;
position: absolute;
text-overflow: ellipsis;
white-space: nowrap;
}
.percentComplete{
float: right;
}
.statusBar{
background: none repeat scroll 0 0 #FFFFFF;
border: 1px solid #EAEAEA;
height: 11px;
padding: 0 2px 0 0;
}
.statusBarPercent{
background-color: #1797C0;
float: left;
height: 9px;
margin: 1px;
max-width: 100%;
}
.statusBarPercentError{
background-color: #CE0000;
}
.upload{
background-color: white;
border: 1px solid #CACACA;
border-radius: 3px 3px 3px 3px;
margin-top: 6px;
padding: 4px;
}
.uploadBox{
background-color: #F8F8F8;
border: 1px solid #EAEAEA;
border-radius: 4px 4px 4px 4px;
color: #333333;
font-size: 12px;
padding: 6px;
width: 350px;
}
.uploadError{
border-color: #CE0000;
}
.uploadTable{
margin-left: auto;
margin-right: auto;
}
Best Answer
This statement rules out using Apex to provide a single download link for all the related files. There are simply too many to process in a single transaction. You will blow past either your SOQL, Heap, or CPU limits.
Your approach should change based on the final destination for all these files. I'd assume since you are potentially retrieving tens of thousands of files it isn't intended for immediate human consumption.
I'd recommend using the Bulk API from an external system to retrieve the required files. This could be combined with the REST API or Partner API SOQL query to find the required ContentVersion records. If you invert the SOQL query to start from the ContentVersion records you can avoid the need to run one query per ContentDocumentLink.