Trying to figure out how best to build interfaces that retrieve external files between 1-10 MB and display them in the UI.
Right now I have a Lightning Aura Component that calls a web service via HttpRequest and retrieves file metadata, so that users can browse externally stored files. A second HttpRequest allows the user to either preview or download these files, depending on file type. Both methods are synchronous currently.
This works, but the current version has to cap file sizes for preview/download at around 5.7 MB, based on the "Maximum size of callout request or response (HTTP request or Web services call)" governor limit of 6 MB.
We have also been concerned about the "Number of synchronous concurrent long-running transactions that last longer than 5 seconds for each org" governor limit, which we could easily hit if there are multiple users requesting files larger than 1 MB at the same time that other long-running synchronous processes are executing in our org.
Needless to say, I was enthused when I heard that asynchronous processing was coming to Aura Components in Summer '19 in the form of support for Continuations (see release notes). But as I began to test this, I realized my callouts were failing at much lower file size thresholds than 6 MB. Apparently continuations have specific limits listed in the Apex Developer Guide whereby callouts and responses can't exceed 1 MB. Bummer.
Is there a special architecture whereby I can retrieve files of up to 10 MB and display them in the UI, which can be deployed to users at scale?
EDIT OCTOBER 2019
No longer concerned about these types of synchronous concurrent transactions due to a Winter '20 change from Salesforce, Callouts Are Excluded from Long-Running Request Limit. While the file size cap of 6 MB is still a minor frustration, we no longer have to worry about hitting governor limits due to the volume of requests for external files.
Best Answer
If you go via apex, the heap limitations will apply.
The workaround is to do a callout from javascript. Yes you can do that.
So if you do a callout from browser, its heap and cpu of device involved and should be alright.
That being said, the thing you have to worry about is, if its an authenticated call, then make sure you dont do a callout from browser passing username and password. Instead do a callout from apex to get access_token and then use that access_token to call 3rd party,
Src: https://developer.salesforce.com/docs/atlas.en-us.lightning.meta/lightning/js_api_calls_platform.htm
Not able to call Composite REST request from LWC