In this specific case, consider client-side rendering instead of Visualforce rendering. I wrote up an example of each, with performance considerations:
Pure Visualforce
First, I'm rendering a list of 10,000 items with pure Visualforce inside a data table.
Controller:
public with sharing class repeatvf {
public repeatvf() {
startDateTime = JSON.serialize(System.now());
}
public class wrapper {
public string href { get; set; }
public string value { get; set; }
public wrapper(string h, string v) {
href = h;
value = v;
}
}
static wrapper[] generatewrappers() {
wrapper[] wrappers = new wrapper[0];
for(integer i = 0; i < 10000; i++) {
wrappers.add(new wrapper('http://www.google.com/search?q='+i,'Search Google for '+i));
}
return wrappers;
}
public wrapper[] getwrappers() {
return generatewrappers();
}
public string startDateTime { get; set; }
public string endDateTime { get { return JSON.serialize(System.now()); } }
}
Page Code:
<apex:page controller="repeatvf" readOnly="true">
<script>
var startDateTime = JSON.parse('{!startDateTime}');
</script>
<div id="output">
<apex:dataTable value="{!wrappers}" var="wrapper">
<apex:column >
<apex:outputLink value="{!wrapper.href}">{!wrapper.value}</apex:outputLink>
</apex:column>
</apex:dataTable>
</div>
<script>
var endDateTime = JSON.parse('{!endDateTime}');
</script>
<script>
function onload() {
var div = document.getElementById('totalTime'), output = document.getElementById("output");
output.style.display = 'none';
div.appendChild(document.createTextNode('Total Generation Time: '+(Date.parse(endDateTime) - Date.parse(startDateTime))));
}
window.addEventListener('DOMContentLoaded', onload, true);
</script>
<div id="totalTime">
</div>
</apex:page>
In my browser, this code consistently runs between values of 1,800 and 2,500 during the time of this trial.
Low High
1834 2687
Remoting
Here is identical code, using remoting instead of pure Visualforce:
Controller:
public with sharing class renderjs {
public renderjs() {
startDateTime = JSON.serialize(System.now());
}
public class wrapper {
public string href { get; set; }
public string value { get; set; }
public wrapper(string h, string v) {
href = h;
value = v;
}
}
static wrapper[] generatewrappers() {
wrapper[] wrappers = new wrapper[0];
for(integer i = 0; i < 10000; i++) {
wrappers.add(new wrapper('http://www.google.com/search?q='+i,'Search Google for '+i));
}
return wrappers;
}
public wrapper[] getwrappers() {
return generatewrappers();
}
@RemoteAction
public static wrapper[] wrappers() {
return generatewrappers();
}
public string startDateTime { get; set; }
public string endDateTime { get { return JSON.serialize(System.now()); } }
}
Page:
<apex:page controller="renderjs">
<script>
var jsStartTime = new Date(), vfRemoteStartTime;
function render(data, event) {
var vfRemoteEndTime = new Date(), jsRenderTime, jsEndTime, div, table, tbody, tr, td, a, ctr, jsTime, vfTime, vfRemoteTime;
div = document.getElementById('outputArea');
table = document.createElement('table');
tbody = document.createElement('tbody');
for(ctr = 0; ctr < data.length; ctr += 1) {
tr = document.createElement('tr');
td = document.createElement('td');
a = document.createElement('a');
a.href = data[ctr].href;
a.appendChild(document.createTextNode(data[ctr].value));
td.appendChild(a);
tr.appendChild(td);
tbody.appendChild(tr);
}
table.appendChild(tbody);
div.appendChild(table);
jsEndTime = new Date();
div.style.display = 'none';
div = document.getElementById('resultArea');
vfTime = Date.parse(vfEndDateTime) - Date.parse(vfStartDateTime);
jsTime = jsEndTime - jsStartTime;
vfRemoteTime = vfRemoteEndTime - vfRemoteStartTime;
jsRenderTime = new Date() - vfRemoteEndTime;
div.appendChild(document.createTextNode('VF Time: '+vfTime+', VF Remote Time: '+vfRemoteTime+', JS Time: '+jsTime+', JS Render Time: '+jsRenderTime));
}
function onload() {
vfRemoteStartTime = new Date();
renderjs.wrappers(render);
}
window.addEventListener('DOMContentLoaded', onload, true);
</script>
<div id="outputArea">
</div>
<div id="resultArea">
</div>
<script>
var vfStartDateTime = JSON.parse('{!startDateTime}'), vfEndDateTime = JSON.parse('{!endDateTime}');
</script>
</apex:page>
In this example, I get to see the effects of rendering locally, including better time stamps. I have four values I can view: The initial loading time, the time spent remoting, the time elapsed in the page from start to end (the "JS" time), and the time spent rendering ("JS Render Time").
Note that I could get VF rendering time using logs, but I'm just interested in a quick demonstration. Note that I end up with a total rendering time of 1,200 to 1,500, at least a third of a second faster, and in many cases up to a second faster.
This page gives me the following values:
VF Time VF Remote Time JS Time JS Render Time
Low High Low High Low High Low High
19 25 1085 1277 1258 1451 91 97
Observations
Assuming that VF Time + VF Remote Time in the second set of code is the same approximate non-rendering time as the first page, I can clearly see that I have a rendering time of over 700 ms. Conversely, my browser is rendering the same data in less than 100 ms.
So, we can see from these examples, that the following cases are true:
The increased time until the page is available stems from two factors: bandwidth and Visualforce. First, we're actually transferring far less data than we would be with pure HTML, because we only transfer the data, not the formatting. Secondly, "expressions" require time to evaluate that are much faster in JavaScript than in Visualforce. Had I used conditional rendering, it would have had a more profound effect.
The user has immediate (<0.03 seconds) that the page is indeed loading, instead of the 1.8 to 2.5 second wait without remoting. That said, we could also gain speed boosts by using rerender-on-load (e.g. the main page simply loads, then has a JS function that calls a reRender). It still wouldn't be faster than pure remoting, however.
The overall time until the page is usable is reduced by up to a second, up to a 60% decrease of loading time. The user doesn't have as long to wait.
That being said, this model won't always work, and shouldn't be advocated as the end-all solution. However, whenever you're rendering a ton of data that won't need to be edited, consider remoting whenever possible to reduce loading time.
Are your webservice calls being invoked asynchronously (via VF Remoting) or are they part of the page load (action method on the <apex:page>)?
If they are being invoked via Visualforce Remoting, you can have a loading image immediately display on the screen and a callback for your VF Remote method that removes the image.
If you are using the action method on the <apex:page> tag, you'll likely want to switch to VF remoting so your page load isn't held up waiting for the callout to complete.
Update: I added some sample code below based on the link above to show how something like this might work. It isn't "working" code, but more along the lines of psuedocode.
Sample VF Page
<apex:page controller="TestController">
<img id="loading-image" src="/myImage.png"></img>
<div id="my-data"></div>
<script type="text/javascript">
var j$ = jQuery.noConflict();
function getRemoteAccount() {
// This remoting call will use the page's timeout value
Visualforce.remoting.Manager.invokeAction(
'{!$RemoteAction.TestController.getCalloutInfo}',
handleResult
);
}
function handleResult(result, event) {
// TODO - update the UI
j$('#my-data').html(result);
// Hide the loading image
j$('#loading-image').hide();
}
</script>
</apex:page>
Sample Apex Class
public with sharing TestController {
@RemoteAction
global static String getCalloutInfo()
{
// TODO - make callout
// return value to UI
return '{"label":"mock-data"}';
}
}
Best Answer
Since no one has posted an answer I'll have a go.
As you suggest, the problem probably isn't the injection of the second page, but the DB call.
There are a number of things you can do to reduce time in the DB such as the obvious narrowing your query, and adding indexes by way of making any fields involved in WHERE clauses indexed by using the "External Id" checkbox in the field configs.
About a year ago, we've also added the ability to add custom indexes by contacting support. Custom indexes can be turned on in fields that didn't previously support indexing (even formula fields). Custom indexes even support a composite index of up to 2 fields.
There was a great blog post about this back in February that you might want to read about. http://blogs.developerforce.com/engineering/2013/02/force-com-soql-best-practices-nulls-and-formula-fields.html