There are some tried and tested patterns for doing this, but if you'd like to save yourself some work and get extra functionality at the same time, you might want to check out Skoodat Relax. Skoodat have built a package up on the platform specifically for scheduling batches with as much power as possible and it sounds like a good match for your needs.
Otherwise, the limit of five jobs is being increased, and you can take the approach you've suggested. Depending on the exact details of the job you're running, you may be able to leverage triggers to do the processing on the other objects and just run the batch on your main record set.
Rather than scheduling a one-time job, schedule a recurring job.
Schedule the job to run on an hourly interval (every hour). As part of the finishing phase of your job, cancel this hourly schedule and replace it with another similar hourly schedule where the first execution is set to be a short period (let's say 5 minutes) from the finish of the job.
This works in a very similar way to using a "one off" schedule (as per your existing implementation) - in both of these implementations the job is rescheduled in the finish phase, but by using a recurring schedule you have the added benefit that if for any reason the job does not execute, the platform will attempt to run it again an hour later, and every hour until it succeeds.
Note that we don't know why the job may fail to execute - but we're assuming that it relates to platform maintenance. Chaining one-off scheduled jobs together relies on the successful start and completion of each job for the integrity of the chain, whereas using a recurring scheduled job provides "auto-resume" behaviour regardless of the successful start / completion of an individual job.
Example process flow:
(1) at 12:00 we schedule a job to run every every hour, at 5 minutes
past the hour: 12:05,13:05,14:05...etc...
(2) at 12:05 the batch manager job is started according to the hourly
schedule, and this checks your custom batch job object records to see
if there is any work currently running or waiting.
It finds that there are no jobs running but there is a job waiting:
"Foo". The batch manager therefore starts the batch process for Foo.
(3) at 13:05 the batch manager job is started according to the hourly
schedule.
On this occasion it finds that job Foo is in progress and so quits
taking no action.
(4) at 13:35 job Foo finishes.
In the finish phase, the existing hourly scheduled job is cancelled,
and another new hourly job is scheduled, this time to run at 40
minutes past the hour: 13:40, 14:40, 15:40...etc…
(5) at 13:40 the batch manager job is due to start according to the
hourly schedule, but this fails (we assume because of platform
maintenance)
(6) at 14:40 the batch manager job is started according to the hourly
schedule.
It finds that there are no jobs running but there is a job waiting: "Bar". The batch manager therefore starts the batch process for Bar.
etc.
Best Answer
You should definitely look at the official documentation on Batch Apex and Apex Scheduler, you will get much more details from there. Highlighting few important things from these documentation, which can help you to understand this behavior based on your queries.
As for what you asked:
Refer to this excerpt from the Apex Scheduler documentation:
and same from Batch Apex:
So there's no guarantee, at what time the jobs will start. It all depends on Salesforce's algorithms and resource availability.
As for your this question:
Refer to the section "Holding Batch Jobs in the Apex Flex Queue" in the Batch Apex documentation. If you have Apex Flex Queue enabled for your org, then you do have a way to "reorder" the jobs. Excerpt below:
However this still does not guarantee immediate processing, jobs are only prioritized but are executed only when resources are available. Refer to later text in the same documentation:
In summary, you can design your jobs to be scheduled at a time, but you cannot guarantee if they will execute exactly at the time you schedule those because of the resource availability.