A non-finite clause might better be defined as a clause headed by a non-finite verbform: an infinitive or a participle. Such a clause by definition1 has no tense. It cannot be the main or matrix clause in a formal sentence, because a matrix clause must be headed by a finite verbform, which defines its Reference Time—the time you are talking about. (see the discussion of tense here.)
Consequently, non-finite clauses only appear in formal sentences as subordinate clauses. Their temporal reference is inferred from context. In most cases that temporal reference is to the Reference Time defined by the matrix clause:
To think this makes him more cheerful may be paraphrased as
That he thinks this makes him more cheerful or
When he thinks this it makes him more cheerful.
A non-finite perfect construction—one in which the HAVE component is cast as a non-finite verborm—works the same way. (Keep in mind that the tense of a perfect construction is expressed by the HAVE component which heads it: a present perfect signifies a present state, a past perfect signifies a past state. Those states, current at Reference Time, arise out of the prior eventuality mentioned by the past participle component.). The semantics of think make it sort of odd in this context, so let's look at do instead.
To have done this makes him more cheerful may be paraphrased as
That he has done this makes him more cheerful or
When he has done this it makes him more cheerful
In the right context, infinitive clauses (including infinitive perfects), readily bear a future reference. For instance:
To think this this will make him more cheerful tomorrow.
To have done this will make him more cheerful in his old age.
1 ‘by definition’ because an English non-finite verbform is defined as one which is not marked for tense. Note that tense here is used in the technical sense “morphologically marked temporal reference”; English has, in this sense, only two tenses, past and non-past.
Best Answer
The bracketed non-finite clauses are neither objects nor adverbs.
These are both catenative constructions. "Agree" and "have" are catenative verbs and the bracketed non-finite clauses are their catenenative complements.
The term 'catentative' is derived from the Latin word for 'chain', for the construction consists of a 'chain' of verbs in which all except the last have a non-finite complement.