Experimenting with whether we can easily automate downloading the weekly backup files, based on Enrico Murru's approach here
We are trying to use c# because we use that a lot already and dont want to spin up node unless we have to.
We are running into authentication issues when we hit the data export page, so am doing some testing in postman.
We can successfully hit the data export page by copying my session Id from my browser cookies, and setting a header with key = "Cookie" and value = "sid=sessionidcopiedfrombrowsercookies…" and doing a GET on the export data url
https://{instanceurl}/ui/setup/export/DataExportPage/d?
However, we cannot get to the data export page by using the access token generated by the username password flow with a connected app set to web scope. We get redirected to the login page, so its not accepting the access token. I have tried passing the access token as a cookie as well as in the authorization header. I have tried adding all possible scopes to the connected app.
This documentation on oauth scopes indicates that web scope should allow for accessing the web ui, and specifically that it "allows use of the access_token on the web."
And this doc on the username password flow indicates that the username password flow specifically generates the access token as session ids, which seems like it should work like the session Id I steal from the browser cookie. Note that I can successfully generate the access token via a POST to the token endpoint, I just cannot use it with web scope.
The username-password flow generates access tokens as Salesforce Session IDs
My question: how can we leverage the access token generated by username – password flow to access the data export page via the web? What am I missing here? Is there a better way to get to the data export page? We can setup an oauth flow, or use a library if it will help.
Note : I understand username password flow is very weak. The integration user will only have permission to export the data, and we will lock down ip ranges, and could use a different flow once we can get it working.
UPDATE – I tried posting to the frontdoor and am not able to hit the data export page. The page that is loaded has a single function redirectOnLoad(). This is what I am assuming is the login page as it did not matter if I included the sid param / access token. Am I supposed to do something with this page to get to the page I want?
Best Answer
Not sure with every detail of my answer, so take it with a grain of salt, but the question is interesting and I hope to learn a lot from this discussion:
Username-password flow does not seem to support scopes. All it supports is access to REST/SOAP and "data". I've tried Visualforce and I've tried frontdoor.jsp. I failed both times. I find it unlikely that an arbitrary Setup area will be accessible via this flow, even with detours.
Regarding the OAuth scope "web", I understand the documentation differently. It says it is to "Manage user data via Web browsers" and "access to customer-created Visualforce pages". Does this not sound fairly limited? Again, even when using a flow that allows for this scope, other areas in the Setup are unlikely to be accessible.
So why don't you use the same strategy as Enrico Murru? His code implicitly uses the SOAP login API, which should not be too difficult to emulate in C#. I understand that OAuth would be the proper way (although by no means the username-password version); but then - would Enrico Murru not have done the same (jsforce at hand), if he could?!
Update 13/02/2022:
I've tested the "web" scope (utilising a user-agent flow) and it allowed me to navigate and do updates via the browser interface everywhere, incl. in the setup area. Based on these tests I'd expect the JWT bearer flow along with the "web" scope to allow the download of the export files and to be the textbook recommendation for the given case. Not sure though, if the gain of safety justifies the additional efforts needed to set this up in your case.
Update 14/02/2022:
To answer your question from the comments, here's my way to test the user-agent flow with "web" scope:
https://openidconnect.herokuapp.com/callback, which is used in related cases by Salesforce. (My connected app was configured on a scratch org. But maybe one should take some basic precaution with the choice of who's getting access to one's Salesforce session ID next, in particular, if the connected app is hosted by a production system.)https://[server name].my.salesforce.com?ec=302&startURL=%2Fsetup%2Fsecur%2FRemoteAccessAuthorizationPage.apexp%3Fsource%3DCA[...very long string...]I opened this in a browser instance that has no open Salesforce sessions. As expected, Salesforce asked for conventional login details, just like for any web login. After entering my credentials, I got the OAuth dialogue, "Allow Access?", which displayed two scopes: "Access the identity URL service" and "Manage user data via Web browsers".https://[my server name].my.salesforce.com/secur/frontdoor.jsp?sid=00D...&retURL=%2Fui%2Fsetup%2Fexport%2FDataExportPage%2Fd%3FAnd here you go!
A note regarding sfdcfox's suggestion: I owe this guy a considerable part of what I know about Salesforce, probably more than anyone else. But I disagree with him here. Not only that "front door" works only with the "web" scope (which was not what you were succeeding in doing with the given flow). "front door" makes only sense for humans and not in a context where a (download) script interacts with Salesforce. As far as I can see there is no use of "front door" in the context of a JWT bearer flow, for example; one can target the retURL straight away. "front door" is just a means for a user, who needs to enter (or be directed to) Salesforce with a session ID at hand, but doesn't have advanced cookie editing capabilities. Or have I missed something?!