[SalesForce] Salesforce deployment using Ant and SFDX

Our project is in this format

after using sfdx force:mdapi:convert -r src it converted to this which doesnt have package.xml

Using sfdx force:source:convert converted project to this format with package.xml

Still not sure how this package.xml should be avoided to achieve partial deployment

I need a way to deploy salesforce changes using SFDX without package.xml for continuous integration and delivery, we use custom build system instead of Jenkins and not able to find an example of build.xml with ANT to deploy changes using SFDX while building, please help.
How to perform SFDX commands like below while building using build.xml is it possible?

sfdx force:mdapi:retrieve -r ./mdAPIZip -u jzaa1 -k src/package.xml

sfdx force:mdapi:deploy -c -f ../mdAPIZip/unpackaged.zip -u jzaa1 -w 10

Best Answer

The Ant-based Force.com Migration Tool and SFDX CLI are both clients of the Metadata API. You need one or the other, but generally not both, as both tools provide Metadata API deployment services. Specifically, you don't need the Force.com Migration Tool to drive Salesforce DX. You can issue commands like sfdx force:mdapi:deploy directly via a shell script or your build system's configuration interface.

As to package.xml: if you're using Metadata API-format source code (typically in a src folder) and not SFDX-format source code (typically in a force-app folder), you'll need a package.xml manifest.

If you're using SFDX-format code, you may not need a manifest. SFDX's force:source commands (such as push, for scratch orgs, and deploy, for non-scratch orgs) don't require a manifest. Likewise, converting your SFDX-format source to MDAPI-format source via force:source:convert will create a manifest for you to make the result deployable using the Metadata API, via the force:mdapi:deploy command. (Note that if you're practicing the Org Development Model, without using source-tracking scratch orgs, you'll likely also have a package.xml under the manifest directory).

If you aren't familiar with the two source formats and aren't currently using Salesforce DX, your code is certainly in Metadata API format and does require a manifest for deployment.

Salesforce provides documentation covering CI setup on various providers, including sample repos. I wrote a series of blog posts on setting up SFDX-based CI using CircleCI some time ago.

Partial Deployments

Partial deployments are a completely separate issue from the above.

Salesforce's tooling does not include any support for partial or delta (i.e., change-only) deployments other than with source-tracking orgs (such as scratch orgs), where you can use sfdx force:source:push.

If you're deploying to sandboxes or production, you'll have to write your own scripts to generate a package.xml representing some arbitrary set of changes that you wish to deploy separately. There's not a single standard way to do that; it will depend on the whole scope of your tool chain and development process.

You do not have to use partial deployments to implement CI and CD. Unless your application is truly enormous, you can deploy the entire application to your persistent environments.

Related Solutions

[SalesForce] Deploy changes from bitbucket using ANT Migration Tool

We are using a very similar setup internally. What we have found works is that

1) All devs work in their own separate sandboxes.

2) All development is done in feature branches.

3) When work is ready for QA/Integration testing feature branches are merged into master.

4) Jenkins build runs whenever there is a commit to master and performs a deploy to the QA/Integration environment.

You may want to check out the Atlassian Guide for Setting up CI. They use Bamboo but most of the issues you will run into with maintaining your package.xml and keeping config changes in sync will be very similar.

Salesforce DX – Deploying SFDX-Converted Metadata with Filtered Package.xml

So ... I finally created a sfdx plugin allowing to do that

https://github.com/nvuillam/sfdx_metadatas_filtering

Install it

sfdx plugins:install sfdx_metadatas_filtering
Use it

sfdx metadatafilter:execute -i mdapi_output_dir -p myPackage.xml -o md_api_filtered_output_dir

Full example

sfdx force:source:convert -d tmp/deployDemoQuali/
sfdx metadatafilter:execute -i tmp/deployDemoQuali/ -p myPackage.xml -o tmp/deployDemoQualiFiltered/
sfdx force:mdapi:deploy -d tmp/deployDemoQualiFiltered/ -w 60 -u DemoQuali

Warnings:

all cases are not taken in account yet, please read carefully the README and contribute if you are in a hurry :)
Windows users: sfdx plugin generator is bugged on windows (hardcode call of linux rm instruction) , so you may use Git Bash to run this code ( at least while it installs the plugin dependencies )

EDIT 26/06/2018

It seems that several plugins are necessary to survive in this cruel world, so I joined all the ones I needed to create (including metadata filtering) into https://github.com/nvuillam/sfdx-essentials

This is the only maintained plugin , please use it :)