This is by no means great, but you can force your way around this by editing the local file used to track changes.
- create a new field in the org
sfdx force:source:pull
-> field gets pulled into force-app/main/default
- move the field to a separate directory (at this point sfdx thinks i've deleted the field)
- open
.sfdx/orgs/<your_scratch_org_username>/sourcePathInfos.json
- delete any entries related to the original path
- remind yourself that benioff is free from original sin, so all bugs are features, but you aren't so try and write some decent code
Big picture, SFDX had grand dreams of letting you organize stuff into multiple folders. But there are major bugs with the implementation, and it's been multiple years at this point and none of them are fixed. If you can just stick to one folder. If you do multiple folders, prepare to pay for it in pain.
Issues I'm aware of
- refreshing a file outside of the default folder ends up with an extra copy of the file in the default package folder and no updates to the one you intended to refresh
- if you move metadata from one package directory to another scratch org commands think you're deleting stuff
- when you pull metadata you've got no good way to control where it goes
GIT: Clone on VS Code. Directory selected ../Documents/Demo-Project/force-app. A folder named remote-project is created with the source content from bitbucket. Got a pop up asking for Open Repository and Add to Workspace. I clicked on Add to Workspace.
This is your issue. You're creating an SFDX project, and then cloning a different SFDX project inside it. When you complete this step, your directory tree probably looks like this:
Demo-Project/
force-app/
main/
default/
aura/
remote-project/
.git/
aura/
democmp/
Only the folder enclosing .git
and its descendants are part of the Git repository, because that's where you chose to clone it. Git cannot see anything outside the repo boundary, so anything that is saved in main/default/aura/
isn't part of the repo and isn't visible to Git.
Conversely, you've told Visual Studio Code, by creating an SFDX project, that you want to be working in an outer directory tree rooted at force-app
. If you look in your sfdx-project.json
file, you should see something like
"packageDirectories": [
{
"path": "force-app",
"default": true
}
],
This tells VSC that your project's primary source directory is under force-app
(not remote-project
), and that's where your SFDX commands are going to save and push source from by default.
If you're starting from an existing SFDX project that's stored in a source code repo, which sounds like it's the case although I suspect something's a little off about the way you're discussing the folder tree, you probably want to just clone that project and then open its root folder in Visual Studio Code, rather than creating a new project locally first.
Best Answer
Before answering Keith's specific questions, I'd like to set the stage by describing the fundamentals of Salesforce DX "Projects", "Package Directories", and the "Default Package Directory".
What is a Salesforce DX "Project"?
In general, a Salesforce DX Project is a new local file structure that collects your org's metadata (code and config), org templates, sample data, and tests. The project root is often the repository root of a version control system (VCS) as well.
Specifically, an SFDX Project exists when you have a local directory that contains the following:
sfdx-project.json
and the directory where it's located becomes the "root" of that Salesforce DX project.packageDirectories
array inside the project'ssfdx-project.json
file..sfdx
where the Salesforce CLI keeps a variety of files and directories that support the internal operation of the CLI for that specific Salesforce DX project.There are a couple more files that you get after running the
sfdx force:project:create
command, but the above list describes the minimum set of files and directories that are required to have a functional Salesforce DX project.What is a Salesforce DX "Package Directory"?
The Salesforce CLI works by scanning all of the Package Directories declared in
sfdx-project.json
for locally added or modified SFDX source metadata. It attempts to synchronize that source with any scratch orgs you point the CLI at withsfdx force:source:push
orsfdx force:source:pull
.Any metadata (Custom Objects, Apex Code, Profiles, etc.) created outside of your SFDX project (like in a scratch org) will be new to the CLI's internal map of your project's metadata. When this happens, the CLI needs to have a place to put newly discovered metadata. This is where the Default Package Directory comes in.
What is the "Default Package Directory" and why is it special?
Defined as part of a project's
sfdx-project.json
file, the Default Package Directory is the CLI's go-to location for storing "Remotely Added" metadata.For example, if you add a new Custom Object named
MyObject__c
to your scratch org using the Setup UI and then runsfdx force:source:pull
, the CLI is going to save the SFDX source for your new object locally in your Default Package Directory.The path to the SFDX source for
MyObject__c
will look something like this:The names for
sfdx-project-dir
andsfdx-package-dir
will be different for your project, but everything else would look exactly like this. The Salesforce CLI will always use the pathmain/default/<metadata-type>
inside of your default package directory when storing remotely added metadata. This also happens when you use thesfdx force:mdapi:convert
command to convert MDAPI source to SFDX source.Without a consistent, known location for the CLI to put remotely added SFDX source, developers would need to pre-define locations for all metadata types. Even if a developer thought of "everything" and defined complex rules for what metadata goes where on a
force:source:pull
, you'd still need a generic default just in case they missed something.In other words, having a clean, simple, consistent default location for remotely added metadata is a feature, not a bug. :-)
And now, time to address Keith's original questions (finally!)
Now that we've covered the fundamentals of SFDX Projects and Package Directories, I'll tackle each of Keith's original questions, one at a time.
I look at
main
as a module inside of your Package Directory. This is where the core set of your org customizations would go if you're a customer. If you're building a managed package,main
is where the "shared" code that your app's features depend on lives.The SFDX-Falcon Template uses this concept of "core" and "feature" metadata quite extensively. The idea is that code/metadata that you put in a feature module can depend on what's in the main module, but not the other way around.
If you're an ISV, logical separation of code/metadata like this doesn't just make it easier to understand and organize your packaged code. It's also a fantastic way to get ready for Second-Generation Packaging (Packaging 2).
So, what about the "redundant"
default
directory inside of main?I don't think it's fair to call the
default
directory redundant. It's literally the default location for the CLI to put remotely added metadata. You might implement your own code organization scheme inside ofmain
, but the CLI is guaranteed to get a known, consistent location to put any new source because it's not interfering with anything "non-default" you might be doing.Finally, if you're wondering why/how you might further organize SFDX source inside of your
main
module, one suggestion is to implement a design pattern based on "separation of concerns". Here's how SFDX-Falcon approaches this:Source: Salesforce DX 201 - Advanced Implementation for ISVs (Slide 33)
The best answer here depends on whether you're an ISV Partner building a managed package or a customer working on customizations to your production org.
If you're a Customer
If you're a customer with a complex, convoluted, monolithic codebase (aka the "Happy Soup") it can be helpful to try to start breaking things up into many small, independent unmanaged packages wherever possible. Better yet, take a look at Developer Controlled Packages (DCPs) which are part of the Packaging 2 Beta in Spring '18.
Either way, you're using separate SFDX Package Directories with the expectation that you'll want to deploy packages independently of each other as part of a regular agile delivery process. Build fast, build small, deploy often.
If you're an ISV Partner
If you're an ISV building a first-generation managed package, all of your metadata is already in a single package (unless you're using extension packages). When you deploy updates to your packaging org, you're typically going to send everything at once.
The CLI makes it easy to do this because
sfdx force:source:convert
operates on only one package directory during the conversion from SFDX to MDAPI source. This makes it easier for ISVs to focus on whole-package deployments when they are ready to push code to their packaging org.The good news is that you can easily organize source by module inside a single Package Directory anyway. All your core code goes into
main
, as I described above. For the rest of your app (ie. your features), you add one or more feature module directories and segment your code, similar to the following.Source: Salesforce DX 201 - Advanced Implementation for ISVs (Slide 34)
This is essentially what Keith C is suggesting when he says "it would be better to just have multiple folders inside the single 'force-app' package directory folder". All that SFDX-Falcon does is formalize a basic template for how one might set up those additional folders.
There is one inconvenience that comes up when organizing your SFDX source, however. The fact that doing an
sfdx force:source:pull
on remotely added metadata results in the CLI creating SFDX source files in<default-package-dir>/main/default/
.I personally don't feel like this is a dealbreaker when it comes to choosing to organize SFDX source outside of
main/default
. Yes, it's inconvenient to move the source files when you do it but the benefits of having well organized (and hopefully well segmented) code is worth the investment. In fact, the bigger and more complex your code base is, the more you'll benefit over time by investing in getting things organized now.Conclusion
Salesforce DX brings us new flexibility when organizing the metadata we use to build apps or customize our production orgs. Understanding how the Salesforce CLI uses SFDX Projects and Package Directories to synchronize metadata with scratch orgs can help you get the most out of your Salesforce DX project.
The use of
default/main
helps the CLI know where to save new metadata. Organizing your own customizations into modules that are siblings tomain
inside of your default Package Directory can make managing large projects easier. Community templates like SFDX-Falcon provide a model for how you can make this happen in your own project, and the investments made now can pay dividends later by having a codebase that's easier to understand and (hopefully!) update.