I’m the original inventor of and lead developer for custom metadata types.

Although we have a variety of use cases in mind, custom metadata types are primarily intended to let you develop peers of Salesforce standard metadata types, such as custom fields, tabs, or workflow rules (obviously in this first release we don’t have the power required to duplicate all the functionality in these types).

Just as you can’t perform CUD on such types in Apex today (you can’t create a new tab, modify a custom field, or delete a workflow rule in Apex), you can’t yet perform CUD on custom metadata types in Apex, short of a metadata API callout. You can help us get that feature prioritized by voting for the idea here. To see how to create tooling using Apex for a custom metadata type, you can look at Jean-Baptiste Pringuey’s sample “configurator.”

Testing is a tricky question, which I address in a blog post I just posted. The use case you describe is definitely a legitimate one. There's been a fair bit of demand for configuration visibility to ftests. See the idea Custom Settings should be treated as a Setup Object for APEX Tests

There’s a lot more planned for custom metadata types than custom settings on steroids. App configurations that don’t have to be re-created in every environment and that can respect managed state (so you can have some that are not deletable or mutable by subscribers) was just a starting place that filled a known need. Aaron’s second blog post gives you a taste of things to come in the near term (safe harbor applies, of course).

Winter `16 features

Layoutable UI for entering and updating records is one of the big features that came out in Winter -- take a look at Custom metadata types: they’re money; actually, even better!

Record-level protection of configuration (as opposed to hiding the entire type, which is both a custom settings and a Summer CMT feature) is another.

The third big feature is field-level control over who can edit values. By setting a field to subscriber editable, you'll be able to default the value in a package and let customers change that value if they need to.

More recent features

It's been a while!

Custom Metadata types now support (as of Summer 17):

• Picklist fields
• Relationships to records of other custom metadata types, to SObject types (EntityDefinition), and to SObject fields (FieldDefinition)
• Text Area (Long) fields
• Asynchronous (but not requiring remote site settings) upsert in Apex via Metadata.MetadataOperations.enqueueDeployment()
• Validation formulas (including traversal of relationships)

It's possible to edit subscriber-controlled fields on public records of public custom metadata types via the metadata API, even if those records are included in a managed package. The main thing you need to remember is that both the type and the record will have a namespace in this case, so if they come from the same managed package, the full name of the record will be

packagens__TheType.packagens__TheRecord

Update for Summer 17:

Starting in Summer 17, you can asynchronously edit subscriber-controlled fields on public or protected records of public or protected types, so long as they're visible to your package's code (so, public or in your package or using your package's type), without needing to perform a callout, using the new Metadata package in Apex. You can also create new records of public or protected types--but note that new records are creating into the subscriber's namespace, not yours, so the only way to keep them hidden is if their type is protected and installed.