On Wed, Apr 06, 2016 at 05:15:43PM -0700, Brandon Philips wrote:

I believe that using this type of well-known scheme is a better
choice today and will stay out of people's way better…

I think blob delegation should be outside the scope of the image
format, since blobs that are content-addressed based on a
cryptographic hash can be fetched from any content-addressable storage
(CAS)—although registries probably want to include hints about hosts
providing blobs required by a given image 1. Baking CAS repository
information into the image manifests requires folks to make
distribution-time decisions at image-creation time (and image authors
and distributors need not be the same people).

 Subject: DNS-based delegation (was: Proposal for a new project: OCI Image Format Spec)
 Date: Sat, 12 Mar 2016 15:34:11 -0800
 Message-ID: <20160312233411.GB10073@odin.tremily.us>

@wking This isn't about blob delegation. This is about delegation of naming for images. Which is explicitly in scope (see the README).

On Thu, Apr 07, 2016 at 06:47:35AM -0700, Brandon Philips wrote:

@wking This isn't about blob delegation. This is about delegation of
naming for images. Which is explicitly in scope (see the README).

Ah, it's just about discovering manifest hashes. Developing and
versioning the separate image/naming/signing components 1 in a
single repository sounds complicated to me (do you make a major bump
when you adjust the naming spec, even though you haven't touched the
image spec?). But discovering manifest hashes using an approach like
this sounds like one of several reasonable registry approaches to me.

@philips Let's make a note here to consider cases such as notary for naming delegation.

Have we decomposed naming delegation to particular set operations?

@stevvooe I don't see how notary relates to name delegation. Do you have a doc?

@stevvooe Ping.

@philips You can read here. The link is fairly straightforward. Let me know if you have more questions.

Perhaps, I am misunderstanding naming delegation. We may need a clear definition and identification specific use cases it solves for the individual user. Part of the crux with naming delegation is that it is great for ISVs and service providers but I'd like to better understand where the user gains benefit. We struggled with our efforts from this angle when we looked into this. If I were, say, a data scientist, why should I have to setup DNS and a web service to deploy software in my local infrastructure?

I do find such strategies as go get or well-known urls to be elegant but getting them reliably deployed can be a nightmare in some organizations (well-known being the clear winner here, perhaps).

We also may want to value offline use cases slightly higher. In typical deployments, there may only be a few organizational mirrors. In the abd examples and some of the proposals we had in this area, it focuses on mirrors in the Internet at large but it seems more likely to have a local mirror, rather than spreading requests across the internet.

Every enterprise I work with wants a trusted repository of data. This is a model they are very familiar with. Think rpm/deb repo or Java's maven repos. How does this name delegation model work for this model? For example imagine nginx comes from nginx.com. But in the "enterprise" they want to mirror the images from nginx.com to their local servers such that when a user pulls "nginx" it goes to the on prem repo and not the internet.

Configuration to declare where the mirror is for nginx.com or more likely
'*'. It is just namespaces that has some default useful semantics.

On Thu, Apr 21, 2016, 7:49 PM Darren Shepherd notifications@github.com
wrote:

Every enterprise I work with wants a trusted repository of data. This is a
model they are very familiar with. Think rpm/deb repo or Java's maven
repos. How does this name delegation model work for this model? For example
imagine nginx comes from nginx.com. But in the "enterprise" they want to
mirror the images from nginx.com to their local servers such that when a
user pulls "nginx" it goes to the on prem repo and not the internet.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#11 (comment)

Does this work today already on rkt or appc? I'd like to understand more. I'm trying to wrap my head around this. The fact that it's model after go is the problem for me. Go has so many distribution problems where other languages like node and Java don't really. But the fundamental difference in the two is that node/Java have a centralized de facto store for everything (npm/mvn central). I have an easier time grasping the idea that name resolution is not decentralized but instead done based on a small set of chosen trusted sources.

@ibuildthecloud Is what you are suggesting actually, technically different than an indefinitely caching proxy with a hand written white list? In that case, such proxys already exist, and no software on OPCs side is necessary. Just set up your "in enterprise" systems to use the companies proxy, and curration and trust management can happen when writting the white list.

Is there room for TUF's delegation notions to get incorporated here? The have a pretty fine grained way of delegating trust for certain packages to different sources, with a well-studied set of security / authenticity guarantees.

On Fri, Apr 22, 2016 at 09:08:29AM -0700, Kamal Marhubi wrote:

Is there room for TUF's delegation
notions to get incorporated here?

Notary [1,2] is a wrapper around TUF. More notes on that and thoughts
about trust delegation here 3.

 Subject: Signature verification after image-format translation
 Date: Fri, 15 Apr 2016 14:50:18 -0700
 Message-ID: <20160415215018.GR23066@odin.tremily.us>

On Fri, Apr 22, 2016 at 9:08 AM Kamal Marhubi notifications@github.com
wrote:

Is there room for TUF http://theupdateframework.com/'s delegation
notions to get incorporated here? The have a pretty fine grained way of
delegating trust for certain packages to different sources, with a
well-studied set of security / authenticity guarantees.

Namespaces in TUF and in particular Notary can use a DNS form.

Here is a concrete example from the docs:
https://github.com/docker/notary/blob/master/docs/getting_started.md#understand-notary-naming

Here are the three use cases as I see it:

1. User doesn't care at all about name delegation, they use an existing registry for all images (e.g. gcr.io/brendanburns/foo, gcr.io/brendanburns/bar, ...)

2. User is running inside their own data center, doesn't trust anyone, doesn't want to go outside their LAN. In this case user still doesn't care about delegation, but they run their own registry and push all images their: (e.g. my-registry.internal/foo, my-registry.internal/bar)

Neither of those two care about name delegation.

3. The interesting middle use case is when someone wants to establish domain based policy (e.g. 'you can only run images from my-company.com) but doesn't want to run their own registry. In that context, they want their end-users to pull my-company.com/bar/foo but have that image actually be hosted on my-company.com.gcr.io/bar/foo

it seems to me that there are two questions here:

1. How do you securely get/set the mapping from my-company.com to my-company.com.gcr.io?

Options here appear to be DNS CNAME records, or a well-known URI.

Pros for CNAME are:

• Natural fit for lots of other examples
• No serving system requirements (can use one of N different cloud DNS providers)

Cons for CNAME:

• requires taking over an entire domain, can't use (my-company.com require registry.my-company.com)
• Harder for clients to write code (DNS libraries are immature compared to HTTP)
• Security requires DNSSEC which isn't fully implemented (or at all) by some providers.

Pros for well known URI:

• Easy to poke at (curl ...)
• Security uses well known HTTPS certs, etc.

Cons for well known URI:

• Extra complexity: have to run a server (though you probably already do)
• More opaque, less elegant approach.

2. Can you manipulate the rest of the image path? If so how?

I think the answer here should be "no" the risks are just to big and the win is minimal.

@brendandburns To clarify, are you saying that the user should be able to configure the naming authority for my-company.com? If a user pulls, my-company.com/bar/foo and gets my-company.com.gcr.io/bar/foo, what is the name of the resulting image?

2. Can you manipulate the rest of the image path? If so how?

I think the answer here should be "no" the risks are just to big and the win is minimal.

We can agree, here. Allowing remotes or local to manipulate paths quickly gets out of hand.

Put differently, if we expressed naming as a function f(name) -> y, what is the type of y? Can we express the naming problem in this manner?

@stevvooe the name of the image is still my-company.com/bar/foo just like if you visit www.company.com and it CNAME's to storage.googleapis.com your browser still shows www.my-company.com in the URL bar.

@stevvooe put another way, this all happens in the DNS subsystem, so the client code pulling the image doesn't even notice. They just see my-company.com resolve to 1.2.3.4 which happens to be the IP for my-company.com.gcr.io

On Wed, Apr 27, 2016 at 3:27 PM Brendan Burns notifications@github.com
wrote:

The interesting middle use case is when someone wants to establish domain
based policy (e.g. 'you can only run images from my-company.com) but
doesn't want to run their own registry. In that context, they want their
end-users to pull my-company.com/bar/foo but have that image actually be
hosted on my-company.com.gcr.io/bar/foo

Yes, exactly.

it seems to me that there are two questions here:

1. How do you securely get/set the mapping from my-company.com to gcr.io?

Options here appear to be DNS CNAME records, or a well-known URI

CNAME doesn't work if you want TLS delegation to be "secure", right?

I think of delegation based on DNS/well-known URI as a "globally
distributed default". If you want to override the default you need to
configure that policy at a cluster/machine level, similar to download
credentials.

Depending on the language we come up with here it will look like something
like this:

"delegationOverrides": [
 {
    "name": "example.com/*",
    "mirrors": ["gcr.io"]
 },
 {
    "name": "quay.io/*",
    "mirrors": ["gcr.io"]
  }
]

2. Can you manipulate the rest of the image path? If so how?

I think the answer here should be "no" the risks are just to big and the
win is minimal.

+1 the entire name should be sent to the delegating party.

It seems to me that TLS should be fine, as long as the thing you are delegating to holds the cert for the original registry.

I think there are two cases to consider (and maybe this is the source of confusion) there is the pure delegation case: "I don't have a registry, but I want a vanity domain" and the mirror case "There's some registry out there, but for various reasons, I want to actually access those images at some other mirror location".

I was really mostly thinking about the former. The mirror one is sort of interesting but doesn't seem like a server-side protocol, but rather a client-specific configuration.

It seems to me that TLS should be fine, as long as the thing you are delegating to holds the cert for the original registry.

I don't see how we can dismiss TLS here. We still need to provide secure connections to both the white label domain and the upstream. They may be different.

How can this happen at a pure dns-level if the primary and delegate domains may have different implementations and certificates? If it really is just pure DNS, why not just tell users to use a CNAME and be done with this? If it really is just CNAMEs, do we even need to specify anything?

@stevvooe I'm not dismissing TLS. The CNAME option would involve a user defining a whole subdomain (e.g. registry.mycompany.com) creating a new different cert just for that subdomain, and storing that cert with their registry provider. And then CNAME over to the registries DNS, (similar to the flow here: https://www.keycdn.com/blog/cdn-ssl-made-easy/)

I think the goal in the spec is to design the best practice for how it would work, so that implementors know what they need to build (e.g. an interface that accepts a cert), even if the actual mechanism doesn't require an implementation.

Anyway, CNAME is one of the two options. A well-known URI is the other option. I tried to list the pros and cons of each in my earlier post

I think that the best effort in this issue is to decide which option is preferred by the maintainers.

If it is easier for me to put the options and pros/cons into a markdown doc and have the discussion in a PR, I can do that too. (and if I missed any pros/cons let me know)

@brendandburns I see. Sorry about the harsh wording here.

How is this different from using TLS with a CNAME as one normally would? It sounds like a very standard usage. Am I missing something? Why does it need to specified? Seems like it belongs in recommendations or best practices.

As an interesting data point, I have suggested this approach situations looking "spruce" up the registry urls. Typically, this is easy if you host your own registry. The problem arises when using providers such as AWS ECR, Docker Registry and GCR (please correct me if I am wrong). They would have to host a private key for each domain. Would such a provision push a requirement for services running a registry to accept certificates to use for white labeling?

then go to one or more registries for the name to hash mapping using the fully resolved image name.

This still delegates the trust model to the registry.

The problem comes when working with multiple registries. Which keys do you trust when they disagree? The only way to manage this is centralization. One approach is to partition the namespace with DNS.

I am not sure that one can honestly separate naming and trust models without redundancy and ambiguity. Can this be done without making such separate systems completely incompatible?

On Mon, May 02, 2016 at 01:47:24PM -0700, Stephen Day wrote:

The problem comes when working with multiple registries. Which keys
do you trust when they disagree?

For example, if the registry (or registries, it doesn't matter) has
two name→hash associations that disagree on the hash, and one is
signed by Alice, and another is signed by Bob, and I trust both Alice
and Bob, then I'll use the newer association. If Alice or Bob is
doing that maliciously, then the solution is to remove them from my
trust store, not to delegate all trust to a single registry.

With the X.509 trust delegation, maybe a debian.org/debian→hash
assertion is signed by a key with a debian.org cert backed by some-CA.
If that sort of thing is in my trust store, then I'll pick the newest
association signed by that key (or any other similar keys supported by
my trust store). So I still don't see a need to centralize at the
registry level.

Perhaps I'm missing something (and I apologize if I do), but wouldn't the ultimate trust model still consist of having the image pulled being signed by a recognized public key? That way, even if it is served by gcr.io/debian, I can still verify it is the Debian image by either transporting the Debian key manually or out-of-band (which is what Notary does)

On Mon, May 02, 2016 at 03:26:19PM -0700, josephschorr wrote:

… wouldn't the ultimate trust model still consist of having the
image pulled being signed by a recognized public key?…

Yeah, I think end-to-end signing is important. Which is why I think
@brendandburns name canonicalization ((2) in 1) needs to be covered
by the trust model 2. In other words, I should be able to fetch a
‘debian.org/debian’ image from gcr.io/debian.org/debian (or a local
file, #23, or wherever, it shouldn't matter) and still be checking
my local trust store to see if I trust the signers to sign the
‘debian.org/debian’ image. That shifts the earlier deconstruction 3
to:

func RegistryLookup(name string, trust TrustStore) (hash string, error)
{
var registry = name;

for (looksLikeDNS(registry)) {
  var resolvedName, _ := getNamespaceDelegate(registry);
  if (resolvedName == registry) {
    break;
  }
  registry = resolvedName;
}

return registryLookup(name, registry, trust)

}

where registryLookup is performing the name (e.g. debian.org/debian)
→ hash lookup and verifying signatures using keys trusted to sign
‘name’. ‘registry’ is just a hint in case registryLookup wants to ask
that domain (e.g. gcr.io), but it could also just ask a local
registry, etc., etc.

The earlier deconstruction 3, on the other hand, was fetching the
resolved name (e.g. gcr.io/debian.org/debian) from the registry,
which means that the canonicalization (or “name resolution”, or
whatever you want to call debian.org/debian →
gcr.io/debian.org/debian) was untrusted (or trusted independently
based on your X.509 trust used implicitly inside
getNamespaceDelegate). This sounds much less reliable (or at least,
it's harder to reason about).

I will re-iterate, there are really two different issues here:

1. I want a name for my image based on my domain (mycompany.com/foo) but I have no intention of hosting my own registry. I should have the ability to redirect you to a registry so that you can know where to get the image. This is name resolution.

2. There is mirroring, which is I am registry a.com but I want to host image that originate at b.com/bar such that if you ask me for that image, I can deliver it, and you (because you trust my signing) know that it is the same image that a.com would have shipped me.

I think that this issue is concerned with 1, not 2.

There is definitely trust involved in name canonicalization, but I think that we have to require that the trust is implemented by having the owner of the domain (mycompany.com) own name canonicalization.

The reason for this is that it is otherwise very difficult for me (as an image owner) to change the canonicalization of the name (say I switch from mycompany.com -> a.com to mycompany.com -> b.com.

On Fri, May 06, 2016 at 10:57:51AM -0700, Brendan Burns wrote:

There is definitely trust involved in name canonicalization, but I
think that we have to require that the trust is implemented by
having the owner of the domain (mycompany.com) own name
canonicalization.

The reason for this is that it is otherwise very difficult for me
(as an image owner) to change the canonicalization of the name (say
I switch from mycompany.com -> a.com to mycompany.com ->
b.com.

This assumes:

1. That X.509 certificates based on a shared CA pool let folks agree
   on which keys can sign for mycompany.com.
2. That images named with a {host}/{other-stuff} pattern belong to
   the keys from (1).

Both of those are reasonable choices for an implementation and are
probably requirements for DNS-based delegation, but I don't think they
should be requirements for all name→CAS implementations. For some
systems there's no need to drag in X.509 at all, and I think we want
to support those systems too.

Hrm, I think all of the signing stuff is out of scope for this issue. In my mind there are two ways that we establish trust:

1. I use DNSSEC to ensure that the DNS resolution is trusted.
2. If we use well-known URIs, then we also use standard HTTPS CA certs to ensure that I trust the data I receive from that well known URI.

People who don't want this can use IP addresses for their registry prefix, which inherently will skip name resolultion.

On Fri, May 06, 2016 at 11:16:46AM -0700, Brendan Burns wrote:

People who don't want this can use IP addresses for their registry
prefix, which inherently will skip name resolultion.

And folks who are using a sneakernet and OpenPGP? I agree that lots
of folks are going to be distributing images over HTTPS involving
their usual DNS and X.509 trust, but I'd rather that wasn't the only
naming/signing scheme that this specification supports. For example,
the #23 use case (project publishing its application), it would be
nice if step 1 for pushing verifiable content wasn't “get a domain and
have a well-known CA certify your ownership of such”. Lots of
projects are already pushing OpenPGP-signed tarballs and Git tags, so
allowing OpenPGP signing for images (without DNS-based namespacing)
would make life easier for them.

@wking in that case they can use any name they want, and then install the appropriate mirror definition in their clients.

This is why I think there are two different issues getting smashed around here.

One is discovery/naming: how do I go from the name of an image to an actual place where I can get a bunch of file hashes.

The other is trust: I have this collection of tarballs, how do I assert that they are the thing that I think they are and that they are sourced from a source I trust.

In your use case, name resolution and discovery aren't necessary. You already have the files in a tarball. Only the trust part is needed.

But for most users that isn't the way they will interact with images.

And for people who distribute images but don't want to get a domain name there are always the existing hosted registries (e.g. gcr.io)

I could imagine defining some sort of null registry localhost? which basically means "There is no corresponding registry"

On Fri, May 06, 2016 at 02:04:54PM -0700, Brendan Burns wrote:

This is why I think there are two different issues getting smashed
around here.

I agree that there are at least two issues, I just don't want to see
resolution for one (DNS-based delegation) negatively impact another
(naming and trust).

And for people who distribute images but don't want to get a domain
name there are always the existing hosted registries (e.g. gcr.io)

With that approach, how will folks sign/verify gcr.io or localhost
images?

@wking

So my thought is that you would sign with your keys prior to uploading the image to the registry, but I haven't thought super deep about the signing problem, since I think its different than name resolution, and I've focused name resolution for this issue.

On Thu, May 19, 2016 at 10:18:18AM -0700, Brendan Burns wrote:

… but I haven't thought super deep about the signing problem, since
I think its different than name resolution, and I've focused name
resolution for this issue.

I think signing is going to be inseparable from name resolution.
There needs to be clarity about which name is being verified 1, and
if it's not the user-supplied name (e.g. it's a canonical version),
there needs to be clarity on the trust model for the canonicalization
2.

On the other hand, signing should be orthogonal from distribution.
So a client should be able to fetch the image from any provider (a
CNAMEd host, local mirror, email attachment, etc.) and still be able
to verify that the image has the name you're looking for. For example
(based on 3):

$ oci-image-tool unpack path/to/my-oci-image.tar.gz debian-1.0

which unpacks the image whose name matches debian-1.0 after checking
that the debian-1.0 entry is appropriately signed.

$ oci-image-tool unpack https://gcr.io/brendanburns/foo mycompany.com/foo

which unpacks the image whose name matches mycompany.com/foo after
checking that the mycompany.com/foo entry is appropriately signed.

With that orthogonal signing in place, how you get from a
user-specified mycompany.com/foo to a fetching location (e.g. a
https://gcr.io/brendanburns/foo mirror) doesn't matter much to me.

@wking
I think we agree. I view name resolution as being solely about figuring out where to download the thing from.

The name that you use for validating signing is still the original name.

Put clearly in an example:

I start pulling company.com/bar/foo which redirects me to the company.com.gcr.io/bar/foo registry, I download the layers from that registry.

But then once downloaded, I verify the images using the original name mycompany.com/bar/foo and using keys that are distributed by that company.

Make sense?

On Thu, May 19, 2016 at 12:34:05PM -0700, Brendan Burns wrote:

But then once downloaded, I verify the images using the original
name mycompany.com/bar/foo and using keys that are distributed by
that company.

Sounds good to me, although it sounds more like the pseudo-code in 1
and text in 2 than the pseudo-code in 3 or text in 4. And if
the original name is still being used (which I like), I think the
redirecting is better described by “mirroring” 5 than by
“delegation” 6 or “canonicalization” 7. And there's no need to
worry about key hosting 8 or the security of a particular sub-step
(like mirror choice) 9.

And with this optional mirror-lookup procedure, I want to make sure
non-DNS image names are still supported 10. They wouldn't get this
DNS-based mirroring, but there are many other possible mirroring
approaches.

And I am absolutely fine with folks mirroring any image they have
legal access to. With the image name being a property of the image
that's independent of mirroring, I don't see why you'd have a problem
with that 11.

I guess I think:

if the owner of the name choses the remapping then it is "delegation" since there no other locations where that image can be found.

if the owner of the client chooses the remapping then it is "mirroring" since it is copied from wherever the original source is located.

but it's semantics, I guess. If we have rough agreement on the goals, do we have a preference for DNS or well-known URL? (or both?)

On Thu, May 19, 2016 at 02:09:02PM -0700, Brendan Burns wrote:

if the owner of the name choses the remapping then it is
"delegation" since there no other locations where that image can be
found.

How many images are licensed “all rights reserved”? I'd guess for
things like Debian images, you could have third-parties choosing to
mirror images as they see fit, although the original author is always
free to bless (or not) those mirrors as they see fit (e.g. they can
say “a.example.com and b.example.com also carry this image”, while
c.example.com and d.example.com are serving copies as well).

If we have rough agreement on the goals, do we have a preference for
DNS or well-known URL? (or both?)

I don't care once things get down to this level. I just get jumpy
when folks start talking about requiring DNS-based names ;).

@wking I think we should require dns compatible names. whether the domain actually exists or not is up to the end user.

On Thu, May 19, 2016 at 02:31PM -0700, Brendan Burns wrote:

I think we should require dns compatible names. whether the domain
actually exists or not is up to the end user.

What's the benefit to requiring DNS-based names but allowing
nonexistant domains? I explained why I want X.509 to be one option
among many for image trust earlier [1,2]. You floated localhost-based
naming earlier 3, but I think we agree that you won't be using an
X.509 certificate chain [4,5], because no CA will certify you as the
owner of ‘localhost’ (I hope ;). And if you're using OpenPGP or
similar, the name is an opaque string as far as trust is concerned.

So folks intending to use DNS-based mirror discovery should be using
DNS-based names (with valid domains) 2. But I don't see how you
benefit from having ‘localhost/my-app-v1.0’ instead of my preferred
‘my-app-v1.0’ as an image name for folks who don't need DNS-based
mirror discovery.

We have been discussing naming for quite some time but I would like to propose to the @opencontainers/image-spec-maintainers that we move this to the post-v1.0.0 milestone for two reasons:

1. Naming is an "optional layer" for the specification scope. We have moved the other "optional layers" out such as signing in signing: start discussion on signing #22 to post-v1.0.0 as well so we can focus on the initial release being the required layers.

2. We all have a really solid idea of how this will work using well-known URIs and pointing at a registry that implements the v2 CAS API. But, we should focus on getting those registry transports working first before we add a layer on top. I know a variety of the registry implementations at Google, Amazon, and Quay.io have this on their roadmap.

We can discuss during the call tomorrow or here. If you are ok or not ok moving to post-v1.0.0 speak up below.

I still want this idea. It was one of the earliest topics, and lengthy discussion.
Keeping this in the milestone "post-v1.0.0" and not closing, though there may be merit in opening a new issue with concise overview.
Particularly, now that there are multiple choices of public registries, including running your own, but having an on-premise registry that is not just a pull-through-cache, which can serve up an identified image, which a verifiable author of said content.