Description
I just hit almost the same issue as #14363 (mine reads "ApplyLayer exit status 1 stdout: stderr: chmod /bin/mount: permission denied") , also under Gentoo. It is related to the use of the kernels built with grsec
such as Gentoo's hardened-sources
package.
After some digging, I changed the following:
echo 0 >/proc/sys/kernel/grsecurity/chroot_deny_chmod
This appears to fix the problem, but hits another problem more like that shown in #14363, "ApplyLayer exit status 1 stdout: stderr: operation not permitted". So I tried adding the following from #14363, just to test if it works. (Note that disabling this option appears to be a very bad idea from a security standpoint!)
echo 0 >/proc/sys/kernel/grsecurity/chroot_deny_mknod
The result was that it worked.
To save anyone the hassle in future, Docker could check for the existence of /proc/sys/kernel/grsecurity/chroot_deny_chmod
and /proc/sys/kernel/grsecurity/chroot_deny_mknod
and, if present, ensure the values of both are 0.
Activity
GordonTheTurtle commentedon Feb 13, 2016
If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
For more information about reporting issues, see CONTRIBUTING.md.
You don't have to include this information if this is a feature request
(This is an automated, informational response)
BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version
:docker info
:Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO
globalcitizen commentedon Feb 13, 2016
Note that the best possible outcome here would be for
mknod
andchmod
during image builds to be executed from outside of anychroot
, in order to maintain compatibility with these useful kernel security features (CONFIG_GRKERNSEC_CHROOT_MKNOD
more so thanCONFIG_GRKERNSEC_CHROOT_CHMOD
). See deny chmod and deny mknod documentation.[-]grsec incompatibilities[/-][+]grsec (in)compatibilities (security feature)[/+]globalcitizen commentedon Feb 13, 2016
Related bug against Gentoo docker package @ https://bugs.gentoo.org/show_bug.cgi?id=574664
nahumd commentedon Feb 23, 2016
For everyone who is using Alpine Linux and getting the following error :
failed to register layer: ApplyLayer exit status 1 stdout: stderr: chmod /bin/mount permission denied
Disabling
chroot_deny_chmod
andchroot_deny_mknod
also fixes the problem.unclejack commentedon Apr 11, 2016
This seems to be out of scope for this project. grsec is somewhat of a moving target because the stable kernel patches aren't available for the general public to use. We can't implement changes in Docker which would weaken security on all other Linux distributions in order to improve grsec compatibility.
The best thing to do is to figure out how all these issues can be addressed and maybe send a PR to add documentation on how to run in such a setup.
I'm going to close this issue now since there's nothing actionable for the Docker community to work on this. Please feel free to continue the discussion.
globalcitizen commentedon Apr 11, 2016
@unclejack The relevant grsec functionality has been stable for many years and is a core feature. It does not vary with Linux distribution or manual patching, it is in fact constant.
Let me try to rephrase the issue.
The purpose of grsec's 'deny mknod in chroot' feature is to stop people breaking out of
chroot
(eg. by creating and manipulating a device representing an important filesystem on the system).The purpose of grsec's 'deny chmod in chroot' feature is to stop people breaking out of
chroot
. (eg. by changing privileges on an exposed device to permit an attack similar to the above)The problem is that Docker currently executes the
mknod
andchmod
commands when setting up a layer from within achroot
. By changing Docker's code to execute these commands from outside of achroot
, the problem would be solved.In short, Docker makes it impossible to use useful security features of
grsec
, and as a result, forces even careful users to accept a higher level of minimum exposure. This is a security anti-feature.Ideally, Docker would fix the problem by making the changes suggested above.
Alternatively, Docker should at least check that, if the kernel sysctls above are present, that their value is zero, and if not then give a meaningful error message.
Closing this issue after months languishing here with no useful feedback demonstrates an unimpressive level of commitment to user security on Docker's behalf.
globalcitizen commentedon Apr 19, 2016
Not surprised to see no response here after one week... the docker team's attitude to security seems to speak for itself.
thaJeztah commentedon Apr 19, 2016
@globalcitizen if you have time to create a PoC PR, feel free to do so. Please note that Docker is open source, and "the docker team" includes you. @unclejack raised his concerns that we may not be able to support this on all distros, without weakening security for some. If you have a PoC PR to discuss this, and shows that that change does not negatively affect other distros, feel free to do so.
globalcitizen commentedon Apr 19, 2016
@thaJeztah I have made quite a few security-related contributions to docker, but I'm not paid, so there is a limit to my contribution. For example, I am not going to learn an entirely new language go just to make a PoC for an issue that I consider, given your supposed commercial trajectory toward production deployments, should be resolved by your paid staff rapidly on security grounds.
From comments it seems @unclejack does not comprehend the nature of grsec, which is a set of kernel patches that can be applied to any distro and usually result in exactly the same sysctls being added in a distro-generic manner.
There is no threat of weakening security, this is an entirely baseless accusation, which I assume is based upon the straw man that running
mknod
andchmod
outside of achroot
is a threat.In fact,
mknod
andchmod
are perfectly safe to run outside of achroot
constructing an image layer if their arguments are validated or limited (eg. path-prefixed) in a standard unix-like fashion (as is no doubt done in many other parts of the docker codebase). This will probably enhance potential portability to different/older/newer Unices since jail-style systems are far from unified and by removing that implicit behavioral dependency within the image layer construction process and replacing it with more portable code you could even open the door to systems without anychroot
-like features (eg. perhaps Windows kernels).ncopa commentedon Apr 21, 2016
@globalcitizen nice to hear that you contribute back. I think it is reasonable that you "scratch your own itch" when you don't pay for the product. In fact this issue affects me too but I have not (yet) bothered enough to actually fix it.
After a quick look it does seems fixable without too much effort. Apparently, the
chroot
in question is there to prevent extracting tarballs with symlinks pointing to places outside the given root so I don't think we can simply remove thechroot
without first verify that the tarball extraction is safe.I can have a look at it later, so I vote for re-opening it.
unclejack commentedon Apr 22, 2016
@globalcitizen I'm very sorry if my comment made it seem I don't understand what the grsec patches do and what they are. Leaving that aside, what I was saying is that we shouldn't make changes to Docker if that means having less secure systems in the absence of the grsec patched kernels.
The points I was making in the previous comment are still valid.
Grsec isn't completely available in the open with all stable patches and all changes. We know why it's not open and that's well understood (trademark violations, abuse and so on). The stories behind this can be read online. We won't discuss this.
There's no simple way for the Docker community to provide support for would be grsec+Docker users. The issue of support is a very serious one because we can't test the exact same setup you're using for grsec. Do we all need Gentoo to run the exact same kernel with the exact same configuration, down to the exact same gcc version with the exact same flags? Do we need to have n versions of the Linux kernel with m versions of the grsec patches to make sure it's all OK?
This goes beyond getting it to work once. It's more about supporting it on the long term and being able to help users when things break due to interactions between kernel code changes and grsec or because of Docker code changes. Everyone expects things to work once a setup is described as OK.
Please keep in mind I'm not against grsec or supporting it with Docker. It's a bad idea if we're giving up security elsewhere to enable it to run on some systems.
17 remaining items