Third time's a charm? :)

In think my comments in the previous proposal still apply;

1. Since we're defining a "stack" not a "group", perhaps the file should be called Dockerstack.yml? (Proposal: stack composition #9175 (comment))
2. For naming and possibly multiple instances of a stack (Proposal: stack composition #9175 (comment))

Thanks (again), will give these builds a try soon.

As has been discussed in fig issue #159 are there plans to allow for sharing (in this example) redis between YAML files? So I can have 2 applications that both get the same redis when I docker up?

@andrewmichaelsmith I agree that we should support links to existing containers. This should actually work in the test build (though I haven't stress-tested it much) - try prepending a link with a slash, e.g.

links:
  - /external-redis:redis

Zooming out, ideally there'd be a way to do "dependency injection" of containers, so I could e.g. specify a stock redis container in group.yml and then override it in production so it points to my already-running, separately-managed Redis instance.

Including the app name in the file.
I’m unsure about making this the default - lots of Fig users want to be able to do it, but I’m worried that it’ll hurt portability (we don’t do it with Dockerfile, and in my opinion it’s better off for it). Alternate approaches include using the basename of the current directory (like Fig does), or generating a name and storing it in a separate, unversioned file.

tl;dr

Are there use cases for referencing the app name besides building test or dev containers? If not, I think you should keep the current fig behavior, but also support referencing other containers in a FROM :web style. If you don't add support for FROM :web, then I do support adding an app_name key to the yaml file, since that removes broken behavior when working in differently-named directories.

I'd love to do FROM :web to build a tests-only container, rather than have to fig build web && FROM myappdir_web.

Using fig for building testing containers

I'm currently using fig to build a container for running tests, which requires knowing the basename of the current directory. This is problematic if devs check out the code into differently-named directories. Let me show you an example, using the sample python app you provided in the ticket description.

In addition to the Dockerfile and fig.yml you provide above, I also have a dockerfiles/test directory which contains the following Dockerfile:

FROM myapp_web
ADD requirements-dev.txt /srv/myapp
RUN pip install -r requirements-dev.txt

requirements-dev.txt:

nosetests

And I add these lines to fig.yml:

test:
    build: dockerfiles/test
    environment:
        DEBUG: True

This lets me do: fig run test nosetests, and lets me avoid polluting the production container with my development dependencies. However, it also means that before running the tests I have to do fig build web so the myapp_web tag will get created.

The detail to note is FROM myapp_web, which is of the format <myapp_basedir>_<fig_base_container_name>. I'd love to do something like FROM :web. Or, if support for that will not be added, then I'd at least like to be independent on the directory name. A nice bonus would be allowing me to run fig run test without first building base (I'd like fig docker composition to just figure that out).

I haven't found a good pattern for building containers for testing, so please let me know if you know of a better pattern that sidesteps these issues without adding new features.

I haven't found a good pattern for building containers for testing, so please let me know if you know of a better pattern that sidesteps these issues without adding new features.

@sbuss Fig previously supported --projectname (or --project-name) Can't remember right now off the top of my head. I codified this in the project Makefile with something like FIG:=fig --project-name foo, then use $(FIG) everywhere. If you're invoking fig through the shell itself, there is an environment variable you can use as well. All in all I've found the directory scoping things very annoying in general. I'd like to opt out of it completely since I run everything inside a VM anyways, so no collisions can happen regardless.

@aanand I previously used fig for development environments for my team. I switched away because it did not expose docker's full functionality or made it awkward in places. The previous paragraph also speaks to this. Does your implementation also postfix everything with _N e.g. some_container_1? Also does your implementation also remove underscores from containers/group names? This may be a small gripe but I found it be very annoying if you ever wanted to do something outside fig's context. There were times when I wanted to start a one off docker container via docker run but had to link to something created with fig. This requires you to know two things: the namespace may change (if not using --project-name as mentioned above), that container name is not what's listed in the config files. Of course you learn these things but it's a strange hurdle to jump and introduces odd dependencies between different things. Does your proposal include referencing containers for use with --link? e.g. --link :posgres:db. If not I think you should certainly consider it. That abstraction should stretch out to all docker aspects of the docker CLI. Also, does this mean docker groups will be built into docker officially and use the same internal APIs as the CLI? I also ran into problems with fig where the API it communicated with did use the same configuration as the daemon (--insecure-registry). Will such concerns go away in your implementation?

This is awesome!

I would say keep the name: optional and use directory as default, like fig does. But it's definitely a needed feature. One thing I don't like about fig is that I am not able to choose what my images are named aside from renaming directories.

And I don't want to be a killjoy here but, this really is just fig. Re-implemented in Go and looking to find its way into the core. So my preference, as a heavy fig & Docker user, would be to keep this stuff separate.

• It's mostly syntactic sugar.
• It already exists as a healthy and active open source project
• Fig is already governed by Docker, so it's direction is not an issue. Any desired improvements can find their way into fig instead.
• The direction we've been hearing about is towards a tight and modular core.

In the argument as to whether to have compose included in the docker binary or separate, I vote included. I've been using fig and found myself wishing I could do docker up instead of fig up.

Other thoughts:

• It would be great to support the --env-file option in compose from the beginning. Having a file for environment variables is a nice way to tweak the container config instead of having to tweak the fig.yml/group.yml files directly, especially for teams where some settings are individual (e.g. a catchall email address for the dev container). (fig issue: Support for --env-file docker/compose#479)
• I find fig adding _1 to the container name really annoying, especially as I don't need the scale option. It would be great if compose didn't do the same. and when a scale option is implemented, subsequent copies can start from _2 onwards.
• As well as having a file (group.yml) which docker up magically knows to use, it would be great to also be able to provide the file directly, e.g. docker up myconfig.yml . I have a folder with several fig.yml files and they each have to be in their own sub-folder to be usable, which is a little annoying. (Not valid - thanks for the tip @itorres )

@aanand 👍

@tomfotherby: about the fig.yml per directory I think you're missing this fig option:

$ fig -h
[...]
-f, --file FILE           Specify an alternate fig file (default: fig.yml)

I'm not sure that I really like the idea of including composition in the main docker binary. I see composition/fig as a (very) convenient rather than core functionality (very as in "I don't really use docker without fig nowadays").

I vote to keep Compose separate. It will be much easier to add the hooks necessary to do cluster management between Compose and Swarm without risking regressions to Docker. Commands like 'docker up' could be added when Swarm is installed onto a system where Docker resides - ie: shell aliases.

I vote to keep Compose separate, since its functions are largely independent from the core and bear quite different responsibilities. Modularity would make maintenance (of both the core & Compose) and third-party integration easier. So far I'm not aware of strong technical reasons against separation.

Please keep it separate.
It's best having different tools, each with a clear and simple goal.
This will make life of users better by making each tool purpose simple to understand and letting them choose which tools to use for each purpose (several projects are working on composition already). This will make life of coders easier by reducing test/scope surface and dependencies.

I agree to separate it. Just like git-review, if you installed it, you can use either git review or git-review.