Closed
Description
On Friday there was a discussion about how pods could be deleted and convey graceful shutdown of processes.
- Deleting a pod implicitly conveys a request to terminate the processes in the pod
- In general, users prefer graceful shutdown of processes: to allow processes the opportunity to cleanly shutdown, which may involve seconds, minutes, or even days in extreme cases
- Processes may occasionally fail to terminate gracefully - they must then be force killed
- Some processes may never terminate due to kernel errors or bugs in code
- While a pod "exists", the name the pod owns cannot be reused
Goals
- Allow users to convey a grace period for shutdown along with a pod and as part of the act of deletion (Consistently support graceful and immediate termination for all objects #1535)
- Avoid creating a different verb for shutdown beyond HTTP delete.
- Allow users to watch and wait for when all of the processes of a pod are no longer running via a specific API call
- Due to the nature of processes this may run forever or for extended periods of time - it cannot be a synchronous http call
- Users who create pods with specific names and then delete those pods often wish to reuse the names of the pods - the longer the interval between when the user deletes and then is able to post with the same name, the more likely the user is to view the delay as a failure of the system, rather than as desired behavior.
- Make it easy and efficient for API consumers to watch on important state transitions within the Pod - created -> scheduled, scheduled -> running, running -> completed.
Non-goals
Assumptions
- If a pod is deleted, the deletion must complete in a finite, bounded by user-input-or-expectation, amount of time -
T(delete)
- in order to free the name for a subsequent create - Processes cannot be guaranteed to terminate within
T(delete)
and so if users wish to continue to watch for process termination, there must be an endpoint that displays the process status of a pod after it has been deleted - A pod name is not guaranteed to uniquely identify the pod across time, but the UID is, so when a user wishes to view the process status of a pod even if it is deleted, they should be able to watch on that pod by its UID (Make deleted objects available from API for some time. #1468)
- If we assume an endpoint that displays pod info along with pod process info even after deletion, that endpoint can be present even prior to deletion and is the natural candidate to watch for process termination.
- Automatically deleting a run-once pod shortly (<30 seconds) after it reaches completion is confusing for end users - pods would disappear without the opportunity to view the UID or react to status
- API consumers may desire to specify the period after which run once pods are deleted at creation time
Options
TBD, sleep
Metadata
Metadata
Assignees
Labels
Type
Projects
Milestone
Relationships
Development
No branches or pull requests
Activity
bgrant0607 commentedon Dec 8, 2014
See also #1535 and #1468.
smarterclayton commentedon Dec 8, 2014
Going to use this as a proposal issue, with those two as reference. Once I get to it... :)
smarterclayton commentedon Jan 24, 2015
This is on my list after uniquification names and pod templatesif no one else gets to it.
bgrant0607 commentedon Mar 5, 2015
One issue that's come up lately: Nothing GCs terminated pods. We could use this issue for that, or file a new one.
smarterclayton commentedon Mar 5, 2015
This is addressed by #5085, and copying the discussion from #1535:
Ok, here's the rough design I'm going with (with various caveats):
Delete(ctx api.Context, name string, options *api.DeleteOptions)
- i.e.DELETE /pods/foo {"kind":"DeleteOptions","gracePeriod":10}
GracePeriod
that is an optional time to delete the resource. If GracePeriod is nil, the default value is used (which comes from the resource type and maybe even from the resource, once pods have a graceful shutdown value). If GracePeriod is 0, termination is immediate (equivalent to current behavior)There is weirdness about removing the pod from the bindings, until the kubelet stops using bound pods we have an unclean setup. It can be worked around in a way that isn't visible to end users though.
smarterclayton commentedon Mar 5, 2015
DeleteOptions should also support "reason" as described by #1535
bgrant0607 commentedon Mar 6, 2015
Actually, reason parameter issue is #1462.
bgrant0607 commentedon Mar 6, 2015
I see a couple issues:
Setting the object TTL is useful for (2). It feels like we need to treat (1) distinctly.
smarterclayton commentedon Mar 6, 2015
soltysh commentedon Dec 1, 2015
@bgrant0607 is there something still that needs working here? IIRC current pod deletion is working as described by @smarterclayton in the issue description. Unless this issue of yours, which, I admit, might be reasonable solution in my use-case in #17940:
bgrant0607 commentedon Dec 1, 2015
@soltysh That specific detail is covered by #1468.
No resources other than pods currently support graceful termination.
We also need to implement server-side cascading deletion, as proposed in #1535.
lavalamp commentedon Dec 3, 2015
Sounds like we can close this -- cascading deletion seems big enough to deserve its own issue, #1535 can serve for now.