Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support backup of v2 and v3 stores #7002

Closed
heyitsanthony opened this issue Dec 14, 2016 · 5 comments
Closed

support backup of v2 and v3 stores #7002

heyitsanthony opened this issue Dec 14, 2016 · 5 comments

Comments

@heyitsanthony
Copy link
Contributor

Right now etcd backups must choose between v2's backup and v3's snapshot. For systems that store v2 and v3 data at once, there is no way to fully back up the cluster. This behavior is somewhat unexpected since presumably a backup of the cluster should restore all the keys that were on the system.

@raoofm
Copy link
Contributor

raoofm commented Mar 30, 2017

I think this should be a really high priority rather than just an enhancement. It puts all the existing v2 users who moved to v3 without migrating data (or with migrating data but not all clients migrated yet) at a huge risk.

@r7vme
Copy link

r7vme commented Jul 28, 2017

i've just experimented, after reading blog post from @mattymo , and found out that v2 backup actually contains both v2 and v3 data.

I have etcd 3.2.0-rc.0+git

To check

Start etcd

mkdir /tmp/etcd-data
etcd -debug -data-dir /tmp/etcd-data

Create v2 and v3 data

for i in `seq 1 10`; do curl http://127.0.0.1:2379/v2/keys/message$i -XPUT -d value="Hello world"; done
# create v3 foo bar (base64 encoded)
curl -L http://127.0.0.1:2379/v3alpha/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'

Create backup

mkdir /tmp/etcd-backup
etcdctl backup --data-dir /tmp/etcd-data --backup-dir /tmp/etcd-backup

Stop existing etcd and start new one from backup

etcd -debug -data-dir /tmp/etcd-backup --force-new-cluster

Check values

~|⇒ etcdctl ls                                                 
/message4
/message5
/message8
/message9
/message2
/message3
/message7
/message10
/message1
/message6
~|⇒ ETCDCTL_API=3 etcdctl get "" --prefix=true 
foo
bar

Is this smth unexpected?

@heyitsanthony
Copy link
Contributor Author

heyitsanthony commented Jul 28, 2017

@r7vme there's probably v3 entries in the wal that are being copied over and then replayed back into the v3 store. It's not an actual backup

@raoofm
Copy link
Contributor

raoofm commented Aug 1, 2017

@xiang90 @heyitsanthony Should this issue be closed in favor of #6925. Then the users can migrate v2 data to v3 and still support v2 clients. So the current problem of taking both v2 and v3 backup as a whole can be dropped as we would then need only v3 backup.

@heyitsanthony
Copy link
Contributor Author

Fixed by #8479

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

3 participants