Skip to content
This repository has been archived by the owner on Dec 18, 2021. It is now read-only.

support protobuf c++ implementation? #4

Closed
yuz11 opened this issue Dec 9, 2017 · 3 comments
Closed

support protobuf c++ implementation? #4

yuz11 opened this issue Dec 9, 2017 · 3 comments

Comments

@yuz11
Copy link

yuz11 commented Dec 9, 2017 •
edited

I pull a large number of data(70w+ rows)，found it pretty slow when it runs to
client.py:506 (1~2 second dealing with one bugket data in a fetch())

        response_data = self._apply(request)
        response = responses_pb2.FetchResponse()
        response.ParseFromString(response_data)

I guess pure python protobuf make it slow.
python protobuf c++ implement may help
would u share the .proto files so that I can make a test by myself?

Regards,
Rafer
@lalinsky
Copy link
Owner

lalinsky commented Dec 9, 2017

You can get the PB files here:
https://github.com/apache/calcite-avatica/tree/master/core/src/main/protobuf

@yuz11
Copy link
Author

yuz11 commented Dec 10, 2017

thx~I'll take a shot

@yuz11
Copy link
Author

yuz11 commented Dec 29, 2017

u can close this issue now. I just run a test, protobuf with c++ implement is about 10 times faster than pure python version.But it's just a matter of python protobuf lib version, do not have to rebuild the message.
python 3.0 or ++ pip install will do; python 2.x, will have to setup by source code.
Regards.Sorry for taking this long.Have been busy these days

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lalinsky @yuz11