Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Throughput problems when compared with Netflix Zuul and Nginx #124

Closed
turgaycelik opened this issue Nov 30, 2017 · 15 comments
Closed

Throughput problems when compared with Netflix Zuul and Nginx #124

turgaycelik opened this issue Nov 30, 2017 · 15 comments

Comments

@turgaycelik
Copy link

turgaycelik commented Nov 30, 2017

We performed some tests by using Apache HTTP Server Benchmarking Tool, ab. Results don't seem promising when compared to Netflix Zuul and Ngnix.

Currently, we are performing Gatling tests for Zuul and Ngnix. Since the current performance of Spring Cloud is not promising, we didn't include it as an alternative at least for now.

Experiment Mean Time Per Request (ms) Request Per Second
Nginx reverse proxy 32.085 6233.40
Zuul (after warmup) 28.422 7036.90
Spring Cloud Gateway 229.058 873.14

PS: Test server is AWS EC2 M4.2xLarge (Eight-Core CPU, 32 GB of Memory)

@turgaycelik turgaycelik changed the title Throughput problems when compared to Netflix Zuul and Nginx Throughput problems when compared with Netflix Zuul and Nginx Nov 30, 2017
@spencergibb
Copy link
Member

Can you give any indication of how you configured the gateway, what version you used?

@turgaycelik
Copy link
Author

turgaycelik commented Dec 1, 2017

Version:
I checked out the code from master on 22/11/2017 (Revision number 9b61b33).
Gateway Configuration:
I added the following method to GatewaySampleApplication class:

@GetMapping("/ind")
public ResponseEntity<Object> proxyIndex(ProxyExchange<Object> proxy) throws Exception {
		return proxy.uri(home.toString() + "/index.html").get();
}

application.yml

management:
    security:
        enabled: false
remote:
  # address of the web server that hosts the static content.
  home: http://ec2-xx-tt-yyy-zzz.us-west-2.compute.amazonaws.com/

Running the Load Test
I used Apache ab for the load test. The test client is another aws ec2 instance residing in the same region with the server ec2 hosting the spring cloud gateway.
ab -n 10000 -c 200 http://ec2-xx-tt-yyy-zzz.us-west-2.compute.amazonaws.com:8080/ind

Status of the master head
I also checked out master today. I see that there are a lot of changes. I performed equivalent configuration. When I run the test with ab, it throws the following error:
apr_pollset_poll: The timeout specified has expired (70007)
To check whether it is a connection error, I sent a request with curl to the URL from the same test client and it successfully returned the static content.

@spencergibb
Copy link
Member

@turgaycelik reactor-netty has issues with http 1.0 and hence ab. reactor/reactor-netty#21

@spencergibb
Copy link
Member

I created a simple benchmark project, so I could share what I've done https://github.com/spencergibb/spring-cloud-gateway-bench

SNAPSHOTS have made some changes that I wonder if they had an impact.

I'd appreciate PRs and any updated stats from your tests. I was just using my laptop.

@kentoj
Copy link

kentoj commented Dec 30, 2017

It looks like we could use h2load to do the testing under HTTP 2.0
See this post.

@rahul-pharande
Copy link

Any update here.@turgaycelik could you please share updated performance numbers with latest spring-cloud-gateway?

@VinodKandula
Copy link

VinodKandula commented Dec 3, 2018

We had similar performance issues with Spring Cloud Gateway(Finchley.SR1), please find comparison metrics below.

				   Spring Boot 1.5.4 + Zuul Gateway	Spring Boot 2.0.4 + Spring Cloud Gateway(Finchley.SR1)
		
Throughput(Req/Sec)				    460				152
Average Response Time(ms)			    107				323

Test Server Configuration: M4.xLarge AWS Instance — 4 Core CPU, 16GB of Memory

It's very confused state if spring cloud gateway can be used in prod, please provide your comments.

The overall system looks like the following

  1. Config Server
  2. Discovery Server (Eureka)
  3. Zuul/Spring Gateway Server
  4. Spring Data (JPA) Rest Repositories(CRUD) Service

All rest endpoints are fired via Zuul/Gateway which uses the discovery server to get the list of available instances.
JMeter is used for Performance tests. It is very straight forward to see the performance results between Zuul vs Sping Cloud Gateway.

#301

@thekalinga
Copy link

@spencergibb Do you have a plan to create a project with standardised setup to compare Zull vs Gateway

@spencergibb
Copy link
Member

@thekalinga yes, at some point

@VinodKandula Any performance test should be done with the latest boot and Greenwich spring cloud.

@dskaredau
Copy link

@VinodKandula can you guys run same perf tests over Greenwich? We are also considering moving from Zuul 1 and now deciding between Zuul 2 or Sping Cloud Gateway

@VinodKandula
Copy link

@spencergibb when we have tested, we have used the latest spring boot and spring cloud versions at that time of our testing and since the performance results were not promising when compared to spring cloud zuul, we have moved away from spring cloud gateway. If you are so confident about the Greenwich release Spring cloud gateway team have to show case/publish the performance results and what the changes made to improve the performance.

@VinodKandula
Copy link

@demoroz we have moved away from spring cloud gateway to spring cloud zuul. And I don't think spring team is supporting zuul 2

@spencergibb
Copy link
Member

@VinodKandula I've published mine #124 (comment)

I'd love to see a project that isn't performant.

@spring-projects-issues
Copy link

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

@spring-projects-issues
Copy link

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants