Categorygithub.com/the-benchmarker/web-frameworks

# README

Which is the fastest?

Measuring response times (routing times) for each framework (middleware). Each framework has to have two features; routing and parsing path parameters.

Result

Last update: 2018-04-10

OS: Darwin (version: 17.3.0, arch: x86_64)
CPU Cores: 8

Ranking (Framework)

  1. router_cr (crystal)
  2. iron (rust)
  3. raze (crystal)
  4. nickel (rust)
  5. japronto (python)
  6. actix (rust)
  7. spider-gazelle (crystal)
  8. fasthttprouter (go)
  9. mofuw (nim)
  10. lucky (crystal)
  11. amber (crystal)
  12. kemal (crystal)
  13. rocket (rust)
  14. clusterpolka (node)
  15. iris (go)
  16. gorilla_mux (go)
  17. echo (go)
  18. phoenix (elixir)
  19. gin (go)
  20. plug (elixir)
  21. aspnetcore (csharp)
  22. vapor (swift)
  23. sanic (python)
  24. perfect (swift)
  25. kitura (swift)
  26. jester (nim)
  27. polka (node)
  28. clusterexpress (node)
  29. akkahttp (scala)
  30. express (node)
  31. roda (ruby)
  32. rack-routing (ruby)
  33. criollo (objc)
  34. sinatra (ruby)
  35. tornado (python)
  36. rails (ruby)

Ranking (Language)

  1. crystal (router_cr)
  2. rust (iron)
  3. python (japronto)
  4. go (fasthttprouter)
  5. nim (mofuw)
  6. node (clusterpolka)
  7. elixir (phoenix)
  8. csharp (aspnetcore)
  9. swift (vapor)
  10. scala (akkahttp)
  11. ruby (roda)
  12. objc (criollo)

All frameworks

Language (Runtime)Framework (Middleware)Max [sec]Min [sec]Ave [sec]
rubyrails350.161733349.403240349.758695
rubysinatra100.885575100.654648100.776388
rubyroda35.20035334.68404734.891702
rubyrack-routing45.33911244.96314245.080912
crystalkemal8.2393217.8477158.080067
crystalrouter_cr6.6432876.4103306.555974
crystalraze6.9318656.6648256.848396
crystallucky8.1458367.7358748.010866
crystalamber8.0900617.8311988.011325
crystalspider-gazelle7.2400157.0462567.192485
goecho10.2059049.6539889.989534
gogorilla_mux9.9718189.4861479.743084
goiris9.7323939.3876829.565645
gofasthttprouter7.5920367.3266337.452470
gogin10.97327910.56021710.811344
rustactix7.1992036.9975287.099141
rustiron6.8012536.6790356.710507
rustnickel6.9366776.7861946.876099
rustrocket8.6571378.3493068.504091
nodeexpress28.08165826.61105127.016107
nodeclusterexpress17.40964815.77368416.309491
nodepolka15.86742915.06690415.401819
nodeclusterpolka9.3633009.1979169.309864
elixirplug11.25384510.59775710.876218
elixirphoenix11.22043510.08791710.668334
swiftvapor12.49826512.17827012.336104
swiftperfect12.90083912.44451612.684239
swiftkitura13.26319812.49361912.940476
scalaakkahttp18.59594816.55333217.078369
csharpaspnetcore10.93226810.82918610.887798
pythonsanic12.98608411.70688912.437696
pythonjapronto7.1165807.0027427.055527
pythontornado176.956387176.746874176.855119
nimjester14.12224214.05089314.082330
nimmofuw8.0122927.7010517.814884
objccriollo57.23316155.85354056.561862

Current target frameworks (middlewares)

See Development section when you want to add new languages or frameworks.

The rule

We want to know the response time (routing time), not a usability. So full-stack framework is at a disadvantage.

  • Each server has no special logics.
  • Each server's executable is named as server_[Lauguage]_[Framework]. (For example, server_ruby_sinatra)
  • There are only 3 routes
    • GET '/' return status code 200 with empty body
    • GET '/user/:id' return status code 200 with the id
    • POST '/user' return status code 200 with empty body

Installation

Required environment -> See Current target frameworks(middlewares)

By using Neph

Neph is a modern command line job processor that can be substitute for make command.

To compile servers and benchmarker,

> neph

For each language,

> neph ruby

For each framework,

> neph rails

See neph.yaml

By using make

To compile servers and benchmarker,

> make

For each language,

> make ruby

For each framework,

> make rails

Usage

You can take a benchmark by

> bin/benchmarker

For each language

> bin/benchmarker ruby

For each framework

> bin/benchmarker rails

For comparison (Comparing rails, kemal and router.cr in this example)

> bin/benchmarker rails crystal

If you take it manually, you can run each server by

> bin/server_[Language]_[Framework]

and run client by

> time bin/client

You can set # of threads and # of the loops of the request(there are 3 requests in a loop) by

> time bin/client -t 16 -r 1000

In the above example, 16 threads requests 1000 * 3 times. So 48000 requests are sent in total.

Using Docker

Setup servers by using docker is under WIP. Currently, crystal and ruby servers are supported. For example

docker-compose up rails

Then you can run your client by

time ./bin/client

Development

  • Give me PR when you want to add other web frameworks
  • Give me PR when you can tuning each framework (under the rule)

Where should I modify when adding new framework

  • /[language]/[framework]/[codes] <- Project itself
  • benchmarker/benchmarker.cr <- Adding it as a target to
  • README.md <- Adding it as a target framework of the list
  • Makefile
  • neph.yaml (optional)

Anyway, you don't have to care about details since maintainer can fix them after merging it. The result will be updated by maintainer.

Contributing

  1. Fork it (https://github.com/tbrand/which_is_the_fastest/fork)
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request

Contributors

# Packages

No description provided by the author