Handling overload effectively with load balancers

  1. 1. Fail Smarter
  2. 2. Load balancers
  3. 3. Actual behave
  4. 4. Abnormal cases

1. Fail Smarter

  • Average latency has a spike, availability down
  • reasons
    • dependencies
    • cascading effect
    • client change peak

2. Load balancers

  • clients - load balancers - servers
  • hardware devices
  • multi tenant - use VIP - for efficiency
  • used for (features)
    • scaling
    • even traffic distribution
    • overload protection
  • how
    • how to pick a server
      • random dice
      • round robin
      • least conns
    • desired algorithm - we use least conns
      • simple
      • reliable
      • even
    • max conns?
    • how to deal with overload?
      • attributes
        • cheap
        • local
        • buffering - having capacity soon
        • priority
      • reject requests - spillover - choose!
        • close the TCP connection
        • no buffer
      • hang on, wait in a queue - surge queue
        • may mkes it take longer
  • some default we set - maxConns - perhost setting
    • little’s law
      • arrival rate * time shopping = people in the store
    • see load balancers how many services we have
    • fleet-wide concurrent requests / host count = avg conns
  • metrics matter
    • latency netwrok latency + 25%
      • client side
      • server side
    • request rate
  • coral server
    • concurrent requests - outstanding request - in one host
  • 33% overhead room - dependency failures

3. Actual behave

  • increase load to see average latency
  • run actual test
  • generate graph with outstanding requests
    • see the cross of client timeout and p99

4. Abnormal cases

  • dependency latency, timeout
  • network packet loss
  • let server decide what’s the maxCon should be
  • coral has
    • connection
    • worker work thread
  • classify requests
    • importance
      • droppable
    • priority
      • order

转载请注明来源,欢迎对文章中的引用来源进行考证,欢迎指出任何有错误或不够清晰的表达。可以在下面评论区评论,也可以邮件至 stone2paul@gmail.com

文章标题:Handling overload effectively with load balancers

文章字数:215

本文作者:Leilei Chen

发布时间:2020-02-04, 09:13:39

最后更新:2020-02-04, 09:14:06

原始链接:https://www.llchen60.com/Handling-overload-effectively-with-load-balancers/

版权声明: "署名-非商用-相同方式共享 4.0" 转载请保留原文链接及作者。

目录
×

喜欢就点赞,疼爱就打赏