IP Management : How to scale ?


  • Clusterisation (master slave) 
  • Replication
  • Persistence with

  •  log file (AOF)
  • Persistence with snapshot (RDB)

Current Architecture


  • ips-provider-api : requested by targetting loop, sizing for a specific time window ?
  • taskmaster : easy horizontal scalability (lock on country)
  • theWatcher : not an issue
  • sdk-task-server : same as wsback

  • postgres-sql : vertical scalabilty only, depend on subscription import
  • redis : ...

Datas in redis

  • IPs -> Subscriptions : no eviction policy, big scalabilty !
  • Subscriptions -> IPs : idem as previous
  • Stats : no eviction policy, no scalability issue
  • Tasks : no eviction policy, vertical scalability
  • Cache : possible eviction, medium scalabily

Scaling with Redis

  • No native horizontal scalability
  • Vertical scalabilty : means to have a limit...
  • Using couchbase for IPs/Subscription : we loose the SET structure that is very usefull for our system
  • Tweemproxy : means that we can loose keys when adding nodes

IPs / Subscription eviction... serious issue ?

  • For reverse DNS : No ! Even if we loose all keys, will be regenerated quickly and data will be able to work with the day-1 S3 export
  • For DNS : Yes ! If we loose a targetted subscription, we will target no user !
  • BUT => It concerns only campaign's subscription !

Our Solution

  • Split Redis into several clusters (ips->subscriptions, subscriptions->ips, stat, cache, queue)
  • Create a small "safe" redis cluster (memory margin to prevent eviction, replication) that contains only subscriptions -> ips keys
  • DNS request will request first the small cluster, then the "big" one
  • A cluster for all "ips->subscription" and "subscription->ips" behind a twemproxy : add possibility to scale horizontally (but we will loose some keys)

New architecture

Architecture - dev

Sizing objectif

  • Subscriptions : 2 millions (top 1 million Alexa x2 countries)
  • Ips per subscription : 5
  • Total : 10 millions IPs

  • 1 postgres, 2 twemproxy, 12 redis

Create a presentation like this one
Share it on social medias
Share it on your own
Share it on social medias
Share it on your own

How to export your presentation

Please use Google Chrome to obtain the best export results.

How to export your presentation

New presentation

by sebastien80a


Public - 9/13/16, 7:15 AM