How to know if WAN IP has changed and receive massive IP address broadcasts

Erik

New Member
#1
The system I am developing potentially has a very large number of clients (lets say one million) that need to periodically update a central server with some information. Clients are written in Java.

The specific use-case is that the server backend needs to have an up to date mapping of IP address to clients. But the client IPs are dynamic and subject to (effectively random) change.

The solution I have in mind requires the clients to ping the server to update their IP. The period ideally should be once every minute, but even 1 ping/10 mins is acceptable.
My questions, in sequence:
  1. 1M pings per 1 min is over 10k/sec. So first off I want to know the approaches can scale to handle such a load. This is to know the options available.
  2. Assuming you have more than one solution in mind, which of these would be the most economical? The cost effectiveness is critically important. I don't have my own data center or static and fat end-point on the net, so the server application will need to run on some sort of provider or ultimately on the cloud.
Notes:
  • I considered running the server from home using my own ISP provided connection, but I am neither sure of the performance issues, nor what my ISP will think about a constant stream of pings.
  • I can't see how the server can auto-discover these IP changes.
 

Admin

Administrator
Thành viên BQT
#2
Erik, your problem is much simpler than it seems to have been made to sound.

This problem been around for a decade maybe two. No need to re-invent the wheel here.

Why Polling/Pinging is a Bad Idea
The dynamic IPs provided by ISPs can have a variable lease time, but will often be at least 24-72 hours. Pinging your server every 1-10m will be a horrible waist of resources potentially making over a 4,320 useless HTTP requests PER CLIENT in a 72 hour period. Each request will be say around 300 bytes * 4,320 wasted http requests equals 1.3mb wasted bandwidth multiplied by your target client count of 1 million clients, you are talking about a monthly wasted bandwidth of ~1.2 TB! And that's just the wasted bandwidth, not the other bandwidth you might need to run your app and provide useful info.

The clients need to be smarter than just pinging frequently. Rather they should be able to check if their IP address matches the DNS on startup, then only when the IP changes, send a notification to the server. This will cut down your bandwidth and server processing requirements by thousands of times.

What you are describing is Dynamic DNS
What you are talking about is "Dynamic DNS" (both a descriptive name for the technology and also the name of one company that provides a SaaS solution).

Dynamic DNS is quite simply a DNS server that allows you to very rapidly change the mapping between a name and an IP address. Normally this is useful for devices using an ISP which only provides dynamic IPs. Whenever the IP changes for the router/server on a dynamic IP it will inform the Dynamic DNS server of the change.

  • The defacto standard protocol for dynamic DNS is well documented. Start here: DNS Update API, I think the specifics you are looking for are here: DynDNS Perform Update. Most commercial implementations out there are very close to the same protocol due to the fact that router hardware usually has a built in DynDNS client which everyone wants to use.
  • Most routers (even cheap ones) already have Dynamic DNS clients built into them. (You can write your own soft client, but the router is likely the most efficient location for this as your clients are likely being NAT'd with a private IP - you can still do it but at a cost of more bandwidth for public IP discovery)
  • A quick google search for "dynamic DNS java client" brings up full source projects like this one: Java DynDNS client (untested, just illustrating the power of search)
Other Considerations for your System Architecture
Lets say the IP-client mapping thing gets resolved. You figured it all out and it works perfectly, you always knows the IP for each client. Would you then have a nice reliable system for transferring files to clients from mobile devices? I would say no.

Both mobiles and home computers can have multiple connection types, Wi-Fi, Cellular Data, maybe wired data. Each of these networks may have different security systems in place. So a connection from a cellular data mobile to a wifi laptop behind a home router is going to look very different than a wifi mobile device connecting to laptop on the same wifi network.

You may have physical router firewalls to contend with. Also home computers may have windows firewall enabled, maybe norton internet security, maybe symantec, maybe AVG, maybe zone alarm, etc... Do you know the firewall considerations for all these potential clients?
 
OP
OP
E

Erik

New Member
#3
Thanks for the breakdown. Your DNS solution should work i think. All clients will query the same dyndns account. and can by doing that see if IP has changed. Is this what you initially meant? That way the client will only ping server if ip has changed. About the firewall problem and how each java client will get the inbound connection, I think UPnP maybe can be used to create port forwarding, not sure. The idea is that the client program is installed on a fixed home pc. If they install it on a Wi-Fi laptop that ip-jumps all day, they have to simply not install the client on that laptop.
 

Từ khóa phổ biến

You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.

Top