This blog post takes a look at the traditional TCP/IP standard of network acceleration, and why we’re not using it for our customers. Instead, we’ll go into some technical detail on the Neumob Protocol (NP); how it’s constructed, and why it’s fast becoming the standard for true app acceleration and error reduction within mobile apps.
TCP’s Legacy Approach, and Where It Falls Down in Apps
HTTP/1.1 only allowed a single request over one TCP connection. If you wanted to make multiple requests, you had to wait for the first request to finish in order to start the second request, and so on. If one request takes longer to finish, it holds up the line for a longer time, and all other requests behind it in the queue have to wait. This problem is called “Head of Line Blocking” (HOLB). You might assume that there would be HTTP/1.1 pipelining, but it still has same HOLB problem and is rarely supported in most browsers and proxies.
To overcome this, web and app developers started making multiple concurrent TCP connections in order to boost speeds. Yet that approach doesn’t scale, since maintaining each TCP connection requires memory and CPU resources.
Google then came-up with SPDY, which later became the foundation to the standard we know of as HTTP/2 today. The idea here is to multiplex requests over a single TCP connection. This approach does solve the problem of HOLB at the HTTP layer, since you now don’t have to wait for a single request to finish before starting other requests. That said, it is still limited by the same HOLB problem at the TCP layer. This is a fundamental limitation of the TCP protocol itself, because it requires “in-order”, or sequential, data.
Let’s look at a visual example. If 5 requests are being sent in a single TCP connection over HTTP/2 and request #3 experiences packet loss, with rest of the requests make it through the receiving end, the receiving TCP stack has to store packets of request #4 and #5 at transport (TCP) layer until it recovers from loss on #3. Only then can it send #3, #4 and #5 to the application layer to be served to the customer. One could argue that HTTP/2 can even make things worse in certain situations.
Why UDP, and How it – and Userland Network Stack – is Evolving
At Neumob, we solve this problem by using UDP under-the-hood for our protocol – the Neumob Protocol (NP). UDP doesn’t suffer from HOLB, as it inherently does not require in-order data delivery.
Our focus has been to create a mobile-first protocol, designed for the mobile apps in which 85-90% of all smartphone activity occurs, rather than taking a legacy protocol designed in the 1990s, and then retrofitting it to work for mobile world.
The Neumob Protocol (NP) accelerates everything within a mobile app, including all of those great (but heavy) 3rd-party calls like videos, images, ad network SDKs and analytics tools that make an app what it is. It doesn’t cache for one domain only, and it doesn’t meekly “tune” TCP. Instead, we chose to develop our own robust UDP-based protocol, 3-POP WAN acceleration architecture and software-defined content routing that dynamically does one thing exceptionally well: speed up the performance of mobile apps, no matter whether its users are in the same city or halfway around the world.
Let’s consider the scenario pictured above with NP. With NP, all of the requests other than #3 would be sent to upper layers as they arrive. That loss on #3 doesn’t block any other request from getting served to the customer. This results in much more responsive web pages, or in our case, in much faster and more responsive mobile apps.
One of the differentiating features of NP is network profiles. More than half of the connections we serve are wireless: 4G, 3G (WCDMA, HSDPA, EVDO_A), 2G (EDGE, CDMA) and so on. Even in the same LTE network, each mobile carrier will have different coverage and latencies, and all of these networks have different characteristics.
Network Detection – And Much More
The Neumob Protocol, or NP, has the ability to detect if the network connection is on, and tune NP connection parameters accordingly. With the Neumob SDK, NP is able to detect the mobile network carrier, the network technology (WiFi, LTE, HSPA etc) and the country in which the device is connecting – then apply different protocol parameters to maximize mobile app speed and error reduction.
Other than that, NP also provides:
- multiplexing of requests over a single UDP connection
- data compression to reduce bandwidth requirements
- innovative loss-detection and loss-recovery methods
- support for secure connections using TLS
This transport layer protocol stack is implemented in userland – not requiring any kernel-level changes, nor does it require any special permission other than sending/receiving UDP packets.
This makes it simple to distribute a new protocol stack without any system software upgrades, and provides for easy customization and easy troubleshooting, while remaining platform independent. These days many OS vendors accept userland networking stack – for example, Apple announced in iOS 11 that the OS will have userland TCP stack. Neumob has pursued this approach from Day 1, and our UDP-based approach continues to be validated by the ongoing work happening with QUIC as well.
Introducing 3-POP Acceleration
Traditionally, web-based CDNs have used edge servers (1 POP) in order to cache static objects efficiently. This is good for small web sites with a low amount of calls, but when the total size of typical libraries grew bigger, CDNs introduced another concept of placing a 2nd level of cache in a few aggregation points (called parent cache, shield cache, super cache, super POP etc), near the origin in order to improve the cache hit rate in the edge server, while reducing access to the origin.
This approach was also useful for accelerating dynamic objects (not cacheable and in need of origin access every time). These days most CDNs support accelerating dynamic content in their own way, but this 2-POP approach is pretty common. Having edge POP and another POP near the origin, and using various middle-mile acceleration techniques between edge POP and a POP near to the origin is foundational architecture the allows CDNs to accelerate dynamic content.
Neumob has expanded this idea to the actual device in the user’s hand. CDNs take what is basically a server-side only approach, with no information about the device itself, and simply assumes it’s “a good client”. This assumes it has a good DNS resolver configuration, so that it can find a nearby edge POP using DNS (or relying on anycast to find a nearby edge POP), and that it knows how to connect using an up-to-date protocol.
Neumob’s 3-POP approach, by contrast, hosts a small and intelligent proxy right in the device itself by virtue of our embedded SDK within the app being used. Traffic from the app travels through our small edge server in the device. This enables us to get unique information about the client, while providing Neumob with the ability to optimize the last mile from the edge of the internet to the device itself, something was not possible in the traditional CDN approach.
For example, Neumob can identify that the device is connecting to a Wifi network or to LTE via a specific mobile carrier, without guessing, which enables Neumob to apply the most appropriate protocol parameters. Neumob is able to fall back properly when anything bad on unexpected happens during content transmission, which reduces errors, collects more detailed metrics about the request, alerts about unusual errors, and more. This is effectively having “an intelligent agent” on the device that’s constantly reporting on network connections. This 3rd POP is implemented in the Neumob SDK when integrated within a mobile app.
How All of This Reduces Errors Within Mobile Apps
Finally, it’s important to underscore how effective the UDP-driven Neumob Protocol is in reducing errors within apps. These errors include timeouts, when an app’s responses effectively “freeze”, and force the user to refresh or navigate elsewhere, since images or other content can’t be delivered.
Errors can also include blank spaces with missing images; third-party-hosted content that never arrives; and even advertisements that are never seen by the user (and therefore can’t be monetized) because of failed delivery. Typical mobile app error rates range from 1% on faster networks such as LTE, to over 12% on 2G & 3G networks, and in countries such as India and China.
By not being inherently limited by HOLB (“Head of Line Blocking), the Neumob Protocol already provides apps with a leg up in reducing these frustrating errors. It also uses innovative loss detection & recovery mechanisms, while providing fine-grained control with the aforementioned 3rd POP implemented right inside the SDK.
TCP has been the communication language of the web, built for a PC-focused internet whose best days are behind it. Since the late 1990’s, CDNs have done a fantastic job of speeding websites to end users on PCs who have robust connections to the Internet. When early smartphones began to change the nature of how we interacted with the internet, the CDNs were right there with us, speeding up mobile websites as well, using many of the same tools they used in the pre-mobile era.
However, the traditional PC-focused Internet and TCP/IP protocol were never designed to support the fast delivery of mobile apps. Both introduce a number of delays throughout the mobile app delivery process, making fast mobile app performance (and low error rates) on end-user devices an elusive goal for most developers.
The Neumob Protocol was created to address these challenges, and because it has been specifically and exclusively engineered for mobile apps, it by necessity incorporates a variety of next-generation improvements and network-driven leaps forward in order to achieve mobile app speed gains of 30-300%, and reduction of in-app errors by up to 90%.