14.1 Core Functionality I

Billions of people use the internet every day. You're one of them. But how much do you actually know about it? Few people are aware of what it takes to make such large amounts of information easily available online. But as a product manager, you'll be expected to know what's going on behind the scenes. You'll need to understand how technology—and specifically the internet—works. This foundational knowledge will help you understand the basics of the work your development team is doing and improve your ability to communicate with them. Prospective employers are also likely to ask you interview questions testing your technical knowledge.

In this checkpoint, you will dive into the vast and intricate mysteries that make up internet technology. You will learn about the many different elements that help the internet function and build a vocabulary of technology terms you will need to be fluent in.

Learning objective

If you are new to technology, this module will introduce a lot of new information. It's okay to get a little overwhelmed. Take your time reading through the checkpoints, take notes, review the concepts, and be patient with yourself. You may not know a lot about technology yet, but with some hard work, determination, and all the support available to you in this program—you will become a true techie soon enough.

By the end of this checkpoint, you should be able to do the following:

• Explain how the internet works
• Answer common interview questions testing your technical knowledge of the internet

What is the internet?

At its most basic, physical level, the internet is a network of phone lines, satellites, cables, and wireless radios that connects computers, servers, mobile phones, and other electronic devices. Those devices use shared standards to communicate with one another. The physical network and shared standards are the backbone of the internet, providing users with apps, websites, videos, and many other services. When someone uses these services, they're "going online."

In many ways, the internet works like the postal service. When you search for something on Google, you're essentially sending a letter to Google to ask for information. Then, Google sends a letter back to you with your search results. When you find the right result in the list of websites Google provides you, you send a letter to that website, asking for its content. In turn, that site "mails" you back that content.

Physical letters go on a journey to get to their destinations—they get sorted at post offices, packed into vans and planes, and delivered, one by one, to the address on their envelope. Similarly, an online request gets packaged, routed, and delivered to computers around the world. Those computers process their results, and the results are packaged, routed, and delivered back to the user. And yet, all this happens in the blink of an eye.

How does it happen? To discover the answer, you'll take a journey through this checkpoint—tracking a single web page from an initial request all the way to the final response. As you learn about the many structures and processes involved, you will be learning about the same process used to locate and load almost any other internet-connected application.

Before you dive deeper into the makings of the internet, check out the video below, which provides a brief history and overview of the internet. Gosh, Vox has a great editing team.

Computers, servers, and applications

When you interact with websites, there's a lot going on "under the hood" of your computer. Several basic computer components working behind the scenes include the following:

• CPU: This stands for central processing unit. The CPU functions as the brains of the computer; it's the part that executes the instructions that make up a computer program.
• RAM: This stands for random access memory. RAM is faster than other memory types, and its function is to temporarily store all the information that is in use. This information goes away once you turn off the computer.
• Memory or hard drive: This is where documents, apps, and other information live in permanent storage. These are retained when you turn off the computer.

Another type of computer to be familiar with is a server. Servers are used to manage networked resources. Their processing and memory capabilities support the internet, freeing up your personal computer to focus on less complex work. Some individuals may have a small server in their home, especially if they do sophisticated technical work or play resource-heavy video games online. But most servers are not located in individual homes. They are usually housed in server farms, also known as data centers. These are spaces full of huge servers that provide computing services for a variety of database servers, file servers, mail servers, print servers, web servers, game servers, and application servers.

But computers and servers are only pieces of the puzzle. When you use the internet, your journey starts with an application, such as a web browser. An application is a tool that performs specific functions for an end user or, in some cases, for another application. An application can be self-contained or contain a group of programs. Requests and responses on the internet are generated from applications running on computers, servers, and other electronic devices.

Operating systems

Applications run on a computer's operating system, or OS. Windows 10, macOS Catalina, and iOS 13 are all examples of operating systems. A computer's operating system is responsible for managing inputs and outputs, such as interpreting mouse movements, rendering text to the screen, and sending data over the computer's network connection. The OS tells the physical aspects of the computer how to interact with software applications.

Programmers write applications that access the physical components of a device on an abstract level; the application responds if the mouse is moved or when instructed to send data out on the network. But if application developers had to handle the infinite combinations of hardware components available to users (such as mouses, printers, and networks), it would be impossible for them to create new applications. That's where the operating system comes in; for example, an OS will interpret how a specific Logitech mouse works and handle the details of sending data on a specific Intel network adapter. By providing applications with a generic way to communicate with those physical components, the OS frees up application developers to focus on the application's function.

DNS

Imagine you are sitting at your computer; you've opened your web browser and typed pmcademy.com in the address bar. When you hit Enter, your browser sends a request to the PMcademy server for data. But in order to complete that task, your computer must know the address of the PMcademy server.

Every computer on the internet has an address, just like a physical residence or business. In this situation, your computer must translate "pmcademy.com" into an Internet Protocol address, or IP address, which operates much like a house number and postal code. An IP address has several digits—such as 172.217.12.174—and every computer that is connected to the internet has a unique IP address. When you make the request to pull up information from the PMcademy website, your request must be addressed to the IP address of the computer or server hosting PMcademy's information.

Fortunately, looking up the IP address associated with a website is a standardized process. This lookup process is called domain name service resolution, or DNS resolution. A domain name service is a public application that translates a domain name, like pmcademy.com, into the appropriate IP address. In this case, your computer would make a DNS request to a DNS server asking for the IP address for the PMcademy website.

There are numerous DNS servers that support the internet—also called nameservers. But the complete list of domains on the internet is enormous, and no single DNS server holds the IP address for every website. In fact, the DNS server that receives your request may not have the PMcademy IP address handy. DNS servers tend to save, or cache, requests that they have received recently. That's one reason—among many others—that Yahoo's website might load more quickly than an obscure website hosted in another region of the world.

DNS servers are distributed. If your DNS server doesn't have the IP address for the domain you're looking for, it passes the request on to a different DNS server, which may have the address. The passing of requests can take a little while—maybe 100 milliseconds—but eventually, your computer receives a response: 104.25.216.111 is the IP address for the PMcademy website, for instance. Once your computer is provided with the appropriate IP address, it stores it. The computer will reuse the address for hours, or even days, saving it just in case your browser needs to access the PMcademy website again soon.

At this point, you may be asking several questions. For instance, how does that DNS request get packaged and sent on the internet? And how does it reach the correct server? And if computers are running multiple applications, how does the operating system know which application should handle the incoming data? In fact, how does the system determine which application requests to run first? You will uncover the answers to these questions below one at a time.

Routing

Internet Protocol (IP) defines the system of addressing computers and servers with IP addresses. But it also sets standard practices for sending data around the internet to those addresses—a process called routing.

For instance, imagine you're a server on the internet. You have a network of computers that you're responsible for, and you know their IP addresses. You have a few direct connections to other servers that you know are responsible for other sets of IP addresses. If any traffic is addressed to an IP address that you're not familiar with, it goes to a server that provides the default connection, known as a default gateway. This gateway directs those requests or responses to servers outside your network.

Now, a computer in your network sends you a piece of data destined for an IP address that's not in your network. You check to see if your peer servers have that IP address in their networks, but you don't find it. So you send the data on to your default gateway connection.

That server then performs the same tasks you did: it looks for a network that can receive data for that IP address, and it sends the data to its default connection if it can't find the information it's looking for. This process goes on and on until eventually the data finds a connection to its destination IP address.

But all this happens instantaneously, despite the fact that the servers involved are physically located all over the world. For example, data can travel from New York to Tokyo in 0.2 seconds. GeoTraceroute is a free web tool written by Gasmi Salim, which you can use to trace the route of accessing a particular website from a particular location. Check it out to get a better sense of what routing looks like.

Before moving on, you should learn about one last thing related to routing: outages. What are they? Typically, routers on the internet try to send your data via the shortest, cheapest route possible. But if there's a disconnection along the preferred path—known as an outage—those routers have backup paths to send data around those outages. IP allows servers to decide how to take any piece of data and move it from one side of the internet to another. It might take several hops to get that data from your computer to its destination, and each router along the way makes a choice about the next hop for that data.

In a way, routing is a little like driving a car. If you were driving on a highway and you hit a major traffic jam, you could take the nearest exit and find another way home on local roads. Similarly, routers on the internet can find other ways to get data from its source to its destination, regardless of obstacles along the way. This makes the internet resilient and adaptable—it's able to circumvent outages, disconnections, and other issues that arise.

Packets, TCP, and wrapping

Data is sent online in units called packets, which are usually between 500 and 1500 bytes, or equal to the amount of text in a medium-length online article. If the data you need to send is bigger than that, like a big image or video, it has to be broken down into smaller packets by the sender. These are then sent across the internet separately and reassembled at the destination.

The process of breaking apart and reassembling packets of data is called transmission control protocol, or TCP. The TCP system is sophisticated enough to know whether or not the data it receives is complete, free of errors, and in order. If TCP receives several small components of a large piece of data, it will reassemble those at the destination. If TCP receives jumbled data, it reorders it. If the data has any errors or is incomplete, TCP will ask the sending computer to resend the data and attempt to fix the issue.

TCP and IP are used together often. In fact, they're used together so often that they're usually referred to as TCP/IP. A packet of TCP data is sent within an IP address packet the same way that a letter is sent in an envelope. Routers across the internet use only the IP part of the packet to make a decision about what to do with the data—just like the post office uses only the envelope to route a letter to its destination. Once the TCP/IP packet arrives at its destination, the receiving computer "opens the envelope;" it strips off the IP information, reorders and reassembles the TCP packets, and sends the data to the appropriate application.

TCP ports

Right now, your computer is probably running several applications at once. And many of them are likely sending and receiving data from the internet. This begs the question: when your operating system receives data on its network, how does it direct that data to the correct application? This is handled by another feature of TCP, called TCP ports.

Revisiting the postal service analogy, imagine a single address (like an apartment building) that has many mailboxes. A TCP port is like one of these mailboxes, located at your computer's IP address. An application will reserve a port via the operating system, and the operating system will send packets addressed to that port to that specific application. Your computer identifies the sending IP address and TCP port—the return address—when it sends a packet of data into the internet. Then, the response comes back to that return address.

Your operating system keeps track of the application that is "listening" on each port, and the incoming packets are delivered to the appropriate application based on the operating system's port mapping. Standards have even been established to assign some of the most commonly used applications to a specific port, which increases technological efficiency. For example, any DNS request you make on your computer will be automatically sent to port 53 on a nameserver.

So as you enter pmcademy.com in your web browser, this prompts your device's operating system to send a DNS request. The DNS request gets routed to the appropriate nameserver, which sends back a response with the server's IP address. And all that occurs just to get your system the information it needs. Once the computer has that information, it can request the web page content from the PMcademy server.

HTTP

Hypertext transport protocol, or HTTP, is the standard used to request and send web data across the internet. In other words, while DNS is the standard protocol used to figure out where the data is located, HTTP is the standard protocol used to request what's there.

HTTP specifies the format and types of data that can go into a request for web data. HTTP requests are wrapped in TCP packets, which are wrapped in IP packets. Computers that respond to HTTP requests are called web servers, and they listen on port 80 or port 443. Web server applications, like Apache or Nginx, handle many common requests, like how iMessage receives incoming SMS messages on an iPhone.

What does this actually look like on your computer? Well, your browser makes a request from the web server by assembling an HTTP request. The receiving web server needs to know exactly what resource you want. This is indicated by a Uniform Resource Locator—more commonly known as a URL—like https://google.com/search?q=pmcademy. Web servers then interpret that URL to identify the data you are requesting.

A URL has a few parts:

• The protocol is the part that comes before the ://. The protocols http and https are the most common, but you might see ftp on occasion, too.
• The domain is a server that runs the website. In this case, it's google.com.
• The path is /search. It's the specific resource you're requesting from the Google server.
• The query string is the bit after the ?—specifically, it's the q=pmcademy. This is the data that will be processed by the server.
• Some URLs have an anchor at the end of the URL, which begins with #. This is processed by the web browser, and it usually takes you to a specific part of the page after it loads. It is not passed to the web server.

When you enter a URL into your browser, the browser application translates it into an HTTP request that the web server can understand. For instance, if you were to click on the following link from the PMcademy's home page—PMcademy's product management website—an HTTP request would be sent. Below, you can see part of the request that is sent over HTTP to load this page:

GET / HTTP/2
Host: www.pmcademy.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36
referer: https://www.pmcademy.com/

There are several key parts to this type of HTTP request:

• The host: A single web server on a single IP address can run the websites for multiple domains. Because of this, you need to tell the web server which website you want—in this case, www.pmcademy.com.
• The method: The GET part above tells the web server that you want the information from the domain and path specified. Another common method in HTTP requests is POST, which is used when a user submits data, like when filling out a form.
• The path: Immediately after the method, you'll see the path of the URL that you want. You can make a request for the domain with no path (or just /, which is the same). That's called the root of the site.
• The user agent: This is the version of the browser and operating system that together are making the request. The example above is from Chrome running on Windows 10. Web servers and web apps can use this information to return different information. For instance, a request made from an iPhone will return a mobile version of a website.
• Referrer: This is misspelled as referer in the HTTP request because that's how it was written in the HTTP standard. This is the site that led to this request. If you searched on Google and clicked on the top search result, the referer would be Google's URL.

So when you request information by entering a pmcademy.com URL into your web browser address bar, the web server translates that into an HTTP request. The web server then passes that HTTP request to the operating system of your device. The operating system sends the request to the PMcademy server via TCP/IP. That packet gets routed across the internet until it's received by the PMcademy server. The PMcademy server's operating system passes the request to a web server application like Nginx, where it will get processed.

HTTP Processing

Some HTTP requests are easy to process, such as requests for static (unchanging) content, like an image or a piece of text. The web server loads the data, wraps it in an HTTP response, and then wraps that in a TCP/IP request for easy transmission. The data then returns to the requester. Easy peasy.

Other HTTP requests, however, need to be processed. For example, your Google search URL for PMcademy requires a more complex series of steps. Google servers need to process the query before they can send the user a result.

The web server app can identify whether requested data is static or needs processing. If it needs processing, the web server sends the request for data to an application that can process it. How does the web server know which application to use? Software engineers configure the web server to recognize URLs that require processing and to use specific applications to process them.

When requests that require processing are received from the web server by the application, the application runs the program developed to respond to that type of request. That program may call other applications and programs on different servers. Once the task is complete, the program will send the response back to your browser via HTTP.

This processing can take milliseconds—the time it takes to conduct a Google search, for instance. Or, it can take several seconds—for example, the system may need more time to process a money transfer between bank accounts. This is an important thing to keep in mind as you design your products and experiences. You'll want to inform users when processing might take longer than they may expect. And as an internet user yourself, you likely have a newfound appreciation for all the data that's exchanged, in just fractions of a second, to get you to any web page on the internet.

The response

The HTTP standard also specifies the format of the response. The response usually has a lot more information than the request, and it looks something like this:

HTTP/2 200
date: Mon, 13 Jan 2020 16:29:44 GMT
content-type: text/html; charset=utf-8
set-cookie: redacted
path=/; domain=.pmcademy.com; HttpOnly; SameSite=Lax
<html>  snip  </html>
]

It includes the following components:

• Status code: This is the 200 in the first line above. The status code tells you if the request was successfully fulfilled. These codes are well-specified in the HTTP standard. When there's an error, the codes typically start with a 4 or a 5, and the browser or app will show an error message. You're probably familiar with common errors like 404 Not Found. The 404 status code indicates that there was no content that matched the request.
• Headers: These provide the information your browser needs. They include the following:
  • Content-type: This gives your browser guidance on how to process the data. For instance, an image needs to be handled differently than text. These content types are well-defined, so browsers handle them consistently.
  • Last-modified: Browsers will cache as much data as possible to avoid network calls or additional processing. If the browser has the content cached and the last-modified timestamp indicates that the content hasn't been modified since it was cached, your browser knows it can use the cached data. It can then close the connection.
  • Set-cookie: This is a list of cookies that the website wants your browser to store. The list identifies each item, using a semicolon to break them up. Although it might look like a single sentence, this is a very common way to transfer multiple pieces of data. It is referred to as a delimited list. A specific type of punctuation mark—semicolons, commas, or | are common—delimits each item from the others. In this case, the delimited list of cookies helps the web server and applications keep track of individual users when they access the site.
• The body: This is the data for whatever content was requested, such as a web page, image, or video.

The HTTP response is often too large for a single packet, especially if it's an image or a video. In these cases, the response is broken into several TCP packets, which are wrapped in IP packets. They are then sent back over the internet and reassembled upon arrival.

Understanding the core functionality of this request/response cycle is a good first step to grasping how the internet works. Next, you'll learn more about advanced internet functionality, like secure websites (https) and HTML. Keep going; it will all come together!