Launching an App
When you use your iPhone you move fluidly between different activities, from texting to web browsing to social media to photography. These separate experiences are all enabled by a complex computer system.
Each time you take your iPhone from your pocket, unlock it, and open an app, you’re bringing an entire system to your fingers. Your phone is a computing device—it’s a physical object that can run programs. It often relies on a whole ecosystem of other computing devices. For example, if you use a printer or a desktop or laptop computer along with your phone, you’re using a computing system—a collection of computing devices that work together to accomplish a specific purpose, like shooting and editing video or creating a podcast.
The word app is common today, but it’s a relatively new term. A much older and more general word for code that runs on a computer is software. (The complementary term hardware refers to a physical machine that can run code—in this case, an iPhone.) Program is another term that also refers to a collection of code that performs a specific task. The term app itself is derived from the word application, another computing term that has existed for a long time. Application software is a category of software that helps users perform a related set of tasks. The words software, program, and application are still used, but nowadays app is the way people refer to a package of software that they install on our phones and other computers, access with a tap or a click, and use to accomplish particular tasks.
Much of what you see on your phone arrives via the internet—a computing innovation originating in the 1960s. The internet is another example of a computing system. The purpose of the internet is to enable any computer in the world to connect to any other computer, without having to know how that connection operates.
The internet, like any computer network, comprises many individual computing devices—computers, routers, switches, and more—and the software that enables it to function. Networks enable their connected devices to exchange data using wired or wireless links. Most networks don’t connect all their component devices directly; they route messages through multiple links to move information from a sender to a recipient.
Search
Searching on the web or in an app seems simple, but there’s a complex process beneath the surface. Your search results are personalized based on the information the service that powers the site or app has about you already—and the ads that appear alongside or among the search results are also targeted.
If you have an account, the service already has information about you, such as your gender and age, which are powerful ways to customize the results you see and to target marketing messages. But there’s lots of other data that can be used as well.
For example, your past searches can influence your current search results. A service can store all your past searches, along with their results. By knowing what you’ve been searching for recently, a service can predict the context of your current search, and show you ads that might be relevant.
This can work even if you don’t have an account for the service. If you use a website or app without being logged in, the service can still collect information about you. Your location can be deduced from the IP address your browser included in its request, along with the kind of computer and operating system you’re running. And your browser can also store cookies—small chunks of data associated with individual sites. A cookie is a way to uniquely identify your computer so that your browsing history on that site can be remembered. Later, when you visit the same site to search for something, the service can recognize your cookie and look up all its recorded data about you.
Online Activity
Creating a New Account
In order to provide you with a greater amount of personalization, websites and apps may ask you to register, or create a new account. When you register, you often provide information that personally identifies you, such as your name and address, birthdate, and perhaps even some of your medical or financial data, depending on the nature of the service. Once you’re registered, a service may associate your profile with information that’s already stored in cookies on your local machine. This means that anything you did before you were known to the website can now be linked to you. This can potentially have a harmful impact, linking your previously anonymous online behavior to you in ways that you didn’t intend or anticipate.
Passwords
Most services require you to choose a strong password when you sign up. This means choosing a password that’s not easy to guess, like your pet’s name, or a simple word or numeric pattern. A strong password shouldn’t be too short, and it should have a mix of uppercase and lowercase letters, numbers, and symbols. The longer and more complex the password, the harder it will be for a program to guess it by trying all of the possible combinations of characters. It can be helpful to use the term “passphrase” instead of “password.” Create a sentence that relates uniquely to you, then include capitalization, punctuation (including spaces between the words), and numbers. You’ll make something that’s very hard to guess, but easy for you to remember.
You shouldn’t reuse the same password on multiple sites, either. If one becomes compromised, then all your personal data is at risk. If you use a password manager, you can create and keep track of unique, complex passwords for each of the sites you log into. A password manager can generate very strong random passwords automatically, and it can fill those passwords automatically on the sites you visit. You’ll need to lock the password manager, too—which is where you’ll use one very strong passphrase so that nobody can steal all the keys to your personal identity.
Tracking
Websites can combine information you have provided to them willingly with information they can gather about your use of the site, such as what you click on or how long you spend looking at a given page. They may also take note of your physical location based on your IP address. Together with your browsing history and other information stored on your computer, websites can form insights and make inferences about an individual by aggregating all of this information. Information that one site knows about you may also be shared with other websites. This might be used to generate ads for events that are similar to events you have attended, or promotions for items you have previously searched for online.
When websites collect data about your online activity, it is often used in ways that make your online experience smoother or more convenient. For example, a website might show you a list of the last five items you looked at, or allow you to resume watching a movie right where you left off. It can also autofill your shipping and billing information to simplify making online purchases.
Episode 2
Logging In
Authentication
Websites use a variety of ways to verify that the person on the other end of the connection is who they say they are. They also may use encryption to ensure that if anyone intercepts the information before it reaches its destination, they won’t be able to read it.
Multifactor Authentication
Many websites use multifactor authentication, which requires the person logging in to provide additional identifying information beyond just the password. This additional information might be in the form of clicking a link in a text message or an email sent to an address already on file, providing answers to security questions that only you would know, or even plugging a physical key directly into your machine. Multifactor authentication is important because each step provides an additional layer of protection that must be broken in order for someone unauthorized to access your account.
Privacy
When using a public wifi network, or even when browsing the web at work or at school, data moving across that network can be easily intercepted and read by anyone else on the same network. Encryption enables private, secure communication by encoding messages between a sender and receiver so that only they can view its content. The sender encrypts, or encodes, the information, and the receiver decrypts, or decodes it.
Most sites use Transport Layer Security to encrypt data, providing privacy from outside snoopers. When you go to a site in your browser, the site and browser perform a digital handshake to establish trust. The site provides a digital certificate of its identity, along with a public key. Your browser traces the ownership of the certificate back to a trusted certificate authority, which verifies that the site is who they say they are, and not a fraudulent one pretending to be legitimate. The handshake relies on your browser to encrypt data using the public key, while the site uses a separate private key to decrypt it.
Once the site’s identity is confirmed, the handshake finishes by generating a unique key that’s shared by both the browser and the site, a process known as symmetric key encryption. This new key is used to encrypt all the data in the session, ensuring that only the sender and receiver can read it.
You can tell if a website is using an encrypted connection by inspecting the url, which will begin with https:// (as opposed to the insecure http:// protocol.)
Security
Apple uses end-to-end encryption to protect iMessage and FaceTime conversations across all devices. With watchOS and iOS, messages are encrypted on the device so that they can’t be accessed without the user’s passcode. iMessage and FaceTime were designed so that there’s no way for Apple (or anyone else) to decrypt data when it’s in transit between devices.
The Internet
Data Transmission
When you upload or download a photo or stream a video, the app you’re using is communicating with the internet to get the data that constitutes it. And the more data you consume, the more important the speed of your connection is. Video contains lots of information—enough to fill a screen full of pixels many times per second and supply audio to the speakers.
The bandwidth of your internet connection measures the maximum amount of data you can receive in a fixed amount of time—in other words, how quickly you can receive data. It’s usually expressed in bits per second. If your bandwidth is high, you can view high quality, high resolution video with stereo sound. If it’s low, you’ll notice a degradation in quality. Perhaps the audio will be reduced to mono, or the video resolution will be lower. And it’s likely that the video compression algorithm will be tuned to produce greater compression at the expense of quality—which you’ll experience as fuzziness, blocky, banded, or grainy areas, strange flickering, and other compression artifacts. In the worst case, your app may pause the video to build a buffer—enough data to play for a few seconds to smooth over any hiccups in the connection.
The internet is made up of lots of smaller networks connected together. If you stream video over a cellular connection, your phone is connecting to a local cell network, which then connects to the internet, and eventually to the computer that’s streaming the video. The cellular network may have lower bandwidth than the rest of the internet, and the connection to your phone is especially vulnerable to reduced bandwidth if your cell signal is weak—usually one or two bars results in poor video. And of course, if you lose your cell signal completely, you won’t be connected to the internet at all, which means you’re out of luck if you’re not in range of a WiFi network.
On the other hand, if you connect via WiFi, you’re accessing the video stream via a WiFi router, which typically has higher bandwidth, and a more direct connection to the rest of the internet. That’s why streaming on WiFi is typically a more pleasant experience. Technologies such as LTE and 5G are efforts to improve cellular network bandwidth.
Protocols
A large factor in the success of the internet is open protocols. A protocol is an agreed-upon set of rules that allows two devices to perform a common function. The internet is built on protocols that handle the transfer of information from one computer to another. And because the protocols are open, meaning that anybody can write software to implement them, many different kinds of computers running many different kinds of software can still coordinate as a team to deliver a stream of video from a server, or a cell phone, or a wireless security camera, via connections as disparate as WiFi, cellular, cable, telephone and cable networks, and others, to devices ranging from phones to laptops to desktops, all running different apps and operating systems. All these many devices and systems speak the common language of basic internet protocols such as IP, TCP, UDP—and RTSP, which is one common protocol for streaming media.
Text and Binary
Every day, you work with text on your phone, your computer, and other digital devices. But the words you see on the screen are actually the result of multiple layers of abstraction, hiding from you the details of the device’s inner workings.
What’s going on underneath all those words?
Characters
All text is composed of individual units called characters. You learned about characters when you learned about Swift’s String type. A text field that holds the string "Hello" is storing the data for five characters: "H" + "e" + "l" + "l" + "o."
Characters in Swift are part of another abstraction, the Unicode standard, that defines how strings are represented. Unicode defines code points—unique numbers—for each of the characters that it can represent. Because it’s designed to encode all written languages, Unicode comprises more than 100,000 characters. Why so many, when the Roman alphabet contains only 52 characters? Consider that other alphabetic scripts, such as Hindi and Thai, have their own character sets. And languages like Chinese and Japanese have tens of thousands of unique characters! Unicode handles all of these, as well as punctuation marks, symbols, and even emoji.
But what if text contains only characters from the Roman alphabet? In this case, Unicode can use a simple representation based on the historic values of ASCII (American Standard Code for Information Interchange). A much older system for encoding text, ASCII uses numbers that range from 0 to 127 to represent 128 characters. That’s enough to cover all uppercase and lowercase letters, plus numbers, punctuation marks, and common symbols. The string "Hello" is represented in ASCII by the number sequence 72 101 108 108 111.
Numbers
All computer data boils down to numbers, and everything a computer does is an abstraction on top of simple math, like addition and multiplication. But the numbers that computers work with aren’t represented by the ten digits people use in the everyday world. Instead, computers rely on only two digits, 0 and 1, which enables them to perform calculations using electrical on/off switches. (Zero is off, and one is on.) The earliest digital computers used vacuum tubes and mechanical switches to represent on/off states, while the modern processor in an iPhone contains billions of microscopic electronic on/off switches, called transistors.
Bits and Bytes
The system of writing numbers with two digits is known as binary. In the binary system, the individual zeroes and ones are called bits. ASCII is a good example of binary—it represents each character using seven bits. When represented in binary, the ASCII code for the string "Hello" looks like this:
1001000 1100101 1101100 1101100 1101111.
To simplify data processing, computers group bits together into chunks. The smallest such chunk—a byte—has eight bits. (Other larger chunks are represented as multiple bytes. For example, words may be two or four bytes.)
Bits and bytes are grouped together to represent abstractions that people use every day, such as numbers, images, and letters.
For example, a byte can represent 256 different values—twice as many as the 128 needed for ASCII encoding. For historical reasons, ASCII only uses seven bits; to create a full byte, a zero is added to the beginning of each number. Here’s the sequence of bytes for the string "Hello":
01001000 01100101 01101100 01101100 01101111.
But binary doesn’t include the concept of spaces, so the word "Hello" would look more like this inside the computer:
0100100001100101011011000110110001101111.
Episode 3
Images and Compression
Taking a Photo
Analog and Digital
The world is alive with analog signals: information that changes smoothly over time. When you hear sounds, you’re sensing continuous waves of changing pressure in the air around you. Your sight works by sensing light waves cast onto your retinas by the lenses in your eyes. But computers work with digital information: huge amounts of individual numbers. In order for computers to process analog signals, they have to digitize them—which involves converting them into a set of discrete values.
Digital photography is a good example of analog-to-digital conversion. The camera sensor in a phone is a rectangle filled with tiny circuits that can measure the amount of light that hits them. Three circuits—one for each of the primary colors of red, green, and blue—are grouped together to make a pixel sensor. (“Pixel” is short for picture element.)
When you take a photo with your phone, the lens of the camera focuses light onto the camera sensor. As continuous signals—like the gradual darkening of a shadow from its center to its edge or the slight variations in color caused by the wrinkles in a shirt—fall across the array of pixel sensors, each pixel sensor converts the light into values for red, green, and blue. The camera sensor then generates a large set of numbers (digital data) describing the values from all the pixel sensors.
This process is called sampling. Computers convert analog signals to digital data by sampling them at discrete intervals to produce numbers. In this case, the intervals are the physical locations and dimensions of the pixel sensors, and the numbers represent colors and intensity of light. The resulting information is an abstraction—an approximate representation of the real-world image that passed through the lens of the camera.
Image Data
The digital information from a camera sensor might look something like this:
010000110110010101100011011010010010000001101110001001110110010101110011011101000010000001110000011000010111001100100000011101010110111001100101001000000111000001101001011100000110010100101110...
Like all digital data, it could represent anything, depending on the abstraction you choose for interpreting it. In ASCII, it’s the text: "Ceci n’est pas une pipe." But because it came from the camera sensor, your phone knows to interpret it as eight pixels:
| Binary | Red | Green | Blue | Color |
|---|
01000011 01100101 01100011 | 67 | 101 | 99 |  |
01101001 00100000 01101110 | 105 | 32 | 110 |  |
00100111 01100101 01110011 | 39 | 101 | 115 |  |
01110100 00100000 01110000 | 116 | 32 | 112 |  |
01100001 01110011 00100000 | 97 | 115 | 32 |  |
01110101 01101110 01100101 | 117 | 110 | 101 |  |
00100000 01110000 01101001 | 32 | 112 | 105 |  |
01110000 01100101 00101110 | 112 | 101 | 46 |  |
Every digital image in your phone is just a giant collection of pixels, each containing information about its levels of red, green, and blue light. Phone cameras can produce images with millions of pixels, and there are many data formats for those images. The example above is a 24-bit color image, using three bytes per pixel (one byte for each color). An image in this format would contain 12 million pixels and be 36 megabytes in size.
Compression
Many of the images you see on your phone are compressed. Compressing an image doesn’t reduce the number of pixels (its resolution), it just reduces the data needed to store the image. Compressed images take less time to download and less space to store.
How is compression possible?
Compression strategies can be grouped into two broad categories: lossless and lossy. Lossless compression preserves all of the original data, so that it can be reconstructed at any time. Lossy compression creates an approximation of the original information, resulting in lower quality.
Run-length encoding is a simple example of lossless compression. If an image contains a lot of redundant information—like a screenshot with a white background and black text that has lots of pixels with identical color—run-length encoding is a good compression strategy. Instead of describing “one white pixel, one white pixel, one white pixel, one white pixel, one white pixel,” the data could be encoded as “a run of five white pixels.” In the uncompressed 24-bit color version, data for these five pixels would contain 15 bytes (three bytes per pixel); whereas the compressed version would contain only three bytes (because all five pixels have the same color), plus some number of bytes to encode the length of the run. The resulting size could be as small as four bytes.
In binary, the data would be compressed from this:
111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
to this:
00000101111111111111111111111111
There are lossless compression formats that work with photos, which usually contain more varied information. Professional photographers often use TIFF (tagged image file format) image files because they want to edit them with maximum fidelity.
But most people aren’t concerned with preserving the exact data from a photo. Lossy image compression—used for the formats you’re probably familiar with, like JPEG and PNG—results in a lower-quality version of the original, but usually takes up less space than a lossless compression. And depending on the compression settings, it may be hard to tell the difference. Lossy image compression typically involves complex math. If you zoom into an image to examine it pixel by pixel, you’ll be able to see the effects of compression. But when you view it on your screen, even a computer desktop, a lossily compressed image can look identical to the uncompressed version.
Posting to Social Media
When you post a photo to a social media app, you expect it to appear in your followers’ feeds. You’d also expect that other users could search your username and see that same post in your profile or timeline. And if you use hashtags, you’d want your posts to show up in those feeds as well.
Of course, the app on your phone isn’t sending that post directly to other people, one by one. To enable social media features, the social media platform runs a service from its data center. This network of computers, called servers, is constantly receiving new data, processing it, and responding to requests from its app—which is running on phones and computers all over the world.
Every time you post or see new updates in your social media feed, you’re interacting with the social media service.
Metadata
Your social media post starts with the photo and the text associated with it. But a social media service requires more specific data to perform the functions you expect of it. This information is called metadata—data about data. Metadata is essential for organizing and managing information. For example, to organize all the posts in your timeline in chronological order, the server has to know what day and time the post was created. And to add your posts to your followers’ feeds, it has to be able to distinguish them from the posts of other users.
The social media service could store all the posts from each user, in chronological order, in a single file, and then search through the file to build the user’s feed or timeline. But that approach is highly inefficient, especially if the service has thousands or millions of users. Instead, the server stores everybody’s posts in one database—a giant repository of information—and then queries the database for posts by specifying criteria, such as the user that authored them.
That’s where metadata comes in. Because the database doesn’t store information in any particular order, it relies on metadata about each post to provide results for queries. A social media post might contain the following metadata:
- ID: A unique identifier for the post itself.
- user ID: The handle of the user who created the post.
- time: The time and date of the post.
- hashtags: The list of hashtags in the post.
- location: The location of the device when the post was created.
- likes: A count of how many times the post was liked.
- replied to: If this post is a reply to another one, the ID of the original post.
The social media service can perform all its tasks by querying the database for certain values of metadata. Changing the post’s metadata, such as the number of Likes, doesn’t change the data itself.
Filtering Data
Building a user’s timeline is simple. The service queries the database to find all the posts whose user ID metadata matches that user’s ID. When the results come back, the service sorts them using the time metadata. Similarly, to display all posts with a particular hashtag, the service queries by hashtag.
Showing replies to a post works the same way. The service gets the ID of the post in question and queries the database for all posts whose replied to metadata contains that ID.
A thread—an original post and its chain of replies—can be built using a series of queries. To find the second post in a thread, the service queries for posts where the replied to metadata has the first post’s ID and where the user ID metadata has the same user ID as the first post. The service repeats the process, updating the replied to part of the query as it finds each new post. When its query for a reply fails to return a result, the service has reached the last post in the thread.
Building Feeds
When you post to social media, the post’s data and metadata are uploaded to the service’s database. This information enables the server to include your post whenever your followers request updates to their feeds.
Here’s how it works. The server performs two steps:
- It queries the database for new posts from each of the followed users—requesting only those posts whose date and time is more recent than the last time the feed was updated.
- It takes the resulting list of new posts from all followed users and sorts it by date and time—or uses an internal algorithm to sort and filter them in some other way.
As your followers refresh their social media app, your post will show up in their feeds, because its metadata causes it to be returned from these database queries.
Parallel and Distributed Computing
In the early days of computing, computers could only do one thing at a time—a model of computation known as sequential computing or serial processing. Building a social media feed using serial processing would require a server to process each followed user, one after the other.
Fortunately, modern computers are capable of parallel processing: They can perform several operations simultaneously, provided the operations are independent of each other. If a computer can execute four queries in parallel, it could potentially complete the job four times as fast.
Social media services rely heavily on parallel processing to perform large numbers of independent tasks very quickly. For example, the query for one followed user doesn’t depend on the queries for other users, so all followed users can be queried simultaneously. The server can also sort the individual user lists by date independently. In fact, the final step—merging those separate lists into one feed—is the only step that can’t be performed in parallel because it’s dependent on all the previous steps.
With parallel processing, a server can process feeds efficiently enough to handle a steady stream of requests, even with hundreds (or thousands) of followed users. But a single server has its limits—and as the number of users of a social media app grows, the demands may eventually overwhelm the server’s capabilities. For example, a server might handle 100 requests per second easily. But if it receives 500 or 1,000 requests per second, it will begin to slow down, causing users to experience delays as they refresh their feeds.
The solution is to rely on distributed computing. Because parallel processes are always independent, they don’t have to run on the same computer. The data center for a social media platform typically houses hundreds or thousands of servers. Each server might store a database of posts from a fraction of the platform’s users. When one server starts building a feed for a user, it can farm out the work of querying and sorting to the servers that manage the followed users. Those servers then relay the lists back to the first server, which merges them and sends the new feed to the requesting user’s app.
A distributed computing model makes the service scalable. That means that, as the social media app grows in popularity, the data center can simply add servers to accommodate the demand.
