APIs

Learning goals

After this lesson, you should be able to:

• Explain what an API is
• Set up an API key for a public API
• Develop comfort in using a wrapper package or URL-method of calling a web API
• Recognize the structure in a URL for a web API and adjust for your purposes
• Explore and subset complex nested lists

You can download a template Quarto file to start from here. Put this file in a folder called data_acquisition within a folder for this course.

APIs

When we interact with sites like The New York Times, Zillow, and Google, we are accessing their data via a graphical layout (e.g., images, colors, columns) that is easy for humans to read but hard for computers.

An API stands for Application Programming Interface, and this term describes a general class of tool that allows computers, rather than humans, to interact with an organization’s data. How does this work?

• When we use web browsers to navigate the web, our browsers communicate with web servers using a technology called HTTP or Hypertext Transfer Protocol to get information that is formatted into the display of a web page.
• Programming languages such as R can also use HTTP to communicate with web servers. The easiest way to do this is via Web APIs, or Web Application Programming Interfaces, which focus on transmitting raw data, rather than images, colors, or other appearance-related information that humans interact with when viewing a web page.

A large variety of web APIs provide data accessible to programs written in R (and almost any other programming language!). Almost all reasonably large commercial websites offer APIs. Todd Motto has compiled an expansive list of Public Web APIs on GitHub. Let’s browse this list to see what data sources are available.

Wrapper packages

In R, it is easiest to use Web APIs through a wrapper package, an R package written specifically for a particular Web API.

• The R development community has already contributed wrapper packages for most large Web APIs.
• To find a wrapper package, search the web for “R package” and the name of the website. For example:
  • Searching for “R Reddit package” returns RedditExtractor
  • Searching for “R Weather.com package” returns weatherData
• rOpenSci also has a good collection of wrapper packages.

In our work with maps, we’ve used the tidycensus package to obtain census data to display on maps. tidycensus is a wrapper package that makes it easy to obtain desired census information:

tidycensus::get_acs(
    year = 2020,
    state = "MN",
    geography = "tract",
    variables = c("B01003_001", "B19013_001"),
    output = "wide",
    geometry = TRUE
)

Extra resources:

• tidycensus: wrapper package that provides an interface to a few census datasets with map geometry included
  • Full documentation is available at https://walker-data.com/tidycensus/
• censusapi: wrapper package that offers an interface to all census datasets
  • Full documentation is available at https://www.hrecht.com/censusapi/

What is going on behind the scenes with get_acs()?

Accessing web APIs directly

Getting a Census API key

Many APIs require users to obtain a key to use their services.

• This lets organizations keep track of what data is being used.
• It also rate limits their API and ensures programs don’t make too many requests per day/minute/hour. Be aware that most APIs do have rate limits — especially for their free tiers.

Navigate to https://api.census.gov/data/key_signup.html to obtain a Census API key:

• Organization: Macalester College
• Email: Your Mac email address

You will get the message:

Your request for a new API key has been successfully submitted. Please check your email. In a few minutes you should receive a message with instructions on how to activate your new key.

Check your email. Copy and paste your key into a new text file:

• File > New File > Text File (towards the bottom of the menu)
• Save as census_api_key.txt in the same folder as this .qmd.

Read in the key with the following code:

census_api_key <- readLines("census_api_key.txt")

census_api_key <- readLines("../census_api_key.txt")

Warning in readLines("../census_api_key.txt"): incomplete final line found on
'../census_api_key.txt'

Navigating API documentation

Navigate to the Census API user guide and click on the “Example API Queries” tab.

Let’s look at the Population Estimates Example and the American Community Survey (ACS) Example. These examples walk us through the steps to incrementally build up a URL to obtain desired data. This URL is known as a web API request.

https://api.census.gov/data/2019/acs/acs1?get=NAME,B02015_009E,B02015_009M&for=state:*

• https://api.census.gov: This is the base URL.
  • http://: The scheme, which tells your browser or program how to communicate with the web server. This will typically be either http: or https:.
  • api.census.gov: The hostname, which is a name that identifies the web server that will process the request.
• data/2019/acs/acs1: The path, which tells the web server how to get to the desired resource.
  • In the case of the Census API, this locates a desired dataset in a particular year.
  • Other APIs allow search functionality. (e.g., News organizations have article searches.) In these cases, the path locates the search function we would like to call.
• ?get=NAME,B02015_009E,B02015_009M&for=state:*: The query parameters, which provide the parameters for the function you would like to call.
  • We can view this as a string of key-value pairs separated by &. That is, the general structure of this part is key1=value1&key2=value2.

key	value
get	NAME,B02015_009E,B02015_009M
for	state:*

Typically, each of these URL components will be specified in the API documentation. Sometimes, the scheme, hostname, and path (https://api.census.gov/data/2019/acs/acs1) will be referred to as the endpoint for the API call.

We will use the httr2 package to build up a full URL from its parts.

• request() creates an API request object using the base URL
• req_url_path_append() builds up the URL by adding path components separated by /
• req_url_query() adds the ? separating the endpoint from the query and sets the key-value pairs in the query
  • The .multi argument controls how multiple values for a given key are combined.
  • The I() function around "state:*" inhibits parsing of special characters like : and *. (It’s known as the “as-is” function.)
  • The backticks around for are needed because for is a reserved word in R (for for-loops). You’ll need backticks whenever the key name has special characters (like spaces, dashes).
  • We can see from here that providing an API key is achieved with key=YOUR_API_KEY.

req <- request("https://api.census.gov") %>% 
    req_url_path_append("data") %>% 
    req_url_path_append("2019") %>% 
    req_url_path_append("acs") %>% 
    req_url_path_append("acs1") %>% 
    req_url_query(get = c("NAME", "B02015_009E", "B02015_009M"), `for` = I("state:*"), key = census_api_key, .multi = "comma")

Why would we ever use these steps instead of just using the full URL as a string?

• To generalize this code with functions! (This is exactly what wrapper packages do.)
• To handle special characters
  • e.g., query parameters might have spaces, which need to be represented in a particular way in a URL (URLs can’t contain spaces)

Once we’ve fully constructed our request, we can use req_perform() to send out the API request and get a response.

resp <- req_perform(req)
resp

<httr2_response>

GET
https://api.census.gov/data/2019/acs/acs1?get=NAME,B02015_009E,B02015_009M&for=state:*&key=31d49a72c27f08d632defd32f7a80918a4187b90

Status: 200 OK

Content-Type: application/json

Body: In memory (1691 bytes)

We see from Content-Type that the format of the response is something called JSON. We can navigate to the request URL to see the structure of this output.

• JSON (Javascript Object Notation) is a nested structure of key-value pairs.
• We can use resp_body_json() to parse the JSON into a nicer format.
  • Without simplifyVector = TRUE, the JSON is read in as a list.

resp_json_list <- resp %>% resp_body_json()
head(resp_json_list, 2)

[[1]]
[[1]][[1]]
[1] "NAME"

[[1]][[2]]
[1] "B02015_009E"

[[1]][[3]]
[1] "B02015_009M"

[[1]][[4]]
[1] "state"


[[2]]
[[2]][[1]]
[1] "Illinois"

[[2]][[2]]
[1] "655"

[[2]][[3]]
[1] "511"

[[2]][[4]]
[1] "17"

resp_json_df <- resp %>% resp_body_json(simplifyVector = TRUE)
head(resp_json_df)

     [,1]       [,2]          [,3]          [,4]   
[1,] "NAME"     "B02015_009E" "B02015_009M" "state"
[2,] "Illinois" "655"         "511"         "17"   
[3,] "Georgia"  "3162"        "1336"        "13"   
[4,] "Idaho"    NA            NA            "16"   
[5,] "Hawaii"   "56"          "92"          "15"   
[6,] "Indiana"  "1344"        "1198"        "18"   

resp_json_df <- janitor::row_to_names(resp_json_df, 1)
head(resp_json_df)

     NAME       B02015_009E B02015_009M state
[1,] "Illinois" "655"       "511"       "17" 
[2,] "Georgia"  "3162"      "1336"      "13" 
[3,] "Idaho"    NA          NA          "16" 
[4,] "Hawaii"   "56"        "92"        "15" 
[5,] "Indiana"  "1344"      "1198"      "18" 
[6,] "Iowa"     "685"       "705"       "19" 

To learn more about JSON, consult the following readings:

1. A Non-Programmer’s Introduction to JSON
2. Getting Started With JSON and jsonlite
3. Fetching JSON data from REST APIs

Exercises: (Homework 7)

1. Write a function called get_acs1() that gets 1-year ACS data for a given year for a set of user specified variables for all states. Function arguments should include:

   • year: Within your function check that this is between 2005 and 2022 and not 2020 (data only available from 2005-2022 except for 2020). If the year is outside this range, include stop("Helpful error message here") to display a helpful error message to the user.
   • variables: A character vector of variable names to obtain. e.g., c("B02015_009", "B01003_001"). Note that these don’t have “E” or “M” at the end. You’ll always return the “E” version (for the estimate). See next argument for the “M” version.
   • include_moe: TRUE/FALSE indicator of whether to additionally include the “M” version (for the margin of error on the estimate). Set the default to TRUE.
   • key: Census API key Your function should include a Sys.sleep(0.5) at the beginning to pause code execution for 0.5 seconds. While the Census API does not seem to have a rate limit when you provide an API key, include this pause just to be safe.

2. Come up with a set of test cases that will fully test your function. Update your function as needed as you run your tests. Show output from successful tests on all test cases.

3. Obtain 1-year ACS data for all states from 2005-2022 (except 2020) for the following two variables: B01003_001 (total population) and B06011_001 (Median Income in the Past 12 Months in 2022 Inflation-Adjusted Dollars).

   • Get both the estimate and the margin of error for these two variables.
   • Put all of this information into a single data frame.
   • Do this with both a for loop and with an appropriate purrr function. (See our iteration day 2 activity for reference.)
   • Verify using the identical() function that you get identical data frames from both approaches.

4. Create visualizations that show how the two variables have changed over 2005-2022 (except 2020).

   • Show both the estimate and margin of error in an effective way.
   • Make sure that your plots have clean labels and a title.

Board Game Geek

The Board Game Geek API is referenced in the Games & Comics section of toddmotto’s public API list.

Our goal is to use the search API at the bottom of the page. Let’s start at the top of the API documentation page to see how to navigate this reference.

• We can see from the XML references at the top that we will be expecting a new output format: XML stands for Extensible Markup Language
• The “Root Path” section tells us the base URL for the Board Game Geeks API endpoints and related APIs: https://boardgamegeek.com/xmlapi2/
• The “Search” section at the bottom of the page tells us:
  • the path for the search endpoint (/search)
  • what query parameters are possible
  • particular formatting instructions for query parameter values

The following request searches for board games, board game accessories, and board game expansions with the words “mystery” and “curse” in the title:

req <- request("https://boardgamegeek.com/xmlapi2") %>% 
    req_url_path_append("search") %>% 
    req_url_query(query = I("mystery+curse"), type = I("boardgame,boardgameaccessory,boardgameexpansion"))

When we use req_perform(), we see from Content-Type that the format of the response is something called XML. We can navigate to the request URL to see the structure of this output.

• XML (Extensible Markup Language) is a tree structure of named nodes and attributes.
• We can use resp_body_xml() to read in the XML as an R object.

resp <- req_perform(req)
resp

<httr2_response>

GET
https://boardgamegeek.com/xmlapi2/search?query=mystery+curse&type=boardgame,boardgameaccessory,boardgameexpansion

Status: 200 OK

Content-Type: text/xml

Body: In memory (1903 bytes)

resp <- resp_body_xml(resp)
resp

{xml_document}
<items total="9" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse">
[1] <item type="boardgame" id="63495">\n  <name type="primary" value="Murder  ...
[2] <item type="boardgame" id="40175">\n  <name type="primary" value="Murder  ...
[3] <item type="boardgame" id="256211">\n  <name type="primary" value="Robins ...
[4] <item type="boardgame" id="238391">\n  <name type="primary" value="Robins ...
[5] <item type="boardgameaccessory" id="289457">\n  <name type="primary" valu ...
[6] <item type="boardgameaccessory" id="291036">\n  <name type="primary" valu ...
[7] <item type="boardgameaccessory" id="309435">\n  <name type="alternate" va ...
[8] <item type="boardgameexpansion" id="256211">\n  <name type="primary" valu ...
[9] <item type="boardgameexpansion" id="238391">\n  <name type="primary" valu ...

The XML output is not packaged in a nice way. (We’d love to have a data frame.) We can use the xml2 package to explore and navigate the XML structure to extract the information we need.

Let’s first use the xml_structure() function to see how information is organized:

xml_structure(resp)

<items [total, termsofuse]>
  <item [type, id]>
    <name [type, value]>
    <yearpublished [value]>
  <item [type, id]>
    <name [type, value]>
    <yearpublished [value]>
  <item [type, id]>
    <name [type, value]>
    <yearpublished [value]>
  <item [type, id]>
    <name [type, value]>
    <yearpublished [value]>
  <item [type, id]>
    <name [type, value]>
    <yearpublished [value]>
  <item [type, id]>
    <name [type, value]>
  <item [type, id]>
    <name [type, value]>
    <yearpublished [value]>
  <item [type, id]>
    <name [type, value]>
    <yearpublished [value]>
  <item [type, id]>
    <name [type, value]>
    <yearpublished [value]>

The key navigation and extraction functions in xml2 are:

• xml_children(): Get nodes that are nested inside
  • Like getting the first level bullet points inside a given bullet point
• xml_find_all(): Finds nodes matching an XPath expression (XPath stands for XML path)
  • XPath expressions are like string regular expressions for XML trees
  • See here for a deeper dive into XPath
• xml_attr(): Selects the value of an attribute (the information to the right of the = in quotes)
  • <node_name attribute_name1="attribute_value1" attribute_name2="attribute_value2">

# Get the item nodes in 2 different ways
resp %>% xml_find_all("item")

{xml_nodeset (9)}
[1] <item type="boardgame" id="63495">\n  <name type="primary" value="Murder  ...
[2] <item type="boardgame" id="40175">\n  <name type="primary" value="Murder  ...
[3] <item type="boardgame" id="256211">\n  <name type="primary" value="Robins ...
[4] <item type="boardgame" id="238391">\n  <name type="primary" value="Robins ...
[5] <item type="boardgameaccessory" id="289457">\n  <name type="primary" valu ...
[6] <item type="boardgameaccessory" id="291036">\n  <name type="primary" valu ...
[7] <item type="boardgameaccessory" id="309435">\n  <name type="alternate" va ...
[8] <item type="boardgameexpansion" id="256211">\n  <name type="primary" valu ...
[9] <item type="boardgameexpansion" id="238391">\n  <name type="primary" valu ...

resp %>% xml_children()

{xml_nodeset (9)}
[1] <item type="boardgame" id="63495">\n  <name type="primary" value="Murder  ...
[2] <item type="boardgame" id="40175">\n  <name type="primary" value="Murder  ...
[3] <item type="boardgame" id="256211">\n  <name type="primary" value="Robins ...
[4] <item type="boardgame" id="238391">\n  <name type="primary" value="Robins ...
[5] <item type="boardgameaccessory" id="289457">\n  <name type="primary" valu ...
[6] <item type="boardgameaccessory" id="291036">\n  <name type="primary" valu ...
[7] <item type="boardgameaccessory" id="309435">\n  <name type="alternate" va ...
[8] <item type="boardgameexpansion" id="256211">\n  <name type="primary" valu ...
[9] <item type="boardgameexpansion" id="238391">\n  <name type="primary" valu ...

# Get the item "type"
resp %>% xml_find_all("item") %>% xml_attr("type")

[1] "boardgame"          "boardgame"          "boardgame"         
[4] "boardgame"          "boardgameaccessory" "boardgameaccessory"
[7] "boardgameaccessory" "boardgameexpansion" "boardgameexpansion"

# The <name> and <yearpublished> nodes are nested within each <item>
resp %>% xml_find_all("item/name")

{xml_nodeset (9)}
[1] <name type="primary" value="Murder Mystery Dinner Party: The Curse of the ...
[2] <name type="primary" value="Murder Mystery Evening: The Curse of the Mumm ...
[3] <name type="primary" value="Robinson Crusoe: Adventures on the Cursed Isl ...
[4] <name type="primary" value="Robinson Crusoe: Adventures on the Cursed Isl ...
[5] <name type="primary" value="Robinson Crusoe: Adventures on the Cursed Isl ...
[6] <name type="primary" value="Robinson Crusoe: Adventures on the Cursed Isl ...
[7] <name type="alternate" value="Robinson Crusoe: Adventures on the Cursed I ...
[8] <name type="primary" value="Robinson Crusoe: Adventures on the Cursed Isl ...
[9] <name type="primary" value="Robinson Crusoe: Adventures on the Cursed Isl ...

resp %>% xml_find_all("item/yearpublished") # Notice that this is length 8 instead of 9!

{xml_nodeset (8)}
[1] <yearpublished value="2008"/>
[2] <yearpublished value="2007"/>
[3] <yearpublished value="2018"/>
[4] <yearpublished value="2019"/>
[5] <yearpublished value="2019"/>
[6] <yearpublished value="2019"/>
[7] <yearpublished value="2018"/>
[8] <yearpublished value="2019"/>

# Get the "primary" or "alternate" designation for each name
resp %>% xml_find_all("item/name") %>% xml_attr("type")

[1] "primary"   "primary"   "primary"   "primary"   "primary"   "primary"  
[7] "alternate" "primary"   "primary"  

Exercises: (not part of Homework 7)

1. Get the board game name (e.g., “Murder Mystery…” and “Robinson Crusoe”)
2. Get the board game ID number (e.g., 63495, 40175)

New York Times

This example will build on the New York Times Web API, which provides access to news articles, movie reviews, book reviews, and many other data. We will specifically focus on the Article Search API, which finds information about news articles that contain a particular word or phrase.

Wrapper package

We will first use the nytimes wrapper package that provides functions for some (but not all) of the NYTimes APIs. You can install the package by running the following in the Console:

install.packages("devtools")
devtools::install_github("mkearney/nytimes")

To get started with the NY Times API, you must register and get an authentication key. Signup only takes a few seconds, and it lets the New York Times make sure nobody abuses their API for commercial purposes. It also rate limits their API and ensures programs don’t make too many requests per day. For the NY Times API, this limit is 1000 calls per day. Be aware that most APIs do have rate limits — especially for their free tiers.

Once you have signed up, verified your email, log back in to https://developer.nytimes.com. Under your email address, click on Apps and Create a new App (call it First API) and enable Article Search API, then press Save. This creates an authentication key, which is a 32 digit string with numbers and the letters a-e.

As with your census API key, save this key in a .txt file, and read it in . tore this in a variable called times_key.

Now, let’s use the key to issue our first API call. We’ll adapt the code we see in the nytimes vignette to do what we need.

library(nytimes)

# Tell nytimes what our API key is
Sys.setenv(NYTIMES_KEY = times_key)

# Issue our first API call
res <- nyt_search(q = "gamergate", n = 20, end_date = "20150101")

# Convert response object to data frame
res <- as.data.frame(res)

This code sent a message to the New York Times and asked for information about 20 articles about Gamergate starting at January 1, 2015 and going backwards in time.

Let’s take a peek at the structure of the results:

colnames(res)

 [1] "id"               "abstract"         "byline"           "document_type"   
 [5] "headline"         "keywords"         "lead_paragraph"   "multimedia"      
 [9] "news_desk"        "print_page"       "print_section"    "pub_date"        
[13] "section_name"     "snippet"          "source"           "subsection_name" 
[17] "type_of_material" "uri"              "web_url"          "word_count"      

head(res)

                                                      id
1     nyt://article/08765e5b-8d12-54dd-be58-39c6d33125c1
2     nyt://article/5e97a537-4e5d-51b4-9571-736913a6e5c4
3     nyt://article/ebad4be5-8e52-5490-a3bb-f5d8f684b902
4 nyt://interactive/26986d5d-2854-5484-86b8-04dbbfea0b27
5     nyt://article/fe201e9b-ea3b-5e9c-bb5a-ef4b0bbd30c4
6     nyt://article/7574b532-e51f-5695-80aa-a5b24ee4e7a2
                                                                                                                                                                     abstract
1                                                           A service lets a person monitor his or her Facebook or Twitter account, for more awareness of one’s online image.
2 Get recommendations from New York Times reporters and editors, highlighting great stories from around the web. Today, great reads from Dean Baquet, Susan Chira and others.
3                                                                                                                                The answers for our ninth annual Op-Ed quiz.
4                                                                                                   Highlights from the year, as chosen by the editors of The New York Times.
5                                                                                             The women’s-magazine editor on the Internet, feminism and reading the comments.
6                                                               Social media companies are often reluctant to become arbiters of appropriate and inappropriate speech online.
                      byline document_type
1           By Farhad Manjoo       article
2      By The New York Times       article
3              By Ben Schott       article
4                       <NA>    multimedia
5 Interview by Susan Dominus       article
6           By Jenna Wortham       article
                                                            headline
1          ThinkUp Helps the Social Network User See the Online Self
2 What We’re Reading: Great Times Reads of 2014 From Our Top Editors
3                         2014: The Year in Questions – Quiz Answers
4                                                   The Best of 2014
5           Jane Pratt on Why Writing for Young Women Never Gets Old
6                  Trying to Swim in a Sea of Social Media Invective
                                                                                                                                                                keywords
1                                                                                                 Social Media+ThinkUp LLC+Facebook Inc+Twitter+Dash, Anil+Trapani, Gina
2 News and News Media+Newspapers+Baquet, Dean+Barry, Ellen+Chira, Susan+Duenes, Steve+Fisher, Ian+Lacey, Marc+Slackman, Michael+New York Times+AFRICA+Delhi (India)+Iraq
3                                                                                                                                                                   <NA>
4                                                                                                                                                                   <NA>
5                                                                                                                       Pratt, Jane+xoJane.com+Women and Girls+Magazines
6                                                                               Computers and the Internet+Cyberharassment+Social Media+Facebook Inc+Twitter+Yik Yak Inc
                                                                                                                                                                                                                                                                                                                                                                                                                       lead_paragraph
1 Anil Dash, a longtime tech entrepreneur and blogger, was recently studying a list of the top words he had used on Twitter over the course of a month during the fall. Mr. Dash has half a million followers on Twitter, and like a lot of people in tech and media circles, he uses the social network to chat with colleagues, to pontificate about technology, politics and pop culture, and to participate in a lot of in-jokes.
2                                                                                                                                                                                                                                                           Get recommendations from New York Times reporters and editors, highlighting great stories from around the web. What We’re Reading emails are sent twice a week. Sign up »
3                                                                                                                                                                                                                                                                                                                                                                                                                         1. B – Uber
4                                                                                                                                                                                                                                                                                                                                                           Highlights from the year, as chosen by the editors of The New York Times.
5                                                                                                                                                                                                                                                                                                                                             The editor talks with Susan Dominus about navigating the pitfalls of online publishing.
6                                                                                                                                     Over the last few months, I’ve watched friends and colleagues endure endless harassment on Twitter. Strangers have hurled offensive, racist names and gendered insults, relentlessly and with little fear of consequence. I’ve come across blog posts that capture similarly awful experiences.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       multimedia
1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          images/2015/01/01/technology/personaltech/01state-illo/01state-illo-thumbWide.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-videoThumb.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-mediumThreeByTwo210.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-mediumThreeByTwo225.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-mediumThreeByTwo440.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-facebookJumbo.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-miniMoth.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-articleLarge.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-blog480.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-blog427.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-tmagArticle.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-jumbo.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-blog225.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-master180.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-popup.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-blog533.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-tmagSF.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-slide.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-superJumbo.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-master495.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-master315.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-square320.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-filmstrip.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-moth.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-mediumSquare149.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-articleInline.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-hpSmall.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-blogSmallInline.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-mediumFlexible177.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-sfSpan.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-largeHorizontal375.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-hpLarge.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-largeWidescreen573.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-videoSmall.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-videoHpMedium.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-videoSixteenByNine600.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-videoSixteenByNine540.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-videoSixteenByNine495.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-videoSixteenByNine390.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-videoSixteenByNine480.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-videoSixteenByNine310.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-videoSixteenByNine225.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-videoSixteenByNine96.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-videoSixteenByNine150.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-thumbStandard.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-thumbLarge.jpg+images/2015/01/01/technology/personaltech/01state-illo/01state-illo-blogSmallThumb.jpg
2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          images/2014/12/31/business/wwrn-ebola/wwrn-ebola-thumbWide.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoThumb.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoLarge.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-mediumThreeByTwo210.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-mediumThreeByTwo225.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-mediumThreeByTwo440.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-facebookJumbo.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoFifteenBySeven1305.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoFifteenBySeven2610.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-miniMoth.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-windowsTile336H.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-articleLarge.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-blog480.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-blog427.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-tmagArticle.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-jumbo.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-blog225.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-master675.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-master180.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-popup.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-blog533.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-tmagSF.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-slide.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-superJumbo.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-master1050.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-master495.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-master315.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-square320.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-filmstrip.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-square640.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-moth.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-mediumSquare149.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-articleInline.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-hpSmall.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-blogSmallInline.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-mediumFlexible177.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-sfSpan.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-largeHorizontal375.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-hpLarge.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-largeWidescreen573.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-largeWidescreen1050.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoSmall.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoHpMedium.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoSixteenByNine600.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoSixteenByNine540.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoSixteenByNine495.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoSixteenByNine390.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoSixteenByNine480.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoSixteenByNine310.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoSixteenByNine225.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoSixteenByNine96.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoSixteenByNine768.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoSixteenByNine150.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-videoSixteenByNine1050.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-thumbStandard.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-thumbLarge.jpg+images/2014/12/31/business/wwrn-ebola/wwrn-ebola-blogSmallThumb.jpg
3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               images/2014/12/30/opinion/30schott/30schott-thumbWide.jpg+images/2014/12/30/opinion/30schott/30schott-videoThumb.jpg+images/2014/12/30/opinion/30schott/30schott-videoLarge.jpg+images/2014/12/30/opinion/30schott/30schott-mediumThreeByTwo210.jpg+images/2014/12/30/opinion/30schott/30schott-mediumThreeByTwo225.jpg+images/2014/12/30/opinion/30schott/30schott-mediumThreeByTwo440.jpg+images/2014/12/30/opinion/30schott/30schott-facebookJumbo.jpg+images/2014/12/30/opinion/30schott/30schott-videoFifteenBySeven1305.jpg+images/2014/12/30/opinion/30schott/30schott-miniMoth.jpg+images/2014/12/30/opinion/30schott/30schott-windowsTile336H.jpg+images/2014/12/30/opinion/30schott/30schott-articleLarge.jpg+images/2014/12/30/opinion/30schott/30schott-blog480.jpg+images/2014/12/30/opinion/30schott/30schott-blog427.jpg+images/2014/12/30/opinion/30schott/30schott-tmagArticle.jpg+images/2014/12/30/opinion/30schott/30schott-jumbo.jpg+images/2014/12/30/opinion/30schott/30schott-blog225.jpg+images/2014/12/30/opinion/30schott/30schott-master675.jpg+images/2014/12/30/opinion/30schott/30schott-master180.jpg+images/2014/12/30/opinion/30schott/30schott-popup.jpg+images/2014/12/30/opinion/30schott/30schott-blog533.jpg+images/2014/12/30/opinion/30schott/30schott-tmagSF.jpg+images/2014/12/30/opinion/30schott/30schott-slide.jpg+images/2014/12/30/opinion/30schott/30schott-superJumbo.jpg+images/2014/12/30/opinion/30schott/30schott-master1050.jpg+images/2014/12/30/opinion/30schott/30schott-master495.jpg+images/2014/12/30/opinion/30schott/30schott-master315.jpg+images/2014/12/30/opinion/30schott/30schott-square320.jpg+images/2014/12/30/opinion/30schott/30schott-filmstrip.jpg+images/2014/12/30/opinion/30schott/30schott-square640.jpg+images/2014/12/30/opinion/30schott/30schott-moth.jpg+images/2014/12/30/opinion/30schott/30schott-mediumSquare149.jpg+images/2014/12/30/opinion/30schott/30schott-articleInline.jpg+images/2014/12/30/opinion/30schott/30schott-hpSmall.jpg+images/2014/12/30/opinion/30schott/30schott-blogSmallInline.jpg+images/2014/12/30/opinion/30schott/30schott-mediumFlexible177.jpg+images/2014/12/30/opinion/30schott/30schott-sfSpan.jpg+images/2014/12/30/opinion/30schott/30schott-largeHorizontal375.jpg+images/2014/12/30/opinion/30schott/30schott-hpLarge.jpg+images/2014/12/30/opinion/30schott/30schott-largeWidescreen573.jpg+images/2014/12/30/opinion/30schott/30schott-largeWidescreen1050.jpg+images/2014/12/30/opinion/30schott/30schott-videoSmall.jpg+images/2014/12/30/opinion/30schott/30schott-videoHpMedium.jpg+images/2014/12/30/opinion/30schott/30schott-videoSixteenByNine600.jpg+images/2014/12/30/opinion/30schott/30schott-videoSixteenByNine540.jpg+images/2014/12/30/opinion/30schott/30schott-videoSixteenByNine495.jpg+images/2014/12/30/opinion/30schott/30schott-videoSixteenByNine390.jpg+images/2014/12/30/opinion/30schott/30schott-videoSixteenByNine480.jpg+images/2014/12/30/opinion/30schott/30schott-videoSixteenByNine310.jpg+images/2014/12/30/opinion/30schott/30schott-videoSixteenByNine225.jpg+images/2014/12/30/opinion/30schott/30schott-videoSixteenByNine96.jpg+images/2014/12/30/opinion/30schott/30schott-videoSixteenByNine768.jpg+images/2014/12/30/opinion/30schott/30schott-videoSixteenByNine150.jpg+images/2014/12/30/opinion/30schott/30schott-videoSixteenByNine1050.jpg+images/2014/12/30/opinion/30schott/30schott-thumbStandard.jpg+images/2014/12/30/opinion/30schott/30schott-thumbLarge.jpg+images/2014/12/30/opinion/30schott/30schott-blogSmallThumb.jpg
4 images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-thumbWide.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-videoThumb.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-videoLarge.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-mediumThreeByTwo210.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-mediumThreeByTwo225.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-mediumThreeByTwo440.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-facebookJumbo.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-miniMoth.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-windowsTile336H.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-articleLarge.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-blog480.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-blog427.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-tmagArticle.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-jumbo.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-blog225.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-master675.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-master180.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-popup.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-blog533.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-tmagSF.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-slide.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-superJumbo.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-master495.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-master315.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-square320.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-filmstrip.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-square640.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-moth.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-mediumSquare149.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-articleInline.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-hpSmall.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-blogSmallInline.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-mediumFlexible177.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-sfSpan.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-largeHorizontal375.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-hpLarge.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-largeWidescreen573.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-videoSmall.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-videoHpMedium.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-videoSixteenByNine600.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-videoSixteenByNine540.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-videoSixteenByNine495.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-videoSixteenByNine390.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-videoSixteenByNine480.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-videoSixteenByNine310.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-videoSixteenByNine225.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-videoSixteenByNine96.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-videoSixteenByNine768.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-videoSixteenByNine150.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-thumbStandard.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-thumbLarge.jpg+images/2014/12/22/multimedia/the-best-of-2014-1419258019643/the-best-of-2014-1419258019643-blogSmallThumb.jpg
5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     images/2014/12/21/magazine/21talk/21talk-videoSmall-v3.jpg+images/2014/12/21/magazine/21talk/21talk-videoHpMedium-v3.jpg+images/2014/12/21/magazine/21talk/21talk-videoSixteenByNine600-v3.jpg+images/2014/12/21/magazine/21talk/21talk-videoSixteenByNine540-v3.jpg+images/2014/12/21/magazine/21talk/21talk-videoSixteenByNine495-v3.jpg+images/2014/12/21/magazine/21talk/21talk-videoSixteenByNine390-v3.jpg+images/2014/12/21/magazine/21talk/21talk-videoSixteenByNine480-v3.jpg+images/2014/12/21/magazine/21talk/21talk-videoSixteenByNine310-v3.jpg+images/2014/12/21/magazine/21talk/21talk-videoSixteenByNine225-v3.jpg+images/2014/12/21/magazine/21talk/21talk-videoSixteenByNine96-v3.jpg+images/2014/12/21/magazine/21talk/21talk-videoSixteenByNine768-v3.jpg+images/2014/12/21/magazine/21talk/21talk-videoSixteenByNine150-v3.jpg+images/2014/12/21/magazine/21talk/21talk-videoSixteenByNine1050-v3.jpg+images/2014/12/21/magazine/21talk/21talk-hpLarge-v3.jpg+images/2014/12/21/magazine/21talk/21talk-largeWidescreen573-v3.jpg+images/2014/12/21/magazine/21talk/21talk-largeWidescreen1050-v3.jpg+images/2014/12/21/magazine/21talk/21talk-articleLarge-v2.jpg+images/2014/12/21/magazine/21talk/21talk-blog480-v2.jpg+images/2014/12/21/magazine/21talk/21talk-blog427-v2.jpg+images/2014/12/21/magazine/21talk/21talk-tmagArticle-v2.jpg+images/2014/12/21/magazine/21talk/21talk-jumbo-v2.jpg+images/2014/12/21/magazine/21talk/21talk-blog225-v2.jpg+images/2014/12/21/magazine/21talk/21talk-master675-v2.jpg+images/2014/12/21/magazine/21talk/21talk-master180-v2.jpg+images/2014/12/21/magazine/21talk/21talk-popup-v2.jpg+images/2014/12/21/magazine/21talk/21talk-blog533-v2.jpg+images/2014/12/21/magazine/21talk/21talk-tmagSF-v2.jpg+images/2014/12/21/magazine/21talk/21talk-slide-v2.jpg+images/2014/12/21/magazine/21talk/21talk-superJumbo-v2.jpg+images/2014/12/21/magazine/21talk/21talk-master1050-v2.jpg+images/2014/12/21/magazine/21talk/21talk-master495-v2.jpg+images/2014/12/21/magazine/21talk/21talk-master315-v2.jpg+images/2014/12/21/magazine/21talk/21talk-miniMoth-v3.jpg+images/2014/12/21/magazine/21talk/21talk-windowsTile336H-v3.jpg+images/2014/12/21/magazine/21talk/21talk-sfSpan.jpg+images/2014/12/21/magazine/21talk/21talk-largeHorizontal375.jpg+images/2014/12/21/magazine/21talk/21talk-verticalTwoByThree735-v2.jpg+images/2014/12/21/magazine/21talk/21talk-square320.jpg+images/2014/12/21/magazine/21talk/21talk-filmstrip.jpg+images/2014/12/21/magazine/21talk/21talk-square640.jpg+images/2014/12/21/magazine/21talk/21talk-moth.jpg+images/2014/12/21/magazine/21talk/21talk-mediumSquare149.jpg+images/2014/12/21/magazine/21talk/21talk-articleInline-v2.jpg+images/2014/12/21/magazine/21talk/21talk-hpSmall-v2.jpg+images/2014/12/21/magazine/21talk/21talk-blogSmallInline-v2.jpg+images/2014/12/21/magazine/21talk/21talk-mediumFlexible177-v2.jpg+images/2014/12/21/magazine/21talk/21talk-facebookJumbo-v2.jpg+images/2014/12/21/magazine/21talk/21talk-thumbStandard.jpg+images/2014/12/21/magazine/21talk/21talk-thumbLarge.jpg+images/2014/12/21/magazine/21talk/21talk-blogSmallThumb.jpg+images/2014/12/21/magazine/21talk/21talk-thumbWide-v3.jpg+images/2014/12/21/magazine/21talk/21talk-videoThumb-v3.jpg+images/2014/12/21/magazine/21talk/21talk-videoLarge-v3.jpg+images/2014/12/21/magazine/21talk/21talk-mediumThreeByTwo210-v3.jpg+images/2014/12/21/magazine/21talk/21talk-mediumThreeByTwo225-v3.jpg+images/2014/12/21/magazine/21talk/21talk-mediumThreeByTwo440-v3.jpg+images/2014/12/21/magazine/21talk/21talk-mediumThreeByTwo252-v2.jpg+images/2014/12/21/magazine/21talk/21talk-mediumThreeByTwo378-v2.jpg+images/2014/12/21/magazine/21talk/21talk-watch308-v2.jpg+images/2014/12/21/magazine/21talk/21talk-watch268-v2.jpg
6                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    images/2014/12/14/business/14-BITS/14-BITS-thumbWide.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoThumb.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoLarge.jpg+images/2014/12/14/business/14-BITS/14-BITS-mediumThreeByTwo210.jpg+images/2014/12/14/business/14-BITS/14-BITS-mediumThreeByTwo225.jpg+images/2014/12/14/business/14-BITS/14-BITS-mediumThreeByTwo440.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoFifteenBySeven1305.jpg+images/2014/12/14/business/14-BITS/14-BITS-miniMoth.jpg+images/2014/12/14/business/14-BITS/14-BITS-windowsTile336H.jpg+images/2014/12/14/business/14-BITS/14-BITS-articleLarge.jpg+images/2014/12/14/business/14-BITS/14-BITS-blog480.jpg+images/2014/12/14/business/14-BITS/14-BITS-blog427.jpg+images/2014/12/14/business/14-BITS/14-BITS-tmagArticle.jpg+images/2014/12/14/business/14-BITS/14-BITS-jumbo.jpg+images/2014/12/14/business/14-BITS/14-BITS-blog225.jpg+images/2014/12/14/business/14-BITS/14-BITS-master675.jpg+images/2014/12/14/business/14-BITS/14-BITS-master180.jpg+images/2014/12/14/business/14-BITS/14-BITS-popup.jpg+images/2014/12/14/business/14-BITS/14-BITS-blog533.jpg+images/2014/12/14/business/14-BITS/14-BITS-tmagSF.jpg+images/2014/12/14/business/14-BITS/14-BITS-slide.jpg+images/2014/12/14/business/14-BITS/14-BITS-superJumbo.jpg+images/2014/12/14/business/14-BITS/14-BITS-master1050.jpg+images/2014/12/14/business/14-BITS/14-BITS-master495.jpg+images/2014/12/14/business/14-BITS/14-BITS-master315.jpg+images/2014/12/14/business/14-BITS/14-BITS-square320.jpg+images/2014/12/14/business/14-BITS/14-BITS-filmstrip.jpg+images/2014/12/14/business/14-BITS/14-BITS-square640.jpg+images/2014/12/14/business/14-BITS/14-BITS-moth.jpg+images/2014/12/14/business/14-BITS/14-BITS-mediumSquare149.jpg+images/2014/12/14/business/14-BITS/14-BITS-articleInline.jpg+images/2014/12/14/business/14-BITS/14-BITS-hpSmall.jpg+images/2014/12/14/business/14-BITS/14-BITS-blogSmallInline.jpg+images/2014/12/14/business/14-BITS/14-BITS-mediumFlexible177.jpg+images/2014/12/14/business/14-BITS/14-BITS-sfSpan.jpg+images/2014/12/14/business/14-BITS/14-BITS-largeHorizontal375.jpg+images/2014/12/14/business/14-BITS/14-BITS-hpLarge.jpg+images/2014/12/14/business/14-BITS/14-BITS-largeWidescreen573.jpg+images/2014/12/14/business/14-BITS/14-BITS-largeWidescreen1050.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoSmall.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoHpMedium.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoSixteenByNine600.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoSixteenByNine540.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoSixteenByNine495.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoSixteenByNine390.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoSixteenByNine480.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoSixteenByNine310.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoSixteenByNine225.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoSixteenByNine96.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoSixteenByNine768.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoSixteenByNine150.jpg+images/2014/12/14/business/14-BITS/14-BITS-videoSixteenByNine1050.jpg+images/2014/12/14/business/14-BITS/14-BITS-thumbStandard.jpg+images/2014/12/14/business/14-BITS/14-BITS-thumbLarge.jpg+images/2014/12/14/business/14-BITS/14-BITS-blogSmallThumb.jpg
          news_desk print_page print_section            pub_date
1          Business          1             B 2014-12-31 22:31:54
2                         <NA>          <NA> 2014-12-30 22:06:27
3              OpEd         19             A 2014-12-29 23:20:06
4 Multimedia/Photos       <NA>          <NA> 2014-12-22 18:25:37
5          Magazine         14            MM 2014-12-19 15:24:37
6          Business          4            BU 2014-12-13 17:37:26
       section_name
1        Technology
2             Blogs
3           Opinion
4 Multimedia/Photos
5          Magazine
6        Technology
                                                                                                                                                                      snippet
1                                                           A service lets a person monitor his or her Facebook or Twitter account, for more awareness of one’s online image.
2 Get recommendations from New York Times reporters and editors, highlighting great stories from around the web. Today, great reads from Dean Baquet, Susan Chira and others.
3                                                                                                                                The answers for our ninth annual Op-Ed quiz.
4                                                                                                   Highlights from the year, as chosen by the editors of The New York Times.
5                                                                                             The women’s-magazine editor on the Internet, feminism and reading the comments.
6                                                               Social media companies are often reluctant to become arbiters of appropriate and inappropriate speech online.
              source subsection_name    type_of_material
1 The New York Times   Personal Tech                News
2 The New York Times            <NA>                News
3 The New York Times            <NA>               Op-Ed
4 The New York Times            <NA> Interactive Feature
5 The New York Times            <NA>                News
6 The New York Times            <NA>                News
                                                     uri
1     nyt://article/08765e5b-8d12-54dd-be58-39c6d33125c1
2     nyt://article/5e97a537-4e5d-51b4-9571-736913a6e5c4
3     nyt://article/ebad4be5-8e52-5490-a3bb-f5d8f684b902
4 nyt://interactive/26986d5d-2854-5484-86b8-04dbbfea0b27
5     nyt://article/fe201e9b-ea3b-5e9c-bb5a-ef4b0bbd30c4
6     nyt://article/7574b532-e51f-5695-80aa-a5b24ee4e7a2
                                                                                                                    web_url
1 https://www.nytimes.com/2015/01/01/technology/personaltech/thinkup-helps-the-social-network-user-see-the-online-self.html
2               https://news.blogs.nytimes.com/2014/12/30/what-were-reading-great-times-reads-of-2014-from-our-top-editors/
3                                   https://www.nytimes.com/2014/12/30/opinion/2014-the-year-in-questions-quiz-answers.html
4                                                 https://www.nytimes.com/interactive/2014/multimedia/the-best-of-2014.html
5                 https://www.nytimes.com/2014/12/21/magazine/jane-pratt-on-why-writing-for-young-women-never-gets-old.html
6                              https://bits.blogs.nytimes.com/2014/12/13/trying-to-swim-in-a-sea-of-social-media-invective/
  word_count
1       1211
2        429
3        588
4          0
5        716
6       1236

Direct access with httr2

The following URL is the URL that accesses the same Gamergate article data that we searched for above with the nytimes wrapper package. Open this URL in your browser (you should replace MY_KEY with the API key you were given earlier).

http://api.nytimes.com/svc/search/v2/articlesearch.json?q=gamergate&api-key=MY_KEY

The text you see in the browser is the response data (in JSON format). This URL has the same structure that we discussed above for the census API:

• http:// — The scheme, which tells your browser or program how to communicate with the web server. This will typically be either http: or https:.
• api.nytimes.com — The hostname, which is a name that identifies the web server that will process the request.
• /svc/search/v2/articlesearch.json — The path, which tells the web server what function you would like to call (a function for searching articles).
• ?q=gamergate&api-key=MY_KEY — The query parameters, which provide the parameters for the function you would like to call. The key value pairs are the following:

key	value
q	gamergate
api-key	MY_KEY

The scheme, hostname, and path (http://api.nytimes.com/svc/search/v2/articlesearch.json) together form the endpoint for the API call.

We can use the httr2 package to build up a full URL from its parts:

req <- request("http://api.nytimes.com") %>% 
    req_url_path_append("svc") %>% 
    req_url_path_append("search") %>% 
    req_url_path_append("v2") %>% 
    req_url_path_append("articlesearch.json") %>% 
    req_url_query(q = "gamergate", `api-key` = times_key)
req

<httr2_request>

GET
http://api.nytimes.com/svc/search/v2/articlesearch.json?q=gamergate&api-key=xy9oy1eczTOTGAFjAfnrmZJO2mpSPvXQ

Body: empty

Copy and paste the resulting URL into your browser to see what the NYT response looks like. The sets of curly braces enclosing key value pairs (key: value) is JSON data.

We can write a function to generate the URL for a user-specified query:

create_nyt_url <- function(query, key) {
    request("http://api.nytimes.com") %>% 
        req_url_path_append("svc") %>% 
        req_url_path_append("search") %>% 
        req_url_path_append("v2") %>% 
        req_url_path_append("articlesearch.json") %>% 
        req_url_query(q = query, `api-key` = key)
}

Let’s use this function to find articles related to:

• Ferris Bueller's Day Off (note the spaces and the apostrophe)
• Penn & Teller (note the spaces and the punctuation mark &)

Let’s see how these queries are translated into the URLs:

req_fb <- create_nyt_url(query = "Ferris Bueller's Day Off", key = times_key)
req_fb

<httr2_request>

GET
http://api.nytimes.com/svc/search/v2/articlesearch.json?q=Ferris%20Bueller%27s%20Day%20Off&api-key=xy9oy1eczTOTGAFjAfnrmZJO2mpSPvXQ

Body: empty

req_pt <- create_nyt_url(query = "Penn & Teller", key = times_key)
req_pt

<httr2_request>

GET
http://api.nytimes.com/svc/search/v2/articlesearch.json?q=Penn%20%26%20Teller&api-key=xy9oy1eczTOTGAFjAfnrmZJO2mpSPvXQ

Body: empty

Special characters (like punctuation and spaces) have to be represented in a special way in URLs–this process is called percent encoding.

We can use req_perform() to send out the request and resp_body_json() to parse the resulting JSON:

resp_pt <- req_pt %>% req_perform() %>% resp_body_json(simplifyVector = TRUE)

Digging into JSON structure

Let’s dig deeper into the structure of the JSON. We can visit the URL in req_pt to view the “Penn & Teller” results. The first part of it looks like this:

{"status":"OK","copyright":"Copyright (c) 2024 The New York Times Company. All Rights Reserved.","response":{"docs":[{"abstract":"Now in its 10th season, “Penn & Teller: Fool Us” has become TV’s premier showcase for magicians and the rare competition show where kindness rules.","web_url":"https://www.nytimes.com/2023/11/02/arts/television/penn-teller-fool-us.html","snippet":"Now in its 10th season, “Penn & Teller: Fool Us” has become TV’s premier showcase for magicians and the rare competition show where kindness rules.","lead_paragraph":"On the magician Shin Lim’s first appearance on “Penn & Teller: Fool Us,” he made cards rise out of a tabletop and vanish, only for them to reappear inside a clear sealed bag, or emerge from his mouth, engulfed in smoke and folded in four.","print_section":"C","print_page":"5","source":"The New York Times","multimedia":

If you stared very hard at the above response, you may be able to interpret it. However, it would be much easier to interact with the response in some more structured, programmatic way. The vast majority of Web APIs, including the New York Times, use a standard called JSON (Javascript Object Notation) to take data and encode it as a string.

To understand the structure of JSON, take the NYT web response in your browser, and copy and paste it into this online JSON formatter. The formatter will add newlines and tabs to make the data more human-readable. You’ll see the following:

{
   "status":"OK",
   "copyright":"Copyright (c) 2024 The New York Times Company. All Rights Reserved.",
   "response":{  
      "docs":[  
      
        # A HUGE piece of data, with one object for each of the result articles
        
      ],
      "meta":{
         "hits":1783,
         "offset":0,
         "time":87
      }
   }
}     

You’ll notice a few things in the JSON above:

• Strings are enclosed in double quotes, for example "status" and "OK".
• Numbers are written plainly, like 1783 ajd 87.
• Some data is enclosed in square brackets [ and ]. These data containers can be thought of as R lists.
• Some data is enclosed in curly braces { and }. These data containers are called Objects. An object can be thought of as a single case or observation in a table.
  • The columns or variables for the observation appear as keys on the left (hits, offset, time).
  • The values appear after the specific key separated by a colon (1783, 0, 87).

Thus, we can think of the meta object above as:

hits	offset	time
1755	0	51

Exploring complex lists

resp_pt is a list. A list is a useful structure for storing elements of different types. Data frames are special cases of lists where each list element has the same length (but where the list elements have different classes).

Lists are a very flexible data structure but can be very confusing because list elements can be lists themselves!

We can explore the structure of a list in two ways:

• Entering View(list_object) in the Console. The triangle buttons on the left allow you to toggle dropdowns to explore list elements.
• Using the str() (structure) function.

Exercise: Explore the information in the resp_pt using both View() and str(). When using str(), look up the documentation and experiment with the max.level and vec.len arguments to control how the output is displayed. Look back and forth between the View() and str() output to find correspondences in how object structure is displayed.

---

We can access elements of a list in three ways:

• By position with double square brackets [[:

# This gets the first element of the list
resp_pt[[1]]

[1] "OK"

• By name with double square brackets [[: (note that list elements are not always named, so this won’t always be possible)

# Accessing by name directly
resp_pt[["status"]]

[1] "OK"

# Accessing via a variable
which_element <- "status"
resp_pt[[which_element]]

[1] "OK"

• By name with a dollar sign $: (Helpful tip: For this mode of access, RStudio allows tab completion to fill in the full name)

resp_pt$status

[1] "OK"

We can retrieve these nested attributes by sequentially accessing the object keys from the outside in. For example, the meta element would be accessed as follows:

resp_pt$response$meta

$hits
[1] 1783

$offset
[1] 0

$time
[1] 116

Exercise: In the resp_pt object, retrieve the data associated with:

• the copyright key
• the number of hits (number of search results) within the meta object
• the abstracts and leading paragraphs of the articles found in the search

Solution

resp_pt$copyright

[1] "Copyright (c) 2024 The New York Times Company. All Rights Reserved."

resp_pt$response$meta$hits

[1] 1783

resp_pt$response$docs$abstract

 [1] "Now in its 10th season, “Penn & Teller: Fool Us” has become TV’s premier showcase for magicians and the rare competition show where kindness rules."                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
 [2] "Who would have guessed that magic’s most recognizable buddy pair would produce the classiest reality show on television?"                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
 [3] "The magic show is earning more than most of the musicals and plays on Broadway."                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
 [4] "  PENN JILLETTE is a tall, bespectacled fellow who looks and sounds like an untenured English instructor driven slightly goofy by too many nights spent grading student term papers. His much shorter partner, known only as Teller, is a gloomy- faced imp who never speaks. In their gray, Brooks Brothers-style suits, neither man is particularly prepossessing, and neither much fits the conventional image of an entertainer. But entertainers they are, and the little show they've brought to the Westside Arts Theater is one of the season's most original infusions of fun.     ''Penn & Teller'' is in the same spirit as such New Wave vaudeville turns as the Flying Karamazov Brothers and Avner the Eccentric. The performers are master magicians and jugglers who fold their tricks into hip comic riffs. Yet Penn and Teller, unlike their peers, will be enjoyed even more by adults than by children. This pair's jokes are often as witty as their stunts are amazing. What's more, their humor and sleights-of-hand merge into a theatrical statement that runs deeper than the act's individual ingredients might suggest."
 [5] "Charlie Brown gets the holiday blues. And amateur magicians try to fool Penn & Teller."                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
 [6] "The magicians perform many familiar tricks in their show at the Marquis Theater, but they also demonstrate a 21st-century edge."                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
 [7] "Strang and Lafean held in bail"                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
 [8] "The duo’s new magic show had a solid first week at the box office."                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 
 [9] "Interview with Penn Jillette and Teller, partners in comedy and magic act, following appearance on The Daily Show, on Comedy Central network; two have their own new weekly television series, Penn & Teller's Sin City Spectacular, on F/X Network (S)"                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            
[10] "  \"Penn and Teller: The Refrigerator Tour\" is to close after its second performance on June 29 at the Eugene O'Neill Theater, 230 West 49th Street. That show will be the 103d regular performance for the comedy; it had 9 previews."                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            

resp_pt$response$docs$lead_paragraph

 [1] "On the magician Shin Lim’s first appearance on “Penn & Teller: Fool Us,” he made cards rise out of a tabletop and vanish, only for them to reappear inside a clear sealed bag, or emerge from his mouth, engulfed in smoke and folded in four."                                                                                                                                                                                                                                                                                                                                                                                          
 [2] "“Penn & Teller: Fool Us” is a reality-TV competition shown on the CW, which is a broadcast network, which is something like a streaming service that’s always on. The show was recently renewed for its seventh season. The only other person I know who watches it is a skilled amateur magician and general magic geek who lives in Chicago. For him, the show is a chance to be exposed to some of the world’s greatest magicians and get an insight into their arcane techniques. For me, who doesn’t particularly like magic and has no intention of trying to do it, the show has a different appeal: It makes me a better person."
 [3] "In one of their onstage illusions, Penn & Teller make coins appear in a fishbowl and hand them to an audience member."                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   
 [4] "PENN JILLETTE is a tall, bespectacled fellow who looks and sounds like an untenured English instructor driven slightly goofy by too many nights spent grading student term papers. His much shorter partner, known only as Teller, is a gloomy- faced imp who never speaks. In their gray, Brooks Brothers-style suits, neither man is particularly prepossessing, and neither much fits the conventional image of an entertainer. But entertainers they are, and the little show they've brought to the Westside Arts Theater is one of the season's most original infusions of fun."                                                   
 [5] "Charlie Brown gets the holiday blues. And amateur magicians try to fool Penn & Teller."                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  
 [6] "Though it’s been four decades since they first teamed up, Penn and Teller are looking terribly of-the-moment these days. Never mind that their latest entertaining exercise in populist hocus-pocus — “Penn & Teller on Broadway,” which opened on Sunday night at the Marquis Theater — includes some of the oldest tricks in any conjurer’s book, including the extraction of a rabbit from a top hat and sawing a woman in half. "                                                                                                                                                                                                    
 [7] ""                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        
 [8] "“Penn & Teller on Broadway,” a new show of high-tech tricks and old-school routines from the modern magic duo, had a strong start on Broadway, according to box office data released on Monday. The show, which opened on Sunday night to mostly good reviews, grossed $1,075,289 for eight performances in its first week at the 1,634-seat Marquis Theater."                                                                                                                                                                                                                                                                           
 [9] "PENN JILLETTE -- the larger, more boisterous half of the comedy and magic act Penn & Teller -- stripped off his gray suit, his red-and-black silk tie and his red Doc Marten boots (size 15) and stood in the green room of ''The Daily Show'' on Thursday evening, naked save for Calvin Klein briefs."                                                                                                                                                                                                                                                                                                                                 
[10] "\"Penn and Teller: The Refrigerator Tour\" is to close after its second performance on June 29 at the Eugene O'Neill Theater, 230 West 49th Street. That show will be the 103d regular performance for the comedy; it had 9 previews."                                                                                                                                                                                                                                                                                                                                                                                                   

# Both (abstract and leading paragraph) at once
resp_pt$response$docs[c("abstract", "lead_paragraph")]

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              abstract
1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Now in its 10th season, “Penn & Teller: Fool Us” has become TV’s premier showcase for magicians and the rare competition show where kindness rules.
2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             Who would have guessed that magic’s most recognizable buddy pair would produce the classiest reality show on television?
3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      The magic show is earning more than most of the musicals and plays on Broadway.
4    PENN JILLETTE is a tall, bespectacled fellow who looks and sounds like an untenured English instructor driven slightly goofy by too many nights spent grading student term papers. His much shorter partner, known only as Teller, is a gloomy- faced imp who never speaks. In their gray, Brooks Brothers-style suits, neither man is particularly prepossessing, and neither much fits the conventional image of an entertainer. But entertainers they are, and the little show they've brought to the Westside Arts Theater is one of the season's most original infusions of fun.     ''Penn & Teller'' is in the same spirit as such New Wave vaudeville turns as the Flying Karamazov Brothers and Avner the Eccentric. The performers are master magicians and jugglers who fold their tricks into hip comic riffs. Yet Penn and Teller, unlike their peers, will be enjoyed even more by adults than by children. This pair's jokes are often as witty as their stunts are amazing. What's more, their humor and sleights-of-hand merge into a theatrical statement that runs deeper than the act's individual ingredients might suggest.
5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               Charlie Brown gets the holiday blues. And amateur magicians try to fool Penn & Teller.
6                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      The magicians perform many familiar tricks in their show at the Marquis Theater, but they also demonstrate a 21st-century edge.
7                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       Strang and Lafean held in bail
8                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   The duo’s new magic show had a solid first week at the box office.
9                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Interview with Penn Jillette and Teller, partners in comedy and magic act, following appearance on The Daily Show, on Comedy Central network; two have their own new weekly television series, Penn & Teller's Sin City Spectacular, on F/X Network (S)
10                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 "Penn and Teller: The Refrigerator Tour" is to close after its second performance on June 29 at the Eugene O'Neill Theater, 230 West 49th Street. That show will be the 103d regular performance for the comedy; it had 9 previews.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             lead_paragraph
1                                                                                                                                                                                                                                                                                                                                                                                            On the magician Shin Lim’s first appearance on “Penn & Teller: Fool Us,” he made cards rise out of a tabletop and vanish, only for them to reappear inside a clear sealed bag, or emerge from his mouth, engulfed in smoke and folded in four.
2  “Penn & Teller: Fool Us” is a reality-TV competition shown on the CW, which is a broadcast network, which is something like a streaming service that’s always on. The show was recently renewed for its seventh season. The only other person I know who watches it is a skilled amateur magician and general magic geek who lives in Chicago. For him, the show is a chance to be exposed to some of the world’s greatest magicians and get an insight into their arcane techniques. For me, who doesn’t particularly like magic and has no intention of trying to do it, the show has a different appeal: It makes me a better person.
3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     In one of their onstage illusions, Penn & Teller make coins appear in a fishbowl and hand them to an audience member.
4                                                     PENN JILLETTE is a tall, bespectacled fellow who looks and sounds like an untenured English instructor driven slightly goofy by too many nights spent grading student term papers. His much shorter partner, known only as Teller, is a gloomy- faced imp who never speaks. In their gray, Brooks Brothers-style suits, neither man is particularly prepossessing, and neither much fits the conventional image of an entertainer. But entertainers they are, and the little show they've brought to the Westside Arts Theater is one of the season's most original infusions of fun.
5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Charlie Brown gets the holiday blues. And amateur magicians try to fool Penn & Teller.
6                                                                                                                                                                                                      Though it’s been four decades since they first teamed up, Penn and Teller are looking terribly of-the-moment these days. Never mind that their latest entertaining exercise in populist hocus-pocus — “Penn & Teller on Broadway,” which opened on Sunday night at the Marquis Theater — includes some of the oldest tricks in any conjurer’s book, including the extraction of a rabbit from a top hat and sawing a woman in half. 
7                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
8                                                                                                                                                                                                                                                                             “Penn & Teller on Broadway,” a new show of high-tech tricks and old-school routines from the modern magic duo, had a strong start on Broadway, according to box office data released on Monday. The show, which opened on Sunday night to mostly good reviews, grossed $1,075,289 for eight performances in its first week at the 1,634-seat Marquis Theater.
9                                                                                                                                                                                                                                                                                                                                   PENN JILLETTE -- the larger, more boisterous half of the comedy and magic act Penn & Teller -- stripped off his gray suit, his red-and-black silk tie and his red Doc Marten boots (size 15) and stood in the green room of ''The Daily Show'' on Thursday evening, naked save for Calvin Klein briefs.
10                                                                                                                                                                                                                                                                                                                                                                                                      "Penn and Teller: The Refrigerator Tour" is to close after its second performance on June 29 at the Eugene O'Neill Theater, 230 West 49th Street. That show will be the 103d regular performance for the comedy; it had 9 previews.

resp_pt$response$docs %>%
    select(abstract, lead_paragraph)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              abstract
1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Now in its 10th season, “Penn & Teller: Fool Us” has become TV’s premier showcase for magicians and the rare competition show where kindness rules.
2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             Who would have guessed that magic’s most recognizable buddy pair would produce the classiest reality show on television?
3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      The magic show is earning more than most of the musicals and plays on Broadway.
4    PENN JILLETTE is a tall, bespectacled fellow who looks and sounds like an untenured English instructor driven slightly goofy by too many nights spent grading student term papers. His much shorter partner, known only as Teller, is a gloomy- faced imp who never speaks. In their gray, Brooks Brothers-style suits, neither man is particularly prepossessing, and neither much fits the conventional image of an entertainer. But entertainers they are, and the little show they've brought to the Westside Arts Theater is one of the season's most original infusions of fun.     ''Penn & Teller'' is in the same spirit as such New Wave vaudeville turns as the Flying Karamazov Brothers and Avner the Eccentric. The performers are master magicians and jugglers who fold their tricks into hip comic riffs. Yet Penn and Teller, unlike their peers, will be enjoyed even more by adults than by children. This pair's jokes are often as witty as their stunts are amazing. What's more, their humor and sleights-of-hand merge into a theatrical statement that runs deeper than the act's individual ingredients might suggest.
5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               Charlie Brown gets the holiday blues. And amateur magicians try to fool Penn & Teller.
6                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      The magicians perform many familiar tricks in their show at the Marquis Theater, but they also demonstrate a 21st-century edge.
7                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       Strang and Lafean held in bail
8                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   The duo’s new magic show had a solid first week at the box office.
9                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Interview with Penn Jillette and Teller, partners in comedy and magic act, following appearance on The Daily Show, on Comedy Central network; two have their own new weekly television series, Penn & Teller's Sin City Spectacular, on F/X Network (S)
10                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 "Penn and Teller: The Refrigerator Tour" is to close after its second performance on June 29 at the Eugene O'Neill Theater, 230 West 49th Street. That show will be the 103d regular performance for the comedy; it had 9 previews.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             lead_paragraph
1                                                                                                                                                                                                                                                                                                                                                                                            On the magician Shin Lim’s first appearance on “Penn & Teller: Fool Us,” he made cards rise out of a tabletop and vanish, only for them to reappear inside a clear sealed bag, or emerge from his mouth, engulfed in smoke and folded in four.
2  “Penn & Teller: Fool Us” is a reality-TV competition shown on the CW, which is a broadcast network, which is something like a streaming service that’s always on. The show was recently renewed for its seventh season. The only other person I know who watches it is a skilled amateur magician and general magic geek who lives in Chicago. For him, the show is a chance to be exposed to some of the world’s greatest magicians and get an insight into their arcane techniques. For me, who doesn’t particularly like magic and has no intention of trying to do it, the show has a different appeal: It makes me a better person.
3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     In one of their onstage illusions, Penn & Teller make coins appear in a fishbowl and hand them to an audience member.
4                                                     PENN JILLETTE is a tall, bespectacled fellow who looks and sounds like an untenured English instructor driven slightly goofy by too many nights spent grading student term papers. His much shorter partner, known only as Teller, is a gloomy- faced imp who never speaks. In their gray, Brooks Brothers-style suits, neither man is particularly prepossessing, and neither much fits the conventional image of an entertainer. But entertainers they are, and the little show they've brought to the Westside Arts Theater is one of the season's most original infusions of fun.
5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Charlie Brown gets the holiday blues. And amateur magicians try to fool Penn & Teller.
6                                                                                                                                                                                                      Though it’s been four decades since they first teamed up, Penn and Teller are looking terribly of-the-moment these days. Never mind that their latest entertaining exercise in populist hocus-pocus — “Penn & Teller on Broadway,” which opened on Sunday night at the Marquis Theater — includes some of the oldest tricks in any conjurer’s book, including the extraction of a rabbit from a top hat and sawing a woman in half. 
7                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
8                                                                                                                                                                                                                                                                             “Penn & Teller on Broadway,” a new show of high-tech tricks and old-school routines from the modern magic duo, had a strong start on Broadway, according to box office data released on Monday. The show, which opened on Sunday night to mostly good reviews, grossed $1,075,289 for eight performances in its first week at the 1,634-seat Marquis Theater.
9                                                                                                                                                                                                                                                                                                                                   PENN JILLETTE -- the larger, more boisterous half of the comedy and magic act Penn & Teller -- stripped off his gray suit, his red-and-black silk tie and his red Doc Marten boots (size 15) and stood in the green room of ''The Daily Show'' on Thursday evening, naked save for Calvin Klein briefs.
10                                                                                                                                                                                                                                                                                                                                                                                                      "Penn and Teller: The Refrigerator Tour" is to close after its second performance on June 29 at the Eugene O'Neill Theater, 230 West 49th Street. That show will be the 103d regular performance for the comedy; it had 9 previews.