Applied Data Analysis

“[This class] will acquaint students with statistical methodology as it is used in the social sciences.”

Expectation…

Silver Bulletin election forecast chart of midterm win probabilities. — via Silver Bulletin

Steve Kornacki at the MSNBC big board on election night. — via ABC

Reality…

Cartoon of a frantic researcher holding red string next to a corkboard covered with sticky-note pictures of an ice cream cone, sun, bar chart, ballot box, and thermometer all connected by tangled red string, illustrating how messy causal inference can be.

Causal Questions in Political Science

Does ideological extremism cause a candidate to win an election?
Does inflation cause presidential approval to decrease?
Does democratization cause countries to become more peaceful?

Machine Learning

Which factors predict the outcome?

Now, you might be thinking: we’ve got tons of data and powerful computers—can’t we just feed it all into a model and let it find the answer? That instinct has a name, machine learning, and it’s worth understanding why it doesn’t quite answer questions like these.

When you do machine learning, you are interested in the relationships between variables. Let’s go back to the election example. There are a lot of different factors we might think are important when it comes to predicting election outcomes. The party of the candidate, how much money they raise, their race, the state of the economy, all sorts of things. And so, in machine learning, we might try to put all of those variables into a model and try to see which ones are most informative when predicting past election outcomes. And we might learn some are more important than others, and then we can use those same variables to try to predict this year’s election. If for example, we see that fundraising has strong predictive power in forecasting who won past elections, but whether a candidate has blonde hair does not, then we would use information about fundraising this year to predict the outcome, but not blonde hair.

Causal Inference

Does this factor cause the outcome?

Now, that is all well and good and fun. But the problem is that we do not necessarily learn which factors CAUSED a candidate to win the election. Only which factors are CORRELATED with the election outcome. Take fundraising. Candidates who raise more money do tend to win more often, so it shows up as a strong predictor. But does the money itself CAUSE the win? Maybe not. Think about who raises a lot in the first place. Donors like to back a winner—so the strong, popular candidates, the ones who already look like they’re going to win, are exactly the ones who attract the most money. And those same strong candidates are the ones who were likely to win anyway, big war chest or not. So fundraising and winning might both be driven by something else entirely: how strong the candidate is. The money didn’t cause the win, it’s just along for the ride.

So as social scientists, we are interested, often, not just in prediction, but in causal inference—understanding which factors actually CAUSE a candidate to win the election.

Correlation and Causation

XKCD comic: correlation does not imply causation, but it does waggle its eyebrows suggestively. — via XKCD

An Example

Does Eating Ice Cream Cause Sunburn?

Let’s zoom in on a bit of a crazy example. Do you think eating ice cream causes sunburn

Ok, right. Now, that’s kind of crazy. Eating ice cream doesn’t cause sunburn. Instead, what we have a case of here is the famous “correlation is not causation” saying. What’s going on, well, there’s a third variable that “confounds” the relationship between these two variables, and that’s the weather. When it is hot and sunny out, as it is in summer, people buy more ice cream and are more likely to get sunburn. However, if we try to use machine learning to discover relationships between variables, if we don’t include data on weather in our model, we might end up discovering a relationship between ice cream sales and sunburn.

So why does this matter? Well, if we work for a sunscreen company and we want to understand what factors increase sales of sunscreen, we might run a model and discover this relationship with ice cream. Then, we could get data on ice cream sales and start to predict when our sunscreen sales will go up. But we do not know that ice cream sales are a CAUSE of sunburn, and thus, sunscreen sales. And if we assume they are, we might make a mistake. What if, for instance, we assumed this relationship was causal and then we went out in the winter and invested in advertising for ice cream—expecting that this would help us increase sunscreen sales. That would be a huge waste of money!

So in this limited context, we already can see the importance of a causal relationship. But as social scientists it’s even more important.

Does Being an Extremist Increase Your Election Odds?

Marjorie Taylor Greene in a red jacket wearing a black face mask reading TRUMP WON, seated in the House chamber.

Marjorie Taylor Greene, via NPR

Wall Street Journal headline: Marjorie Taylor Greene Easily Wins Re-Election in Georgia.

Washington Post headline: The media can't ignore Marjorie Taylor Greene. Can they figure out how to cover her?

via Washington Post

Now, if we just want to know who’s going to win an election. Correlation may be enough. But as social scientists, we often don’t care about correlations between two variables—say being an extremist lawmaker and getting reelected. We really care to know if being an extremist lawmaker CAUSES one to increase their reelection chances. Because if that’s true, then we might expect candidates to be more extreme, which is perhaps bad for democracy.

Now, it could totally be true. Maybe being a more extremist lawmaker makes you more likely to get reelected. But maybe something else is going here. The media likes to present interesting, novel, shocking stories. And extremists candidates say a lot of shocking things. So the media wants to cover those candidates and what they have to say. But the normal boring politicians who say normal boring things don’t get media coverage. People don’t want to read stories like “Republican politicians says he supports tax cuts.” So in this story, extremists are getting media attention and normal politicians aren’t. So when voters go to polls, they are more familiar with the extremists than their normal boring opponents. Then extremists are more likely to win because voters are more familiar with them than their opponents. And here, we would have a spurious relationship just like the ice cream example.

These are very tricky problems to solve. But we really care about the answers. After all, if it’s about extremism, then maybe we think that there will be more extremist candidates in the future? But if it’s about media, then maybe the answer is that we should think about changing the media’s incentives to report sensationalist stories, by, say, refusing to read them.

But of course, this is very very hard to sort out!

Why Not Just Control for Everything?

What is “everything?”
Can you measure “everything?”
Controls can change what we’re estimating.

So now maybe you’re thinking…ok, well if the problem is that there are all these other factors floating around messing with our causal inference, let’s just “control” for them. Or include them in our model somehow.

But actually, that’s not a great solution either. Why not? Well for starters, what should we control for? Do you know all of the factors that might affect both X and Y? Both extremism and election outcomes? Probably not. So that kind of gets you in trouble right there.

Next, let’s just suppose you did know everything you needed to control for. Can you measure it all? Maybe, but that is also pretty hard. Maybe there is something with TV commercials that is confounding the relationship—but you can’t get that data on how many TV commercials all House incumbents ran. Or maybe it’s just something that’s super hard to measure, like personality. So that’s the next problem.

Finally, let’s suppose you really knew all of the factors and could measure them all well. Well, including everything in the model is going to potentially introduce spurious relationships.

For example, suppose we want to know about the effects of education on turnout. But we also think that political knowledge affects turnout. But then there’s a problem, because education probably also increases political knowledge. So by controlling for political knowledge, we’re estimating the effect of education holding political knowledge constant. That’s different than the effect of education by itself.

So, as you can see, this is pretty tricky. And the solution is not to control for everything, but rather, to think about how to design research studies to try to isolate the factors we care about and believe are causing the outcome. That’s why we’re going to spend a whole session on it—and we will just be scratching the surface.

About Me

Professor Benjamin Noble headshot.

From St. Louis, MO.
My research: congressional and presidential rhetoric, polarization.
I enjoy yoga, surfing, and cooking.
My favorite San Diego coffee shop: Lovesong (North Park).

Our Class

Our course syllabus is linked on Canvas.
Two meetings per week, fully remote over Zoom (link on Canvas).
- Monday & Wednesday, 11am–1:50pm.
- Each session mixes lecture and an in-class group lab.
Lectures are recorded and posted to Canvas. Please keep your camera on.

Alright, turning now to some of the logistics of this course. You all should have access to Canvas, which is going to be the primary place where you can find information about the course this session. Most importantly, you can find the syllabus on Canvas which covers class policies, course readings, homework, and exams. I’ll go over all of that momentarily.

So this class meets on Monday and Wednesday, from 11am to 1:50pm, and we are fully remote—everything happens over Zoom. The zoom link for class will always be the same, and it is on Canvas. Each session is going to mix lecture and a hands-on group lab. I’ll present the new material, and then you’ll get into your groups and work through an exercise that helps reinforce the lecture. That practice is really important, because it’s one thing to listen to me talk about coding and data analysis, but the only way you’re really going to learn is by doing it yourself.

Because we are remote, I will be recording the lectures and posting them to Canvas after class, so you can review or catch up if you need to. Breakout rooms, where the lab work happens, are not recorded. Please keep your camera on during class. We have a lot of group work and discussion, and seeing each other matters.

Our Textbooks

Our textbooks →
Both free online (see links on syllabus).
Reading can be done before/after lecture, should be done before class.

Cover of Causal Inference: The Mixtape by Scott Cunningham.

Cover of The Effect by Nick Huntington-Klein.

To help us learn the material we have two textbooks. Causal Inference: The Mixtape by Scott Cunningham and The Effect by Nick Huntington-Klein. Both of these textbooks are totally free and available online. You can access them by clicking the links of the syllabus. These books, especially the causal inference book, can be a bit technical and include a lot of mathmatical notation. As this is an applied class, our focus is going to be understand the concepts. While sometimes math notation can help with that, it can also be intimidating. So I don’t want you to focus too much on the math or get too stressed out if you don’t understand every sigma you see.

It’s kind of up to you when you do the reading. You might find it helpful to do the reading before lecture and then see me present the material. Alternatively, you might find it more helpful to come to lecture, watch me present the material, work through the lab, and then read about it to reinforce your understanding. That’s totally up to you.

Your Grade: Attendance and Participation (15%)

Attend lecture synchronously.
Keep your camera on.
Participate actively in labs and discussion.
One excused class absence, no questions asked.

Your Grade: Labs (25%)

In-class group work, submit your own files on Canvas.
Graded on participation and effort; you don’t need to finish.
Everyone will be called on at least once to walk the class through their solutions; answers will be posted after class.
One excused absence drops that day’s lab.

Now, one key component of your grade is going to be these in-class lab assignments. Each class session will include a group lab, and you submit them at the end of the class period. You can submit even if you haven’t finished. All I’m looking for is that you made a good effort, so as long as you submit something and have made some effort, you will get full credit. Now, even though you’re working in groups, you still run the code yourself and fill in the blanks with your group. Working through it yourself is how you learn to read and interpret the output.

After lab time, I’ll go over the solutions with the class. I’ll call on a few students each session to share their screen and explain what their group did. I’ll rotate through students over the course of the session, so everyone should expect to be called on at some point. It’s a chance to learn from each other and to make sure each of you is genuinely engaging with the material.

Your one excused class absence also drops the lab from that day. If I grant an excuse for an additional absence, the lab may be dropped or made up, depending on the situation.

Your Grade: Homework (30%)

Three individual homework assignments with conceptual and applied questions.
Completed on Canvas; you run provided R code in DataHub and enter your answers.
Must be done individually.
Late submissions minus one letter grade per 24 hours; no late submissions after solutions are posted.

The next component is homework. These build on what we do in lab and help you get practice using these methods and analyzing data. You will complete three homework assignments over the session. Each one is done on Canvas. You’ll run R code I provide in DataHub to get the numbers, then enter your answers in the Canvas quiz. You do not submit a code file, just the Canvas quiz.

This homework needs to be done individually. You’ll do the labs in groups, but you should complete the homework on your own. You may use the textbooks, your notes, the internet, and AI, but you may not consult with other students in the class.

I will accept late submissions, but they lose one letter grade every 24 hours they are late. Solutions will be posted one session after each homework is due, at which point late work cannot be accepted.

Your Grade: Oral Exam (25%)

One-on-one conversation with me over Zoom (about 8-10 minutes), during the last two class days.
I give you a short scenario; you talk me through how you’d answer a causal question about it.
No coding or analysis. It’s about reasoning, not execution.
You’ll get practice scenarios ahead of time; more details closer to the date.

Your Grade: Final (5%)

Asychronous final between Friday 7/24 through Friday 7/31.
Must be completed in one sitting (budget 2-3 hours).
Similar to the homeworks, but longer and more comprehensive.
Completed alone; open book, notes, internet, etc.

Your Grade

Attendance and participation (15%)
Labs (25%)
Homework (30%)
Oral Exam (25%)
Final (5%)

A Brief Note on R

I’ll be teaching this class using the R programming language.
You’ll mostly run code I provide and read the output.
No prior programming experience needed; you won’t write R from scratch.
Today’s lab and homework get you set up.

if (my_grade >= 90) {
  print('POLI 170 is the best!')
} else {
  print('POLI 170 stinks!')
}

A Brief Note on AI/LLMs

Feel free to use AI.
But, use it responsibly.

I want to take a minute now to briefly talk about AI and Large Language Models. In this class, I encourage you to use those tools—whether it’s ChatGPT, Claude, Gemini, or others. You are encouraged to use AI for anything we do in this class—for the labs, homework, and even the final.

However, with great power comes great responsibility. You need to be responsible in how you use these tools. As you probably know, AI models are not always correct or accurate. What they say is not always true or correct. After all, these models do not know the answers, they are often predicting the most likely word given a series of words that came before. This typically approximates knowledge, but it is prone to making mistakes and “hallucinating.” So, as a responsible user, you need to check the output of these models. They can be very helpful, but I often find that they are most helpful when you are stretching one or two steps beyond your current level of knowledge.

So, if you use AI unthinkingly and just copy/paste stuff into your homework without checking or validating or being pretty sure about what’s going on, you’re going to get a bad grade. When you’re coding, the checking can end up being pretty easy. Because you run the code and you do or don’t get the output you expect. The stuff I am teaching you in this class is not brand new material—people have been teaching this stuff for a long time and the LLMs likely know a lot about this and can help you understand concepts and respond to your specific questions. But again, this can be dangerous and tricky. There is no guarantee that the LLMs will give you the right answer all of the time. Therefore, you should converse with the LLM about statistical concepts, but unlike R code, it’s not as easy to check if you’re getting the right output. So, you should not only rely on the LLM, you should use it in combination with the textbooks, lectures, labs, and office hours to ensure you’re getting the right information.

Office Hours

Low stakes opportunity to get help or ask questions about the course/assignments.
Remote on Zoom (same link as class), Monday & Wednesday, 2:30–3:30pm.

Have a Question?

See if the answer is on the syllabus!
Attend office hours.
Email me (responses within 24-48hr, or longer on weekends).

How to Succeed

In this class (and in any stats class)

You are a math person!
(Obviously) do the reading, come to class, participate in lab.
Iterate, you don’t have to understand everything on the first go.
Get help.

Finally, before we wrap up, I’d like to give you a few tips on how to succeed in this class and any statistics based class.

First and foremost, I want to tell you that you are a math person. Now, perhaps you already believe that because you’ve selected into this class. But if you’re not sure, or if you’re uncertain, or if you’re here for the coding rather than the statistics, then I want you to know that you ARE a math person. You can do this. Now, that doesn’t mean it will be easy. It doesn’t mean that you won’t have to work at it. It doesn’t mean that some people won’t have an easier time than you. But that doesn’t mean you’re not a math person.

Now, this next one is pretty obvious (but even so, I have to say it because some people will still not follow this advice)—you should do the reading, come to class, and participate in lab. I put all of this together to help you learn this material, not for my own health and wellbeing. Keep in mind that every homework I assign you, I have to grade. Every reading I assign, I also have to read—and then write a lecture about it. Every lab, I have to write and test and make sure it works. That takes a lot of time and effort on my end! So, if I am assigning it, I probably think it’s important or helpful. So you should follow along!

Next, and perhaps my most important advice, is that you should iterate. What does that mean, it means you should not expect to understand everything on the first go. In fact, you might not even understand everything at the end of this class. And that’s alright! My goal is to show you these tools and concepts multiple time in multiple ways—in the book, in lecture, in the lab exercises. Hopefully, these different modes will accommodate different learning styles and expose you to the material multiple times so you can start to integrate it into your thinking. But even then, don’t think you need to have a perfect understanding of everything at the end of a week. You can revisit this material over and over, and in fact, because you will better understand it each time, the next time you come back to it, you will have the skills to understand it a little deeper until you really fully understand it. Even I learn new stuff or come to new understandings about this material each time I go through it.

Finally, if you’re confused or are having trouble, get help! Ask an LLM. Talk to others in the class. Come to my office hours. Don’t struggle alone. My best advice is that if you get stuck, spend 15-30 minutes trying to figure out the answer on your own, and if you can’t then go get help. Don’t just keep struggling and don’t give up.