to uncover causal relationships, cease making an attempt to invent a time machine and run an experiment as a substitute! Understanding causal relationships affords the data wanted to provide desired outcomes by motion. On this article, I’m going as an example the ability of experimental design by utilizing a time-machine-based conceptual train. My aim is to persuade you that extra may be discovered about causality by experimentation than utilizing a time machine.
Why are time machines helpful in a causal thought experiment?
Utilizing a time machine for a thought experiment feels ridiculous, and in some ways it’s. However it additionally has a attribute that makes it precious for exploring hypothetical outcomes. Time machines might give us one thing that we, in our time-bound state, can’t see – counterfactuals. Because the identify implies, a counterfactual is one thing that didn’t occur. They don’t seem to be observable by definition as a result of they by no means occurred. Counterfactuals what would’ve occurred underneath completely different circumstances. They provide solutions to questions like – “Would I’ve gotten sick if I didn’t eat that fuel station sushi?” If we had a time machine nonetheless, we might reverse the clock, do one thing completely different and see what occurs. Within the case of the sushi, I might restart the day, not eat the sushi and see if I nonetheless get sick. In different phrases, we might observe the in any other case unobservable counterfactuals.
The counterfactuals discovered by the point machine might then be in comparison with what truly occurred (we might name it a ‘factual’ I suppose…) to know the influence of an intervention. For our unlucky sushi instance, me getting sick is the ‘factual’ – it truly occurred. If I had a time machine, I might rewind time, not eat the sushi and observe what would’ve occurred, that is the counterfactual. I might then evaluate the factual with the counter factual to determine causality. Let’s say that I went again in time, stored all the pieces in my day the identical besides consuming the sushi. If I nonetheless obtained sick (factual = counterfactual), I do know that the sushi didn’t trigger the sickness as a result of I might’ve been sick both method. If I didn’t get sick nonetheless (factual ≠ counterfactual), then I can conclude that the sushi brought about my sickness. With a time machine, establishing causality for particular person occasions can be that straightforward!
At first look, it looks as if our time machine can also be an superior causality deducing machine! With the ability to observe counterfactuals can be very highly effective, however we will truly make extra helpful causal deductions utilizing well-designed experiments. Which is nice as a result of, time machines don’t exist, however well-designed experiments do! Let’s get into how designed experiments may be higher than utilizing a time machine.
The causality of particular person occasions is just not generalizable
Whereas a time machine would reply lots of curiosity-driven ‘what if’ causal questions, the learnings we might acquire from observing counterfactuals wouldn’t be generalizable to different, comparable (however not the identical) conditions. In my sushi instance, I might fulfill my curiosity by understanding if the sushi made me sick – however the data I gained wouldn’t serve any pragmatic goal for future selections. All I do know is that on that particular day, at that particular fuel station, at that particular time, that particular serving of sushi made me sick. I don’t know what would occur if I modified any of the bolded circumstances.
We are able to acquire generalizable data, which we wouldn’t get from the time machine, by designing an experiment. Generalizable data could be very helpful as a result of it might assist us make good selections sooner or later!
Think about that I ran an experiment that randomly assigned a number of courageous souls to eat fuel station sushi or restaurant sushi. This experiment would inform me if on common, fuel station sushi makes folks sicker than restaurant sushi. That is already an enchancment from the ‘time machine’ method as a result of the outcomes apply to the inhabitants of those that I sampled as a substitute making use of to me solely.
However, I might be smarter concerning the design of the experiment to get much more data! As an alternative of merely assigning folks to fuel station or restuarant sushi, I might assign folks specifc fuel stations at particular occasions or the restuarant at particular occasions. By including these two new variables (time and fuel station location) I can’t solely be taught if fuel station sushi makes folks sick extra usually, I may be taught if there are variations between the three fuel stations that serve sushi in my city and if time of day additionally has an influence.
On this experiment, I don’t immediately observe counterfactuals, however the randomized project helps confounders common out so I can estimate the common remedy impact (ATE) virtually as if I might observe counterfactuals.
How do the experiment learnings differ from my time machine learnings? The experiment is (1) utilizing a number of folks, (2) a number of sushi servings, (3) a number of fuel stations and (4) a number of occasions of day. In consequence, I can take away lots of causal insights that I and different folks can use. For instance, I might perceive if typically, fuel station sushi makes folks sicker than restaurant sushi in my city. I might additionally be taught if some fuel stations make folks extra sick than others and if shopping for sushi at some occasions is worse than others. This information may help me, and different folks make future selections. It’s way more helpful than understanding that the sushi from one fuel station and one time made me sick!
Along with all the variables that we will management, we will embrace covariates in our evaluation. Covariates are elements that we can’t management however are necessary. On this instance, covariates might be issues like earlier medical circumstances or age. By together with covariates within the evaluation, we will additionally be taught if there are any interplay results between the covariates and the therapies.
Beneath is a abstract that compares what we might be taught with a time machine to what we will be taught with experiments.
Now that we perceive the wealthy depth of causal relationships that we will perceive utilizing experimentation, let’s transition to discussing how the number of outcomes underneath an experiment is extra highly effective than a single end result (the one counterfactual) that we might observe with a time-machine run.
Designed experiments quantify the causal relationships; single counterfactuals don’t
Direct remark of a single counterfactual doesn’t give any concept of the power of the overall causal relationship. If I’m going again in time after I obtained sick as soon as to check if the sushi made me sick, I might be taught that it did, or it didn’t trigger my sickness. I nonetheless wouldn’t have any concept of the likelihood that I’ll get sick if I fulfill my sushi craving at a fuel station once more sooner or later! Is it deterministic, i.e., will get sick each single time I eat fuel station sushi? Is it probabilistic, will I get sick fifty p.c of the time? I simply don’t have sufficient data to know.
The experiment we designed within the earlier part wouldn’t solely assist us perceive if fuel station sushi makes folks sick, it could additionally assist quantify the connection. For instance, the experiment would possibly discover that on common, consuming fuel station sushi makes you 5 occasions extra more likely to get sick than restaurant sushi.
Experimental design generalizes higher, and it additionally quantifies the causal relationship higher! If we return in time and take a look at one counterfactual, we will’t know the likelihood of observing the identical end result underneath comparable circumstances, with experimentation we will!
Wrapping it up
My aim in writing this text was to debate why I might nonetheless use experimental design to study causal relationships even when I had a time machine that allowed me to look at counterfactuals.
The principle causes experimental design is healthier is as a result of:
- It generates generalizable causal learnings (versus one particular case)
- It supplies the power of relationships to tell future selections
I hope this thought experiment deepened your understanding of the strengths of experimental design!
