Counterfactual : the Effect of Education and Experience on Salary

Creative Commons License

aGrUM

interactive online version

This notebook is a ‘probabilistic’ (or noisy) version following the example from “The Book Of Why” (Pearl, 2018) chapter 8 page 251 (see the notebooks BoW-c8p* below).

Counterfactuals

In [1]:
from IPython.display import display, Math, Latex,HTML

import pyAgrum as gum
import pyAgrum.lib.notebook as gnb
import pyAgrum.causal as csl
import pyAgrum.causal.notebook as cslnb
import os
import math
import numpy as np
import scipy.stats


In this example we are interested in the effect of experience and education on the salary of an employee, we are in possession of the following data:

Employé

EX(u)

ED(u)

\(S_{0}(u)\)

\(S_{1}(u)\)

\(S_{2}(u)\)

Alice

8

0

86,000

?

?

Bert

9

1

?

92,500

?

Caroline

9

2

?

?

97,000

David

8

1

?

91,000

?

Ernest

12

1

?

100,000

?

Frances

13

0

97,000

?

?

etc

  • \(EX(u)\) : years of experience of employee \(u\). [0,20]

  • \(ED(u)\) : Level of education of employee \(u\) (0:high school degree (low), 1:college degree (medium), 2:graduate degree (high)) [0,2]

  • \(S_{i}(u)\) [65k,150k] :

    • salary (observable) of employee \(u\) if \(i = ED(u)\),

    • Potential outcome (unobservable) if \(i \not = ED(u)\), salary of employee \(u\) if he had a level of education of \(i\).

We are left with the previous data and we want to answer the counterfactual question What would Alice’s salary be if she attended college ? (i.e. \(S_{1}(Alice)\))

We create the causal diagram

In this model it is assumed that an employee’s salary is determined by his level of education and his experience. Years of experience are also affected by the level of education. Having a higher level of education means spending more time studying hence less experience.

In [2]:
edex = gum.fastBN("Ux[-2,10]->experience[0,20]<-education[0,2]->salary[65,150];"
                  "experience->salary<-Us[0,25]")
edex
Out[2]:
G education education salary salary education->salary experience experience education->experience Us Us Us->salary Ux Ux Ux->experience experience->salary

However counterfactual queries are specific to one datapoint (in our case Alice), we need to add additional variables to our model to allow for individual variations: * Us : unobserved variables that affect salary.[0,25k] * Ux : unobserved variables that affect experience.[-2,10]

In [3]:
# no prior information about the individual (datapoint)
edex.cpt("Us").fillWith(1).normalize()
edex.cpt("Ux").fillWith(1).normalize()
# education level(supposed)
edex.cpt("education")[:] = [0.4, 0.4, 0.2]
In [4]:
# To have probabilistic results, we add a perturbation. (Gaussian around the exact values)
# we calculate a gaussian distribution

std = 1

Experience listens to Education and Ux :

\[Ex = 10 -4 \times Ed + Ux\]
In [5]:
edex.cpt("experience").fillFromDistribution(scipy.stats.norm,loc="10-4*education+Ux",scale=std)
edex.cpt("experience")
Out[5]:
experience
education
Ux
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
0
-2
0.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.00000.00000.0000
-1
0.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.00000.0000
0
0.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.0000
1
0.00000.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.0000
2
0.00000.00000.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.0000
3
0.00000.00000.00000.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.0000
4
0.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.0000
5
0.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.0000
6
0.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.0001
7
0.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39900.24200.05400.0044
8
0.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00010.00450.05420.24310.40080.24310.0542
9
0.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00010.00470.05730.25700.42380.2570
10
0.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00020.00630.07720.34590.5703
1
-2
0.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.0000
-1
0.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.0000
0
0.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.00000.00000.00000.00000.0000
1
0.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.00000.00000.00000.0000
2
0.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.00000.00000.0000
3
0.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.00000.0000
4
0.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.0000
5
0.00000.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.0000
6
0.00000.00000.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.0000
7
0.00000.00000.00000.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.0000
8
0.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.0000
9
0.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.0000
10
0.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.0001
2
-2
0.57030.34590.07720.00630.00020.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.0000
-1
0.25700.42380.25700.05730.00470.00010.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.0000
0
0.05420.24310.40080.24310.05420.00450.00010.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.0000
1
0.00440.05400.24200.39900.24200.05400.00440.00010.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.0000
2
0.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.0000
3
0.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.00000.00000.00000.00000.00000.0000
4
0.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.00000.00000.00000.00000.0000
5
0.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.00000.00000.00000.0000
6
0.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.00000.00000.0000
7
0.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.00000.0000
8
0.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.00000.0000
9
0.00000.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.00000.0000
10
0.00000.00000.00000.00000.00000.00000.00000.00000.00010.00440.05400.24200.39890.24200.05400.00440.00010.00000.00000.00000.0000

Salary listens to Education, Experience and Us :

\[S = 65 + 2.5 \times Ex + 5 \times Ed + Us\]
In [6]:
edex.cpt("salary").fillFromDistribution(scipy.stats.norm,loc="65+2.51*experience+5*education+Us",scale=std)
gnb.showInference(edex)
../_images/notebooks_65-Causality_Counterfactual_17_0.svg

Our question was : What would Alice’s salary be if she attended college ?

To answer this counterfactual question we will follow the three steps algorithm from “The Book Of Why” (Pearl 2018) chapter 8 page 253 :

Step 1 : Abduction

Use the data to retrieve all the information that characterizes Alice

From the data we can retrieve Alice’s profile : * \(Ed(Alice)\) : 0 * \(Ex(Alice)\) : 8 * \(S_{0}(Alice)\) : 86k

We will use Alice’s profile to get \(U_s\) and \(U_x\), which tell Alice apart from the rest of the data.

In [7]:
ie=gum.LazyPropagation(edex)
ie.setEvidence({'education': 0, 'salary' : "86", "experience":8})
ie.makeInference()
newUs = ie.posterior("Us")
gnb.showProba(newUs)
../_images/notebooks_65-Causality_Counterfactual_21_0.svg
In [8]:
ie=gum.LazyPropagation(edex)
ie.setEvidence({'education': 0, 'salary' : "86", "experience":8})
ie.makeInference()
newUx = ie.posterior("Ux")
gnb.showProba(newUx)
../_images/notebooks_65-Causality_Counterfactual_22_0.svg
In [9]:
gnb.showInference(edex,evs={'education': 0, 'salary' : "86", "experience":8},targets={'Ux','Us'})
../_images/notebooks_65-Causality_Counterfactual_23_0.svg

Step 2 & 3 : Action And Prediction

Change the model to match the hypothesis implied by the query (if she had attended university) and then use the data that characterizes Alice to calculate her salary.

We create a counterfactual world with Alice’s idiosyncratic factors, and we operate the intervention:

In [10]:
# the counterfactual world
edexCounterfactual = gum.BayesNet(edex)
In [11]:
# we replace the prior probabilities of idiosynatric factors with potentials calculated earlier
edexCounterfactual.cpt("Ux").fillWith(newUx)
edexCounterfactual.cpt("Us").fillWith(newUs)
gnb.showInference(edexCounterfactual,size="10")
print("counterfactual world created")
../_images/notebooks_65-Causality_Counterfactual_27_0.svg
counterfactual world created
In [12]:
# We operate the intervention
edexModele = csl.CausalModel(edexCounterfactual)
cslnb.showCausalImpact(edexModele,"salary",
                       doing="education",
                       values={"education":"1"})
G Ux Ux experience experience Ux->experience salary salary experience->salary education education education->experience education->salary Us Us Us->salary
Causal Model
$$\begin{equation*}P( salary \mid \text{do}(education)) = \sum_{Us,Ux,experience}{P\left(Us\right) \cdot P\left(salary\mid Us,education,experience\right) \cdot P\left(experience\mid Ux,education\right) \cdot P\left(Ux\right)}\end{equation*}$$
Explanation : Do-calculus computations

Impact

Alice’s salary if she attended college is lower than her actual salary, that’s because in the counterfactual world where she attended college she had less time to work hence her diminished salary.

We can prove it perfoming a complete inference in the counterfactual world. Since education has no parents in our model (no graph surgery, no causes to emancipate it from), an intervention is equivalent to an observation, the only thing we need to do is to set the value of education:

In [13]:
gnb.showInference(edexCounterfactual,targets={"salary",'experience'},evs={'education':"1"},size="10")
../_images/notebooks_65-Causality_Counterfactual_30_0.svg

Indeed the expected “experience” decreased.

The result (salary if she had attended college) is given by the formaula:

\[\sum_{salary} salary \times P(salary^* \mid RealSalary = 86k, education = 0, experience = 8, education^*=1)\]

Where variables marked with an asterisk are inobservable.

In [14]:
formula, adj, exp = csl.causalImpact(edexModele,"salary",
                                     doing="education",
                                     values={"education":"1"})
gnb.showProba(adj)
../_images/notebooks_65-Causality_Counterfactual_32_0.svg
In [15]:
salary=edexModele.observationalBN()["salary"]
adj.expectedValue(lambda v : salary.numerical(v["salary"]))
Out[15]:
82.41408794898389
\[S_1(Alice) = 82k\]

Alice’s salary would be \(\$82.4\) if she had attended college !

pyAgrum.causal.counterfactual

In pyAgrum, we can directly use a function that answers counterfactual queries using the previous algorithm.

In [16]:
help(csl.counterfactual)
Help on function counterfactual in module pyAgrum.causal._causalImpact:

counterfactual(cm: pyAgrum.causal._CausalModel.CausalModel, profile: Optional[Dict[str, int]], on: Union[str, Set[str]], whatif: Union[str, Set[str]], values: Optional[Dict[str, int]] = None) -> 'pyAgrum.Potential'
    Determines the estimation of a counterfactual query following the the three steps algorithm from "The Book Of Why"
    (Pearl 2018) chapter 8 page 253.

    Determines the estimation of the counterfactual query: Given the "profile" (dictionary <variable name>:<value>),what
    would variables in "on" (single or list of variables) be if variables in "whatif" (single or list of variables) had
    been as specified in "values" (dictionary <variable name>:<value>)(optional).

    This is done according to the following algorithm:
        -Step 1-2: compute the twin causal model
        -Step 3 : determine the causal impact of the interventions specified in  "whatif" on the single or list of
        variables "on" in the causal model.

    This function returns the potential calculated in step 3, representing the probability distribution of  "on" given
    the interventions  "whatif", if it had been as specified in "values" (if "values" is omitted, every possible value of
    "whatif")

    Parameters
    ----------
    cm: CausalModel
    profile: Dict[str,int] default=None
      evidence
    on: variable name or variable names set
     the variable(s) of interest
    whatif: str|Set[str]
      idiosyncratic nodes
    values: Dict[str,int]
      values for certain variables in whatif.

    Returns
    -------
    pyAgrum.Potential
      the computed counterfactual impact

Let’s try with the previous query :

In [17]:
cm_edex= csl.CausalModel(edex)
pot=csl.counterfactual(cm =cm_edex,
                       profile = {'education': 0, 'experience':8, 'salary' : "86"},
                       whatif={"education"},
                       on={"salary"},
                       values = {"education" : 1})
In [18]:
gnb.showProba(pot)
../_images/notebooks_65-Causality_Counterfactual_39_0.svg

We get the same result !

If we omit values:

We get every potential outcome :

In [19]:
pot=csl.counterfactual(cm =cm_edex,
                       profile = {'experience':8, 'education': '0', 'salary' : '86'},
                       whatif={"education"},
                       on={"salary"})
In [20]:
# pot contains the result for all value of education
for label in pot.variable("education").labels():
    gnb.flow.row(f"for education = {label}",gnb.getProba(pot.extract({"education":label})))
for education = 0
for education = 1
for education = 2

What would Alice’s salary be if she had attended college and had 8 years of experience ?

In [21]:
pot=csl.counterfactual(cm =cm_edex,
                       profile = {'experience':8, 'education': 0, 'salary' : '86'},
                       whatif={"education", "experience"},
                       on={"salary"},
                       values = {"education" : 1, "experience" : 8})
In [22]:
gnb.showProba(pot)
../_images/notebooks_65-Causality_Counterfactual_47_0.svg

if she attended college and had 8 years of experience Alice’s salary would be 91k !

In the previous query, Alice’s salary if she attended college was lower than her actual salary, that’s because in the counterfactual world where she attended college she had less time to work hence her diminished salary.

In this query, Alice’s counterfactual salary was higher than her actual salary (+5k corresponding to one level of education), that’s because in the counterfactual world Alice attended college and still had time to work 8 years, so her salary went up.

if she had more experience :

Some counterfactual can not be computer : With this profile, an experience of 10 is nont possible…

In [23]:
pot=csl.counterfactual(cm =cm_edex,
                       profile = {'experience':8, 'education': 0, 'salary' : '86'},
                       whatif={"experience"},
                       on={"salary"},
                       values = {"experience" : 10})
pot
Out[23]:
salary
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
nannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannannan

indeed experience can not be 12

In [24]:
twin=csl.counterfactualModel(cm = csl.CausalModel(edex),
                       profile = {'experience':8, 'education': 0, 'salary' : '86'},
                       whatif={"experience"})
gnb.showInference(twin.observationalBN(),size="10",evs={'education': 0, 'salary' : '86'})
../_images/notebooks_65-Causality_Counterfactual_52_0.svg

## CounterfactualS as a function

We can now fill the holes in :

Employé

EX(u)

ED(u)

\(S_{0}(u)\)

\(S_{1}(u)\)

\(S_{2}(u)\)

Alice

8

0

86,000

?

?

Bert

9

1

?

92,500

?

Caroline

9

2

?

?

97,000

David

8

1

?

91,000

?

Ernest

12

1

?

100,000

?

Frances

13

0

97,000

?

?

etc

In [25]:
def mean(p):
    return sum([p.variable(0).numerical(i)*p[i] for i in range(p.variable(0).domainSize())])
def affCounterfactualForStudent(model,name,ex,ed,sa,value):
    try:
        s0=csl.counterfactual(cm = model,
                              profile = {'experience':str(ex), 'education': ed, 'salary' : str(sa)},
                              whatif={"education"},
                              on={"salary"},
                              values = {"education" : value})
        print("{:5.1f}| ".format(mean(s0)),end="")
    except:
        print(" --  | ",end="")
def forStudent(model,name,ex,ed,sa):
    print("| {:20}| {:2.0f}| s{:1}|{:3.0f}|    | ".format(name,ex,ed,sa),end="")
    for value in range(3):
        affCounterfactualForStudent(model,name,ex,ed,sa,value)
    print()

print("| Name                | Ex|Ed | S |    |  s0  |  s1   |  s2 |")
print("-----------------------------------    ----------------------")
d=csl.CausalModel(edex)
forStudent(d,"Alice",8,0,86)
forStudent(d,"Bert",9,1,92)
forStudent(d,"Caroline",9,2,97)
forStudent(d,"David",8,1,91)
forStudent(d,"Ernest",12,1,100)
forStudent(d,"Frances",13,0,97)
| Name                | Ex|Ed | S |    |  s0  |  s1   |  s2 |
-----------------------------------    ----------------------
| Alice               |  8| s0| 86|    |  87.5|  82.4|  78.2|
| Bert                |  9| s1| 92|    |  97.9|  92.9|  87.9|
| Caroline            |  9| s2| 97|    | 107.9| 102.9|  97.9|
| David               |  8| s1| 91|    |  96.2|  91.1|  86.1|
| Ernest              | 12| s1|100|    | 105.6| 100.6|  95.5|
| Frances             | 13| s0| 97|    |  97.9|  92.9|  87.8|

Note that, contraty to the notebook ‘BoW-c8p251*’, there is no equality between the input salary (86 for Alice) and the expected counterfactual if Alice had this salary (For alice, \(s0=87.5\)). Of course, this is due to noise that we introduced in the model.

Note also that this “noisy” version allow to answer that can not be answered in the deterministic version in ‘BoW-c8p251*’.

In [ ]:

In [ ]: