For example, if both Lilliputians and Brobdingnagians apply to Glubbdubdrib University, demographic parity is achieved if the percentage of Lilliputians admitted is the same as the percentage of Brobdingnagians admitted, irrespective of whether one group is on average more qualified than the other.
Contrast with equalized odds and equality of opportunity , which permit classification results in aggregate to depend on sensitive attributes, but do not permit classification results for certain specified ground-truth labels to depend on sensitive attributes.
See "Attacking discrimination with smarter machine learning" for a visualization exploring the tradeoffs when optimizing for demographic parity. Making decisions about people that impact different population subgroups disproportionately. This usually refers to situations where an algorithmic decision-making process harms or benefits some subgroups more than others. If Big-Endian Lilliputians are more likely to have mailing addresses with this postal code than Little-Endian Lilliputians, then this algorithm may result in disparate impact.
Contrast with disparate treatment , which focuses on disparities that result when subgroup characteristics are explicit inputs to an algorithmic decision-making process. Factoring subjects' sensitive attributes into an algorithmic decision-making process such that different subgroups of people are treated differently. Contrast with disparate impact , which focuses on disparities in the societal impacts of algorithmic decisions on subgroups, irrespective of whether those subgroups are inputs to the model.
In other words, equality of opportunity measures whether the people who should qualify for an opportunity are equally likely to do so regardless of their group membership. For example, suppose Glubbdubdrib University admits both Lilliputians and Brobdingnagians to a rigorous mathematics program. Equality of opportunity is satisfied for the preferred label of "admitted" with respect to nationality Lilliputian or Brobdingnagian if qualified students are equally likely to be admitted irrespective of whether they're a Lilliputian or a Brobdingnagian.
For example, let's say Lilliputians and Brobdingnagians apply to Glubbdubdrib University, and admissions decisions are made as follows:. See "Equality of Opportunity in Supervised Learning" for a more detailed discussion of equality of opportunity. Also see "Attacking discrimination with smarter machine learning" for a visualization exploring the tradeoffs when optimizing for equality of opportunity. Lilliputians' secondary schools offer a robust curriculum of math classes, and the vast majority of students are qualified for the university program.
Equalized odds is satisfied provided that no matter whether an applicant is a Lilliputian or a Brobdingnagian, if they are qualified, they are equally as likely to get admitted to the program, and if they are not qualified, they are equally as likely to get rejected. Examples of fairness constraints include:.
Some commonly used fairness metrics include:. Many fairness metrics are mutually exclusive; see incompatibility of fairness metrics. Assuming that what is true for an individual is also true for everyone in that group. The effects of group attribution bias can be exacerbated if a convenience sampling is used for data collection. In a non-representative sample, attributions may be made that do not reflect reality. See also out-group homogeneity bias and in-group bias.
Implicit bias can affect the following:. For example, when building a classifier to identify wedding photos, an engineer may use the presence of a white dress in a photo as a feature. However, white dresses have been customary only during certain eras and in certain cultures. The idea that some notions of fairness are mutually incompatible and cannot be satisfied simultaneously.
The Algorithmic Accountability Act does this, but it puts significant new responsibility for these assessments in the hands of the Federal Trade Commission. A different approach would be to amend the existing anti-discrimination statutes to require disparate impact assessments for automated decision systems used in the contexts covered by these laws.
The assessments should be provided to the appropriate regulatory agency charged with enforcing the anti-discrimination laws and to the public. Each agency could also be assigned the responsibility to conduct its own disparate impact assessment, and to have new authority, if necessary, to obtain data from developers and companies for this purpose.
Agencies could also be authorized to work with outside researchers to conduct these assessments, and to approve certain researchers to receive data from developers and companies to conduct these assessments. Finally, the agencies might be required to work with developers and companies to determine which data might be revealed to the public at large in ways that would not compromise privacy or trade secrets so that independent researchers could conduct their own assessments.
Longer term, an improvement in the accuracy and fairness of algorithmic systems depends on the creation of more adequate datasets, which can only be done through real-world action.
But the creation of new and more adequate data might involve expensive data gathering that would not have a private-sector payoff. It might also involve increasing benefit eligibility under existing algorithmic standards in order to reach qualified members of protected classes. For instance, credit companies might extend loans to members of protected classes who just miss traditional eligibility standards and examine the results to better identify creditworthiness.
There are limits, however, to resolving longstanding disparities through adjustments in algorithms. As many analysts have pointed out, substantive reforms of housing policy, criminal justice, credit allocation, insurance, and employment practices, to name just a few, will be needed to reduce widespread inequities that have persisted far too long.
The promise of automated decision systems—and especially the new machine learning versions—is to dramatically improve the accuracy and fairness of eligibility determination for various private-sector and public-sector benefits. The danger is hidden exacerbation of protected-class disparities. The way forward is to look and see what these systems are doing. Every business manager knows that it is impossible to manage what you do not measure.
So, the first essential step is to measure the extent to which these systems create disparate impacts. This recommendation is for the tradition of disclosure and assessment as the way to improve the operation of organizational systems. Progress in eliminating protected-class disparities first needs awareness. Difficult conversations might lie ahead in determining what to do if these assessments reveal disparities.
But if we do not face them directly, they will only get worse and will be all the more damaging for being done in secret.
The Brookings Institution is a nonprofit organization devoted to independent research and policy solutions. Its mission is to conduct high-quality, independent research and, based on that research, to provide innovative, practical recommendations for policymakers and the public. The conclusions and recommendations of any Brookings publication are solely those of its author s , and do not reflect the views of the Institution, its management, or its other scholars. The findings, interpretations, and conclusions in this report are not influenced by any donation.
Brookings recognizes that the value it provides is in its absolute commitment to quality, independence, and impact. Activities supported by its donors reflect this commitment. Report Produced by Center for Technology Innovation. Facebook has taken steps to stop the use of its targeting mechanisms for discrimination in housing advertising. See also Robert Avery, et al. Introduction Algorithmic or automated decision systems use data and statistical analyses to classify people for the purpose of assessing their eligibility for a benefit or penalty.
West and John R. Related Books. Broadband Edited by Robert W. Crandall and James H. In the case of employment law, this burden shifting paradigm is statutory. HUD v. See Altitude Express Inc. Zarda and Bostock v. Clayton County for sexual orientation, R. Harris Funeral Homes v. Muhammad Ali, et al. For a discussion of the technical ways algorithms and data can introduce bias into a decision-making system, see: Solon Barocas and Andrew D.
For further discussion of these normative issues, see MacCarthy above at footnote For instance, see: Sandra G. TechStream When do consumers prefer algorithmic versus human decisionmakers? Derek E. Bambauer and Michael Risch. Dawson , Kevin C. Desouza , and James S. Post was not sent - check your email addresses! Sorry, your blog cannot share posts by email. For example, women's work histories are more likely to be interrupted by multi-month gaps — maternity leave, for example — and that might count against them in the machine learning system's model.
So, we should be able to adjust the confidence thresholds for men and women independently. If that means we have to tell the machine that to put a man's application into the Approved pile requires a 0.
Demographic parity: No, says Expert 3. The composition of the set of people who are granted loans should reflect the percentage of applicants: if 30 percent of the applicants are women, then 30 percent of the pool of approved applicants ought to be women. Equal opportunity: Expert 4 disagrees because if you just want equal proportions of men and women to get loans, you could randomly offer loans to women whether or not the individuals are good risks.
What you don't want is for 90 percent of the male good risks to make it into the acceptance pile, but only 40 percent of the female good risks to do so. Expert 4 goes on to argue that that meets the lender's objective of identifying loan-worthy applicants, but avoids favoring one gender over another in terms of risk.
Equal accuracy: Expert 5 thinks Expert 4 was on the right track, but instead of making sure that the genders end up with the same percentage of successes i. That is, the percentage of right classifications as loan-worthy or as not should be the same for both genders. If we use the Equal Opportunity approach espoused by Expert 4, the system will be wrong about who it approves — they don't pay back their loans — the same percentage of the time regardless of gender.
But suppose it turns out that among the people the system has rejected, it's wrong about women twice as often as it's wrong about men. In such a case, more women than men who deserved loans are being denied. So, argues 5, the system ought to be tuned so that the percentage of times it's wrong in the total of approvals and denials is the same for both genders.
0コメント