{"id":4135,"date":"2022-11-22T18:07:37","date_gmt":"2022-11-22T18:07:37","guid":{"rendered":"https:\/\/2024.thenexus.today\/index.php\/2022\/11\/22\/consent-automated-systems-and-discrimination\/"},"modified":"2024-01-20T05:23:39","modified_gmt":"2024-01-20T05:23:39","slug":"consent-automated-systems-and-discrimination","status":"publish","type":"post","link":"https:\/\/2024.thenexus.today\/index.php\/2022\/11\/22\/consent-automated-systems-and-discrimination\/","title":{"rendered":"Consent, Automated Systems, and Discrimination (FTC Comments)"},"content":{"rendered":"

Submitted to regulations.gov a couple of hours before last night’s deadline. \u00a0An earlier version of points 2-4 appeared in the extended remix of my public comments<\/a> in September.<\/em><\/p>\n

Thank you for your attention to the pressing issues of commercial surveillance and data security. \u00a0As the author of the Nexus of Privacy Newsletter<\/a>, I write about commercial surveillance and other connections between technology, policy, and justice. \u00a0My career includes founding a successful software engineering startup<\/a>; General Manager of Competitive Strategy at Microsoft; and co-chairing the ACM Computers, Freedom, and Privacy Conference<\/a>. In fall 2021, I was a member of the Washington state Automated Decision-making Systems Workgroup<\/a>, and I have testified about privacy legislation in over a dozen Washington state legislature hearings.<\/p>\n

My comments focus primarily on consent, discrimination, algorithmic error, and automated systems, although also touch on questions related to data minimization and other aspects of privacy. \u00a0In summary:<\/p>\n

    \n
  1. Consent is a vital complement to data minimization and completely prohibiting some commercial surveillance activities. \u00a0Opt-out is not meaningful affirmative consent, and an opt-in approach to regulation will enhance innovation. (Questions 26, 73-81)<\/strong><\/strong><\/strong><\/li>\n
  2. Algorithmic error and discrimination is pervasive across multiple sectors \u2013 and the harms fail disproportionately on the most vulnerable people. (Questions 53, 57, 65, 66, 67)<\/strong><\/strong><\/strong><\/li>\n
  3. The FTC should build on the recommendations of Algorithmic Justice League\u2019s <\/strong>Who Audits the Auditors<\/strong><\/a>, the White House OSTP\u2019s <\/strong>Blueprint for an AI Bill of Rights<\/strong><\/a>, and the <\/strong>California Privacy Protection Agency\u2019s AI equity work<\/strong><\/a>. (Questions 41-46, 56, 67)<\/strong><\/strong><\/strong><\/li>\n
  4. The FTC should develop its regulations working with the people most likely to be harmed by commercial surveillance \u2013 and prioritize their needs. (Questions 29, 39, 43)<\/strong><\/strong><\/strong><\/li>\n<\/ol>\n
    \"A
    Malkia Cyril, at Personal Democracy Forum 2017, originally tweeted by @anxiaostudio<\/figcaption><\/figure>\n

    2. Algorithmic error and discrimination is pervasive across multiple sectors \u2013 and the harms fail disproportionately on the most vulnerable people (Questions 53, 57, 65, 66, and 67)<\/h3>\n

    Technology can be liberating \u2026 but it can also reinforce and magnify existing power imbalances and patterns of discrimination. \u00a0As documented by researchers like Dr. Safiya Noble in Algorithms of Oppression<\/em> and Dr. Joy Buolamwini, Timnit Gebru, and Inioluwa Deborah Raji in the Gender Shades <\/em>project, today\u2019s algorithmic systems are error-prone and the datasets they\u2019re trained on have significant biases.<\/p>\n

    These errors \u2013 and the discrimination these systems cause \u2013 are prevalent across all sectors. For example, Color of Change’s Black Tech Agenda<\/a> talks about \u201cthe harm that algorithmic has done to Black communities regarding equitable access to housing, health care, employment, education, credit, and insurance.” \u00a0Similarly, facial recognition errors led to the arresting innocent Black people like Nijeer Parks, Robert WIlliams, and Michael Oliver<\/a>; and Algorithms that detect hate speech online are biased against Black people<\/a>. \u00a0Of course, it is not only Black communities that are harmed by this \u2013 \u00a0health care allocation algorithms discriminate against disabled people<\/a>, automated risk assessments discriminate against Hispanic people<\/a>, fraud-detection systems harm against unemployed people<\/a>, \u2026 the list goes on.<\/p>\n

    And as this list indicates, the harms are usually much greater on the historically underserved communities the FTC has committed to protecting.<\/p>\n

    Regulation is clearly needed, and it needs to be designed to protect the people who are most at risk. \u00a0However, the traditional list of \u201cprotected classes\u201d is not sufficient. Some algorithmic discrimination is intersectional \u2013 for example, Gender Shades and follow-on work highlight that facial recognition is even more inaccurate for Black women. \u00a0And, other forms of algorithmic discrimination can disproportionately harm groups like unemployed people or renters<\/a> who are not considered \u201cprotected classes\u201d.<\/p>\n

    3. The FTC should build on the recommendations of Algorithmic Justice League\u2019s Who Audits the Auditors<\/a>, the White House OSTP\u2019s Blueprint for an AI Bill of Rights<\/a>, and the California Privacy Protection Agency\u2019s AI equity work<\/a> (Questions 41-46, 56, 67)<\/h3>\n
    \"A<\/figure>\n

    There are a lot of challenges in regulating algorithms and automated decision systems. \u00a0Fortunately, there is a lot of excellent work out there for the FTC to build on.<\/p>\n

    The Algorithmic Justice League\u2019s founder Joy Buolamwini, research collaborator Inioluwa Deborah Raji, and Director Of Research & Design Sasha Costanza-Chock (known for their work on Design Justice) recently collaborated on Who audits the Auditors: Recommendations from a field scan of the algorithmic auditing ecosyystem<\/a>, the first comprehensive field scan of the artificial intelligence audit ecosystem.<\/p>\n

    The policy recommendations in Who Audits the Auditors<\/em> highlight key considerations in several specific areas where FTC rulemaking could have a major impact.<\/p>\n

      \n
    1. Require the owners and operators of AI systems to engage in independent algorithmic audits against clearly defined standards<\/li>\n
    2. Notify individuals when they are subject to algorithmic decision-making systems<\/li>\n
    3. Mandate disclosure of key components of audit findings for peer review<\/li>\n
    4. Consider real-world harm in the audit process, including through standardized harm incident reporting and response mechanisms<\/li>\n
    5. Directly involve the stakeholders most likely to be harmed by AI systems
      in the algorithmic audit process<\/li>\n
    6. Formalize evaluation and, potentially, accreditation of algorithmic auditors.<\/li>\n<\/ol>\n

      The Blueprint for an AI Bill of Rights<\/a> announced in September by the White House Office of Science and Technology Policy (OSTP) is also very valuable. \u00a0The detailed recommendations in Algorithmic Discrimination Protections<\/a>, Data Privacy<\/a> and Notice and Explanation<\/a> sections relate directly to many of the questions in the ANPR.<\/p>\n

      And the details of the rulemaking matter a lot. \u00a0Too often, well-intended regulation has weaknesses that commercial surveillance companies, with their hundreds of lawyers, can easily exploit. \u00a0Looking at proposals through an algorithmic justice lens can highlight where they fall short.<\/p>\n

      For example, the Who Audits the Auditors<\/em> recommendations highlight ways that proposed American Data Privacy and Protection Act (ADPPA) consumer privacy bill falls short of effective regulations:<\/p>\n

        \n
      1. ADPPA doesn’t require independent auditing, instead allowing companies like Facebook to do their own algorithmic impact assessments. \u00a0And government contractors acting as service providers for ICE and law enforcement don’t even have to do algorithmic impact assessments! \u00a0As Color of Change’s Black Tech Agenda<\/a> notes “By forcing companies to undergo independent audits, tech companies can address discrimination in their decision-making and repair the harm algorithmic that bias has done to Black communities.”<\/li>\n
      2. ADPPA doesn\u2019t require affirmative consent to being profiled \u2013 or even offer the opportunity to opt out.<\/li>\n
      3. ADPPA doesn\u2019t mandate any public disclosure of its algorithmic impact assessments \u2013 not even summaries or key components<\/li>\n
      4. ADPPA doesn’t have any requirement for including real-world harms \u2013 or even measuring the impact.<\/li>\n
      5. ADPPA doesn’t have any requirement at all to involve external stakeholders in the assessment process \u2013 let alone directly involving the stakeholders most likely to be harmed by AI system.<\/li>\n
      6. ADPPA allows anybody at a company to do an algorithmic impact assessment \u2013 and it’s not even clear whether it allows the FTC rulemaking authority for potential evaluation and accreditation of assessors or auditors<\/li>\n<\/ol>\n

        Taking all of this valuable work into account starting early in the process will lead to more effective regulations.<\/p>\n

        4. The FTC should develop its regulations working with the people most likely to be harmed by commercial surveillance \u2013 and prioritize their needs (Questions 29, 39, 43)<\/h3>\n

        As Afsenah Rigot points out in Design From the Margins<\/a>, a design process that centers the most impacted and marginalized users from ideation to production results in outcomes that are highly beneficial for all users and companies. \u00a0While Rigot focuses on product design, and AJL\u2019s recommendations make a similar point about involving the stakeholders most likely to be harmed by AI systems in an auditing process, For example, A. Prince Albert III Hiding OUT: A Case for Queer Experiences Informing Data Privacy Laws<\/a>, suggests using queer experiences as an analytical tool to test whether proposed a privacy regulation protects people\u2019s privacy and \u00a0Stress-testing privacy legislation with a queer lens<\/a> illustrates the insights from this approach.<\/p>\n

        So as the FTC develops these regulations, it\u2019s critical to involve the people who will be most impacted \u2013 and to make sure their needs are prioritized. \u00a0As well as the protected classes, it’s also vital to look at how proposed regulations affect pregnant people, rape and incest survivors, immigrants, unhoused people, and others who are being harmed today<\/em> by commercial surveillance. \u00a0Regulations that protect them will wind up protecting everybody.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

        Submitted to regulations.gov a couple of hours before last night’s deadline. \u00a0An earlier version of points 2-4 appeared in the extended remix of my public comments in September. Thank you for your attention to the pressing issues of commercial surveillance and data security. \u00a0As the author of the Nexus of Privacy Newsletter, I write about […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[421,16,1],"tags":[462,463,464,468],"class_list":["post-4135","post","type-post","status-publish","format-standard","hentry","category-software","category-tales-from-the-net","category-uncategorized","tag-algorithmic-justice","tag-automated-decision-systems","tag-ftc","tag-public-comment"],"_links":{"self":[{"href":"https:\/\/2024.thenexus.today\/index.php\/wp-json\/wp\/v2\/posts\/4135","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/2024.thenexus.today\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/2024.thenexus.today\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/2024.thenexus.today\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/2024.thenexus.today\/index.php\/wp-json\/wp\/v2\/comments?post=4135"}],"version-history":[{"count":1,"href":"https:\/\/2024.thenexus.today\/index.php\/wp-json\/wp\/v2\/posts\/4135\/revisions"}],"predecessor-version":[{"id":4343,"href":"https:\/\/2024.thenexus.today\/index.php\/wp-json\/wp\/v2\/posts\/4135\/revisions\/4343"}],"wp:attachment":[{"href":"https:\/\/2024.thenexus.today\/index.php\/wp-json\/wp\/v2\/media?parent=4135"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/2024.thenexus.today\/index.php\/wp-json\/wp\/v2\/categories?post=4135"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/2024.thenexus.today\/index.php\/wp-json\/wp\/v2\/tags?post=4135"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}