Regulating Algorithmic Discrimination in AI-Assisted Recruitment

Lora Yousufziy, AI For Good Summit 2026

Executive Summary

Artificial Intelligence (AI) recruitment technologies are increasingly used across Australia to screen and assess job applicants. While these technologies may reduce administrative burden and improve efficiency, they risk reproducing labour-market inequalities through proxy discrimination. Automated hiring systems may rely on variables that indirectly correlate with protected attributes including race, sex, disability and age. Their opacity often prevents affected applicants from identifying how decisions were made or challenging potentially discriminatory outcomes. As their use expands, these systems may reinforce existing structural disadvantages, limit workforce participation and undermine confidence in fair hiring.

This paper recommends amending the Fair Work Act 2009 (Cth) to mandate algorithmic audits and representative data for automated recruitment decision-making systems. Employers using these systems at scale would be required to conduct regular audits to identify discriminatory outcomes, take reasonable steps to ensure data representativeness and retain documentation for regulatory oversight. Key barriers include limited technical expertise and limited transparency of third-party systems, while broader risks include increased compliance costs for employers and the potential for superficial compliance practices. Despite these challenges, this reform aims to adopt a preventative approach aimed at reducing discriminatory outcomes before harm occurs.  

Problem Identification

AI is increasingly used in recruitment processes to screen, rank and assess job applicants, reducing administrative burden and managing high application volumes through automated early-stage decisions. Employers commonly use automated resume screening and video-interview assessment tools to predict role suitability by scoring candidates against predefined criteria and ranking their likely performance (Sony et al. 2025, p.2). In 2025, 61 per cent of Australian organisations reported having used AI in recruitment ‘moderately’ or ‘extensively’ in 2025 (Responsible AI Index 2025, p.58).

These systems are typically machine-learning models trained on historical employment data to predict candidate suitability or performance. However, they may disadvantage groups such as women, racial minorities and people with disabilities (Sheard 2025, p.2). In practice, these systems may rely on variables (such as postcode, employment gaps, education history or speech patterns) that function as proxies for protected attributes under Australian discrimination law, including race, sex, disability and age (Australian Human Rights Commission, 2020 p.32). This creates proxy discrimination, systematically disadvantaging protected groups without explicitly using those attributes. Although Australia’s anti-discrimination framework prohibits discrimination, it requires complainants to identify the specific ‘condition, requirement or practice’ responsible for the disadvantage (e.g. Sex Discrimination Act 1984 (Cth) s 5(2)). The opaque nature of AI screening systems means affected applicants often cannot identify how decisions were made or why they were excluded from recruitment processes (Sheard 2025, p.4). 

If proxy discrimination in AI hiring systems is left unaddressed, these technologies risk entrenching existing labour market inequalities by systematically excluding already disadvantaged groups from employment opportunities. This would affect individuals with protected attributes by limiting income, career progression and financial independence. At a broader level, it may increase long-term economic marginalisation, deepen reliance on income support and reinforce occupational segregation across industries. It also risks undermining confidence in the legitimacy of recruitment processes, particularly where individuals are unable to understand or challenge decisions that significantly affect their economic opportunities.

Context

Background

AI hiring systems increasingly use machine-learning techniques to screen, rank and assess job applicants using recruitment and workplace data (Sheard, 2025, p.278). Their growing use raises concerns that automated recruitment processes may reinforce existing labour-market inequalities through algorithmic discrimination.

A central cause of algorithmic discrimination is the reliance on historical recruitment data that may already reflect inequalities or discriminatory hiring practices. Machine-learning systems identify patterns associated with previous recruitment outcomes and apply those patterns to future hiring decisions, creating a risk that discriminatory practices become embedded within automated systems (Sheard, 2025, pp.278-279). 

This issue was illustrated by Amazon’s AI recruitment tool, which disadvantaged female applicants after being trained on historical hiring data from a male-dominated technology sector (Cofone, 2019, p.1397). The system subsequently penalised resumes containing indicators associated with women, including references to women’s colleges or organisations (Cofone, 2019, p.1398). 

Discrimination may also occur through proxy variables. Seemingly neutral factors such as postcodes, employment gaps, accents and education history may indirectly correlate with race, gender, disability, or socioeconomic status (Yanisky-Ravid and Hallisey, 2019, p.449). Similarly, automated video interviews and speech-recognition technologies may disadvantage candidates with disabilities or non-native English speakers (Sheard, 2025, p.285).

The opacity of AI systems further complicates these risks. Applicants are often unaware that automated systems have been used to assess them, what data has been relied upon, or why they were rejected (Zuiderveen Borgesius, 2018, p.10). Employers may also restrict access to algorithmic systems and training data through intellectual property protections and trade secrecy, limiting the ability of applicants and regulators to identify discriminatory outcomes.

Australian Policy Landscape

Australia’s federal anti-discrimination framework consists primarily of the Sex Discrimination Act 1984 (Cth), Racial Discrimination Act 1975 (Cth), Disability Discrimination Act 1992 (Cth) and Age Discrimination Act 2004 (Cth). These statutes prohibit discrimination, with the Australian Human Rights Commission (AHRC) responsible for administering federal anti-discrimination law. However, these  were developed primarily to regulate human decision-making, rather than opaque algorithmic systems. The Privacy and Other Legislation Amendment Act 2024 (Cth) amended the Privacy Act 1988 (Cth) to require entities subject to the Australian Privacy Principles to include information in their privacy policies where personal information is used by a program to make decisions that may significantly affect an individual's rights or interests. However, this is a transparency reform only. While it requires APP entities to disclose that automated decision-making is being used, it does not regulate whether the system is fair, whether its data is representative, or whether it produces biased outcomes. The reform therefore improves transparency for applicants, but does not directly address the risk of discriminatory outcomes in AI-assisted recruitment.

Case Studies

International regulators have increasingly recognised the limitations of existing anti-discrimination frameworks in addressing algorithmic discrimination. For example, New York’s Local Law 144 imposes regulatory obligations on employers using automated hiring systems by requiring recent independent bias audits, public transparency regarding audit outcomes, and notification to candidates subjected to automated assessment processes (New York Department of Consumer and Worker Protection. (n.d.). 

Policy Options

An effective policy would ensure that AI-hiring systems used in Australia do not produce discriminatory outcomes against protected groups, particularly where discrimination occurs through proxy variables rather than explicit protected attributes. Success would be indicated by a measurable reduction in system features that contribute to unjustified disparities in hiring outcomes. This would be assessed through regulatory monitoring and strengthened transparency requirements. 

Option 1: Shift the evidentiary burden where AI systems are used

This option would amend federal anti-discrimination legislation to create a rebuttable presumption of discrimination where an employer relies on an automated decision-making system in recruitment and a complainant establishes a reasonable inference that the process may have disadvantaged them on the basis of a protected attribute. In practice, this would shift the evidentiary burden to employers to demonstrate that the AI system does not produce discriminatory outcomes and that reasonable steps were taken to prevent bias in its design. This reform would address the current barrier where job seekers are required to prove discrimination despite lacking access to the algorithmic systems or training data used in hiring decisions. 

Policy development and legislative drafting would be led by the Attorney-General’s Department, which administers federal anti-discrimination legislation. Complaint handling would continue through the AHRC. The AHRC would not receive new enforcement powers under this reform but would apply the amended evidentiary standard when investigating and conciliating discrimination complaints involving AI-assisted hiring systems.

Implementation costs are estimated at approximately $2-3 million, which is consistent with comparable discrimination law reforms. For example, the Australian Government allocated $5.8 million over four years to support education and compliance activities associated with introducing the positive duty on employers through the Respect@Work reforms (Australian Government, Budget 2022-23, p.55). 

The primary advantage of this option is that it addresses the key structural barrier in existing discrimination law by correcting the evidentiary imbalance faced by job applicants. Due to the opaque nature of AI-hiring systems, applicants are rarely able to access the algorithmic logic or training data necessary to prove discriminatory outcomes. By shifting the evidentiary burden to employers once a disproportionate impact is demonstrated, the reform would improve access to justice. A potential limitation is that employers may face greater legal uncertainty during the early stages of implementation while regulators develop guidance on how the presumption of discrimination should operate in practice. 

Option 2: Prohibit the use of proxy variables

This option would amend federal anti-discrimination legislation to explicitly recognise proxy discrimination in algorithmic decision-making systems. The legislation would prohibit employers from relying on variables in automated hiring systems that function as substitutes for protected attributes and are not reasonably necessary for the role. In practice, this would target the mechanism through which AI discrimination often occurs, where variables such as postcode, education history or employment gaps function as indirect indicators of race or socioeconomic status. Unlike  Option 1, this would not alter the evidentiary burden. Complainants would still need to establish that a proxy variable was used and that it produced discriminatory disadvantage. However, the reform would make the legal standard clearer by expressly prohibiting the use of proxy indicators where they are not reasonably necessary for the role. As outlined in Option 1, policy development and legislative amendments would be led by the Attorney-General’s Department, with complaint handling through the AHRC. 

Implementation costs are estimated at approximately $1-2 million, which is lower than Option 1 because this reform primarily involves clarifying the scope of existing discrimination provisions rather than introducing new procedural obligations for employers. The main advantage of this option is that it directly targets the technical mechanism through which discrimination often occurs in AI hiring systems. However, a key limitation is that identifying the proxy variable would still largely fall on the complainant in the first instance. The AHRC may investigate the alleged proxy during complaint handling, but this would be technically difficult where algorithms rely on large numbers of interacting features or are developed by third-party vendors. Effective enforcement may therefore require regulators to rely on technical expertise to determine whether a particular variable operates as a proxy for a protected attribute. 

Option 3: Require mandatory algorithmic audits and representative training data for AI hiring systems

This option would amend the Fair Work Act 2009 (Cth) (FWA) to require employers using AI-assisted hiring systems to conduct regular algorithmic audits to assess whether their systems produce discriminatory outcomes against protected groups. Employers would also be required to demonstrate that training datasets used to develop or deploy AI hiring tools are sufficiently representative of the populations affected by those systems. These obligations would apply to organisations above a specified size threshold or to employers using AI systems in large-scale recruitment processes. In practice, this reform would focus on preventing discriminatory outcomes before they occur by requiring organisations to test and monitor algorithmic hiring systems for bias.

Legislative amendments would be developed by the Department of Employment and Workplace Relations (DEWR), which administers Australia’s workplace relations framework. Regulatory oversight and enforcement would be undertaken by the Fair Work Ombudsman (FWO), which already monitors employer compliance with obligations under the FWA. Implementation costs are estimated at approximately $8-12 million over four years, reflecting the need to develop algorithmic auditing standards, regulatory guidance and compliance monitoring capability. Comparable digital regulatory expansions in Australia have required similar levels of funding. For example, the Australian Government allocated $8.71 million in 2022-23 to expand the regulatory capacity of the Office of the Australian Information Commissioner (Australian Government, Budget March 2022–23). 

The advantage of this option is that it focuses on preventing discriminatory outcomes before they occur, rather than relying solely on legal complaints after harm has occurred. By requiring organisations to assess algorithmic systems for bias and data representativeness, this approach directly targets the structural causes of proxy discrimination in AI hiring systems. However, the main limitation is that implementing algorithmic audits may require specialised technical expertise, meaning employers and regulators may need access to independent technical auditors to ensure meaningful compliance.

Policy Recommendation

Option 3, to amend the FWA to introduce mandatory algorithmic audits and representative data requirements for automated decision-making systems used in recruitment, is recommended as the most effective approach to alleviate discriminatory outcomes in recruitment processes. 

Unlike option 1, which relies on individual complaints and operates only after harm has occurred, this reform requires employers to proactively assess and address discriminatory outcomes before and during the use of automated systems. Unlike option 2, which targets proxy variables that may be difficult to identify in complex systems, this option regulates both the data used and the outcomes produced. This makes it more adaptable to evolving technologies and better suited to addressing structural forms of discrimination that are not easily traceable to a single variable, particularly where biased training data may reproduce existing inequalities (Borgesius 2018, p.11).

Implementation

DEWR should lead legislative design by drafting amendments to the FWA and consulting with relevant stakeholders, including the FWO, the AHRC, unions, recruitment agencies and AI technical experts. This consultation would assist in defining the minimum content of algorithmic audits, data representativeness requirements, and appropriate compliance timelines. The FWO would then be responsible for issuing regulatory guidance, enforcement, monitoring compliance and investigating breaches. This approach builds on an existing regulatory structure and avoids the need to establish a new oversight body. 

Amendments to the FWA should apply to employers with 15 or more employees, thereby excluding small business employers as defined in s 23 of the Act. The amendments should also apply to employees and organisations using automated decision systems in high-volume recruitment processes. This threshold targets employers most likely to rely on automated systems at scale, while avoiding disproportionate compliance burdens on small businesses.

The legislation should impose two core obligations. First, employers should be required to conduct algorithmic impact audits before deploying automated hiring systems and at regular intervals thereafter. This would draw on New York City’s Local Law 144, which requires employers using automated employment decision tools to ensure the tool has undergone an independent bias audit within the previous year, with audit results publicly available. Audits should be conducted by an external auditor with relevant technical expertise, separate from both the employer and the software vendor. Audits should assess whether the system produces disproportionate adverse outcomes for protected groups, whether variables relied upon are reasonably connected to the inherent requirements for the role and whether the data used is representative of the relevant workforce of the applicant pool. Where representative data is not reasonably available, employers should be required to document the limitation and implement measures to mitigate potential bias.

Where an audit identifies a material risk of discriminatory outcomes, the employer should be required to prepare a remediation plan within 60 days. This may include modifying data inputs, removing unjustified proxy variables, requiring human review, or suspending use of the system until the risk is addressed if the system has already been deployed. Failure to complete an audit or implement remediation should allow the FWO to issue compliance notices and seek civil penalties for ongoing non-compliance. Where an audit of a deployed system identifies evidence that protected groups have already been adversely affected, the FWO should refer the matter to the AHRC for investigation under federal anti-discrimination law. 

Second, employers should be required to retain audit documentation and provide it to the FWO upon request. This should include audit reports, data sources, categories of inputs used, decision criteria, known limits, bias mitigation steps and evidence that identified risks have been addressed.

Measuring Success

An indicator of success would be that at least 95% of employers have completed compliant algorithmic audits within 24 months of implementation. A second indicator would be evidence that identified risks of bias are being actively addressed, demonstrated through documented system modifications or changes to data inputs. A third indicator would be increased transparency and regulatory oversight, reflected in annual reporting by the FWO showing high levels of compliance with audit and documentation requirements. Evaluation would be undertaken by the FWO through annual public reporting, supported by targeted compliance reviews and random audit checks of employers.

Costs

Total government cost is estimated at $8-$12 million over four years. This estimate is based on comparable expansions of regulatory oversight in relation to complex digital systems. For example, recent funding increases to the Office of the Australian Information Commissioner to strengthen enforcement under the Privacy Act 1988 involved multi-million dollar allocations to support specialist staff, technical expertise and complaint handling capacity. Additional funding of $5.5 million over two years was also provided to support investigations into major privacy incidents (Australian Government, Budget October 2022-23). 

Barriers and Risks

A key implementation barrier is the limited availability of technical expertise required to design, conduct, and interpret algorithmic audits. Both regulators and employers may face challenges in accessing personnel with the necessary data science and legal capability to assess whether automated decision-making systems produce discriminatory outcomes. This may initially limit the consistency and quality of audit results, as well as the FWO’s ability to evaluate audit documentation, identify non-compliance and determine whether enforcement action is required. However, this barrier can be addressed through targeted investment in specialist staff within the FWO and the development of clear regulatory guidance and standardised audit frameworks to support employers. 

A further barrier is the practical difficulty of assessing data representativeness and bias in complex systems, particularly where decision-making tools are developed by third-party vendors. Employers may have limited visibility over how these systems operate, which could reduce their ability to demonstrate compliance. This may be mitigated by requiring contractual transparency obligations between employers and vendors. However, this would require further legislative or regulatory intervention.  

Economically, the reform may increase compliance costs for employers, particularly large organisations reliant on automated recruitment processes. Despite these risks, the increasing use of automated decision-making in employment contexts and the potential for systemic discrimination justify a preventative regulatory response. 

References

Ajunwa, I. (2021). An auditing imperative for automated hiring systems. Harvard Journal of Law & Technology, 34(2), 621-700. https://par.nsf.gov/servlets/purl/10267581 

Australian Bureau of Statistics. (2026). Labour force, Australia, May 2026. https://www.abs.gov.au/statistics/labour/employment-and-unemployment/labour-force-australia/latest-release 

Australian Human Rights Commission. (2021). Human Rights and technology: Final Report. https://humanrights.gov.au/__data/assets/file/0023/46931/Ahrc_rightstech_2021_final_report_10.pdf 

Australian Human Rights Commission (2020), Using artificial intelligence to make decisions: Addressing the problem of algorithmic bias’ Technical Paper. 

Australian Institute of Health and Welfare (2022) People with disability in Australia. https://www.aihw.gov.au/reports/disability/people-with-disability-in-australia-2022-in-brief/contents/about 

Attorney-General’s Department. (n.d.). Australia’s anti-discrimination law. Australian Government. https://www.ag.gov.au/rights-and-protections/human-rights-and-anti-discrimination/australias-anti-discrimination-law#:~:text=Complaints%20of%20unlawful%20discrimination,Australian%20Human%20Rights%20Commission%20website.

Cofone, I.N. (2019). Algorithmic discrimination is an information problem. Hastings Law Journal, 70(6), 1389-1444. https://repository.uclawsf.edu/cgi/viewcontent.cgi?article=3867&context=hastings_law_journal 

Commonwealth of Australia. (2022). Budget measures: Budget Paper No. 2, 2022-23. https://archive.budget.gov.au/2022-23/bp2/download/bp2_2022-23.pdf 

Commonwealth of Australia. (2022, October 25). Budget measures: Budget Paper No. 2, October 2022-23). https://archive.budget.gov.au/2022-23-october/bp2/download/bp2_2022-23.pdf 

Lai, B. (2022). Privacy Legislation Amendment (Enforcement and Other Measures) Bill 2022 (Bills Digest No. 30, 2022-23). Parliament of Australia. https://www.aph.gov.au/Parliamentary_Business/Bills_Legislation/bd/bd2223a/23bd030?utm_source=chatgpt.com 

New York Department of Consumer and Worker Protection. (n.d.). Automated employment decision tools (AEDT). https://www.nyc.gov/site/dca/about/automated-employment-decision-tools.page

Menodza, B., Szollosi, M., & Leiman, T. (2021). Automated decision making and Australian discrimination law. Journal for the Australian and New Zealand Societies for Computers and the Law, 93, 10-15. https://www8.austlii.edu.au/cgi-bin/viewdoc/au/journals/ANZCompuLawJl/2021/4.html 

National Artificial Intelligence Centre. (2025). Australian Responsible AI Index 2025: Final Report. https://www.fifthquadrant.com.au/content/uploads/Australian-Responsible-AI-Index-2025_Full-report.pdf

Sheard, N. (2025). AI may be exposing jobseekers to discrimination. Here’s how we could better protect them. The University of Melbourne. https://findanexpert.unimelb.edu.au/news/131271-ai-may-be-exposing-jobseekers-to-discrimination.-here%E2%80%99s-how-we-could-better-protect-them 

Sheard, N. (2025). Algorithm-facilitated discrimination: A socio-legal study of the use by employers of artificial intelligence hiring systems. Journal of Law and Society, 52(2), 269-291. 

Sheard, N. (2022). Employment discrimination by algorithm: Can anyone be held accountable? UNSW Law Journal, 45(2), 617-648. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3866363 

Sony et al. (2025). Bias in AI-driven HRM systems: Investigating discrimination risks embedded in AI recruitment tools and HR analytics. Social Sciences & Humanities Open, 12, Article 102082. https://www.sciencedirect.com/science/article/pii/S2590291125008113?fr=RR-2&ref=pdf_download&rr=9e7e04efb8e257dd 

Yanisky-Ravid, S and Hallisey, S.K. (2019). “Equality and privacy by design:” A new model of artificial intelligence data transparency via auditing, certification, and safe harbor regimes. Fordham Urban Law Journal, 46(2), 428-486. https://ir.lawnet.fordham.edu/ulj/vol46/iss2/5/ 

Zuiderveen Borgesius, F.J. (2018). Discrimination, artificial intelligence, and algorithmic decision-making. Council of Europe. https://rm.coe.int/discrimination-artificial-intelligence-and-algorithmic-decision-making/1680925d73 

The views and opinions expressed by Global Voices Fellows do not necessarily reflect those of the organisation or its staff.